I've got a medium-sized (50MB) CSV lookup file with two columns (email address and server name) that I want to use. I tried a straight upload and managed to put down our Splunk instance because replication failed and blocked all searches. Can I dribble the file in 100K lines at a time using outputlookup append=t
? Or does the replication just take the whole lookup bundle and try and replicate everything?
Please note: I do not have access to the file system; whatever the solution is, I have to be able to do it from Splunk Web.
Thanks!
The problem isn’t going to be fixed by “dribbling in” the csv one piece at a time.
Depending on the version of splunk you have limits.conf on the search heads and indexers will have a default setting of 800MB or 2GB for search bundle replication (I think it’s 2GB since 6.6).
You’re going over that limit by x MEgabytes when you upload the csv... and causing the issue.
There are several solutions documented for this.
Hi There,
That seems a tad bit more than a medium size csv you have there, how many records have you got within it? Have you looked into utilising a KV store instead?