I have a light weight forwarder pointing two indexers . I get a batch data everyday in a single file . The file size is 28 GB on an average .
All the events in that file gets ingested in one single indexer . I noticed splunk is not able to distribute the events within a single file . It distributes events to indexers based on file level .So in case I get 10- 11 file on some day , one single file gets ingested to one single indexer and other file on other indexer. Is there any solution that can help me distribute the data on event level.
I am using Splunk 6.1.7
I suspect what you are running into is a forwarder will send data to an indexer for 30s OR until it hits an end of file and then switch. If these are single files as part of a batch job I'm not surprised you are seeing the behavior you are describing. I'm not sure there is a good way to adjust that. You could check out the forceTimebasedAutoLB setting but I'm not sure this is the best use case for that setting.
thanks Runals , Seems like there is not concrete solution to this . I will have to break my files to multiple smaller files .
I do not want to change the forceTimebasedAutoLB setting , as it might truncate an event itself.
