What does this splunkd.log event mean? 07-19-2013 04:19:02.641 -0400 INFO Archiver - Archiving large_file=/opt/splunk/etc/apps/MAXMIND.new/bin/city.dat of size_in_bytes=54745499 (exceeding concerning_threshold=52428800)

[mctester]

I'm seeing a repeated pattern of events in splunkd.log, relating to several .dat files in the MAXMIND app. What is the event trying to tell me?

07-19-2013 04:16:57.956 -0400 INFO  Archiver - Archiving large_file=/opt/splunk/etc/apps/MAXMIND.new/bin/org.dat of size_in_bytes=125619791 (exceeding concerning_threshold=52428800)
07-19-2013 04:16:58.168 -0400 INFO  Archiver - Archiving large_file=/opt/splunk/etc/apps/MAXMIND.new/bin/city.dat of size_in_bytes=54745499 (exceeding concerning_threshold=52428800)
07-19-2013 04:18:00.186 -0400 INFO  Archiver - Archiving large_file=/opt/splunk/etc/apps/MAXMIND.new/bin/org.dat of size_in_bytes=125619791 (exceeding concerning_threshold=52428800)
07-19-2013 04:18:00.399 -0400 INFO  Archiver - Archiving large_file=/opt/splunk/etc/apps/MAXMIND.new/bin/city.dat of size_in_bytes=54745499 (exceeding concerning_threshold=52428800)
07-19-2013 04:19:02.420 -0400 INFO  Archiver - Archiving large_file=/opt/splunk/etc/apps/MAXMIND.new/bin/org.dat of size_in_bytes=125619791 (exceeding concerning_threshold=52428800)
07-19-2013 04:19:02.641 -0400 INFO  Archiver - Archiving large_file=/opt/splunk/etc/apps/MAXMIND.new/bin/city.dat of size_in_bytes=54745499 (exceeding concerning_threshold=52428800)

I see it so often in the logs that I'm concerned that I have a problem, but there's nothing in the message to confirm a) that there actually IS a problem, and b) what to do about it

[Mick · Accepted solution]

This event happens in the context of distributed search. It is coming from bundle replication, which is attempting to tar up all of your app files to push the search bundle to your indexers. In order to make this manageable, Splunk has a default limit of 50MB, which can be tuned with the following setting in distsearch.conf, in the [replicationSettings] stanza -

concerningReplicatedFileSize =
* Any individual file within a bundle that is larger than this value (in MB) will trigger a splunkd.log message.
* Where possible, avoid replicating such files, e.g. by customizing your blacklists.
* Defaults to: 50

However, the better solution here would be to simply blacklist these, and any other large files that are not necessary for searching on the indexers. Read the information here about controlling the size of your replicated bundles - http://docs.splunk.com/Documentation/Splunk/5.0.4/Deploy/Configuredistributedsearch#Limit_the_knowle...

And then for any changes you want to make to white & blacklist settings, you can edit the distsearch.conf file - http://docs.splunk.com/Documentation/Splunk/5.0.4/admin/Distsearchconf

[bpaul_splunk]

This 50 MB threshold is in older versions of Splunk. Starting with Splunk 6.6.0, the threshold value has been increased to 500 MB.

concerningReplicatedFileSize =
* Any individual file within a bundle that is larger than this value (in MB)
will trigger a splunkd.log message.
* Where possible, avoid replicating such files, e.g. by customizing your blacklists.
* Defaults to: 500

[jrodman]

To clarify, it's not really a limit that changes behavior, it's just a point at which we complain.

Splunk thinks that you don't want to be search-replicating 50MB files because it will be slow and cause some ram to be used pointlessly. These warnings are just to tell you that you have giant files that you probably want to handle some other way, as mick says, or with mounted bundles.

We also partially work around this in 5.0 and later by trying not to send files that have not changed.