Hello,
I keep getting warning messages that my dispatch directory is full (5GB) even though the dispatch dir size is less than 1 GB. And also, my queries stop running, hence I have to clean up the dispatch dir to make Splunk run again.
Kindly advise.
It is not how big the volume is, it is how much free space is left. Try this command:
df /opt/splunk/var/run/splunk/dispatch/
It will show you how big the hosting volume is, how much space is used and how much is available. The Available will be > 500M if you are getting that warning/symptoms (unless you have changed the default).
Ah of course you are right @richgalloway ! The limit can be changed here in limits.conf if you cant make more space available dispatch_dir_warning_size = <int>
https://docs.splunk.com/Documentation/Splunk/7.2.3/Admin/Limitsconf
The 5GB warning is not about size, it's about free space. It's possible for the dispatch directory to be completely empty and still get this warning because the disk doesn't have at least 5GB available.
The fix is to free up files elsewhere on the disk, not just the dispatch directory.
Hi @kamal_jagga
Interesting problem
Good luck with your hunt
Thanks for the reply. I had already checked for these things but wasn't of any help.