Hello,
We are running 4.2.5-113966.
Scanning splunkd.log, I see we are getting LOTS of errors (~2600 yesterday) of the form (always a pair of errors):
02-24-2012 08:01:54.076 -0600 ERROR StreamedSearch - preparing to stream search, failed to mkdir /usr2/splunkshare1/splunk-ui/var/run/splunk/dispatch/remote_<name of the other Splunk Search Head>-splnkcpo_subsearch_subsearch_scheduler_ nobody SplunkDeploymentMonitor_RE0gc291cmNldHlwZXMgdG9vIGxpdHRsZSBkYXRh_at_1330092000_8faa392bb73975e
f_1330092112.2_1330092112.3<name of THIS Splunk Search Head>: File name too long
02-24-2012 08:01:54.077 -0600 ERROR SearchResults - Unable to open output file: path=/usr2/splunkshare1/splunk-ui/var/run/splunk/dispatch/remote_<name of the other Splunk Search Head>-splnkcpo_subsearch_subsearch_scheduler_ nobody SplunkDeploymentMonitor_RE0gc291cmNldHlwZXMgdG9vIGxpdHRsZSBkYXRh_at_1330092000_8faa392bb73975ef_1330092112
.2_1330092112.3<name of THIS Splunk Search Head>-splnkcpo/info.csv.tmp error=File name too long
It would seem that there is some mechanism / algorithm that is generating these long file names -- longer than Linux can handle.
Any ideas??
Thx!
Long Path Tool is useful here
You have to checkout long path tool, it's an automated software for this type of errors
Long Path Tool helped me in this situation. http://PathTooDeep.com
Long Path Tool helped me in this situation. http://PathTooDeep.com
Hi,
It smells like a http restriction, instead of OS filesystem restriction. Splunk use REST API to communicate with splunkd (default 8089/tcp).
The dispatch sid name is a little strange;
But, this should not happen in 4.2.5 because the sid of dispatched search should not use <name of THIS Splunk Search Head>-splunkcpo
Are you using the same version 4.2.5 for both Splunk instance?
Maybe you should file a Support case to ask to investigate more about the issue.