Hi Experts
I am facing a strange issue with Splunk DB Connect. It is missing the intervals of scheduled inputs. The pattern is random and anytime it misses 1 or 2 scheduled interval. The interval set is 300s. This is happening for all the inputs and also for the internal splunk-system-user api check which dbconnect does. The strange thing is that the state value is getting updated. When it runs after missing the interval it does not include the events which it should have captured in the previous(missed) schedule. There are no errors in dbx or splunkd logs also.
Please see dbx logs
3/12/15
2:52:34.453 PM
2015-03-12 14:52:34.453 monsch3:INFO:Scheduler - Execution of input=[xxx] finished in duration=151 ms with resultCount=2001 success=true continueMonitoring=true
host = zzz source = /opt/splunk/var/log/splunk/dbx.log sourcetype = dbx_debug
3/12/15
2:42:31.414 PM
2015-03-12 14:42:31.414 monsch3:INFO:Scheduler - Execution of input=[xxx] finished in duration=156 ms with resultCount=2730 success=true continueMonitoring=true
host = zzz source = /opt/splunk/var/log/splunk/dbx.log sourcetype = dbx_debug
3/12/15
2:37:29.855 PM
2015-03-12 14:37:29.855 monsch3:INFO:Scheduler - Execution of input=[xxx] finished in duration=147 ms with resultCount=1988 success=true continueMonitoring=true
host =zzz source = /opt/splunk/var/log/splunk/dbx.log sourcetype = dbx_debug
