Splunk Search

Why is my search skipping?

kteng2024
Path Finder

Hi,

index=_internal source=*metrics.log group=searchscheduler | timechart partial=false span=10m sum(dispatched) sum(skipped) 

The above is the search i am using to find out number of skipped searches . But my question is , how to find the reasons why this search is skipping and how to stop them skipping like any configuration change?

1 Solution

woodcock
Esteemed Legend

Run the MC Health Checks to verify your compliance to HW Standards. By far the most common reason to skip a search is that the previous run has not completed so you will enter an infinite loop if you continue to (try to) run it again and again. There are 2 main causes. If the search is too aggressive, try scaling back the earliest/latest and also the frequency of running it. Do anything you can to optimize the search. The other cause is overwhelmed indexing tier. The Health Check will tell you if your HW is not according to minimum requirements. It really should be obvious when you need more Indexers and the symptoms like you are seeing is a very common one, as well as slow ad-hoc seraches that sometimes cannot complete and timeout.

View solution in original post

pradeepkumarg
Influencer

If you search for scheduler logs, you can find a field reason explaining why the search was skipped.

index=_internal sourcetype=scheduler status=skipped

Rsolution depends on what the reason is. could be capacity or some user reaching his allocated max disk space, etc..,

woodcock
Esteemed Legend

Run the MC Health Checks to verify your compliance to HW Standards. By far the most common reason to skip a search is that the previous run has not completed so you will enter an infinite loop if you continue to (try to) run it again and again. There are 2 main causes. If the search is too aggressive, try scaling back the earliest/latest and also the frequency of running it. Do anything you can to optimize the search. The other cause is overwhelmed indexing tier. The Health Check will tell you if your HW is not according to minimum requirements. It really should be obvious when you need more Indexers and the symptoms like you are seeing is a very common one, as well as slow ad-hoc seraches that sometimes cannot complete and timeout.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...