Splunk Search

Most of searches are getting deferred

Rukmani_Splunk
Path Finder

Hi all,
We are using SoS app for monitoring our schedules. We are working on reducing the schedules which are being skipped . But what is case about the deferred searches ? Its keep on increasing ? how to reduce them.
thoughts pls

Tags (1)

yannK
Splunk Employee
Splunk Employee

A deferred search is a search that couldn't be executed right now, because of the system or role search concurrency limit. Therefore they are executed a few seconds later. This is an expected behavior.
By example is you have a dashboard with 10 searches, but a limit of 6 concurrent searches, some panels will load after the first ones completed.

If you look at the audit logs, you can find how long they were deferred before being executed.
If a search is deferred too long, it will finally be skipped: skipped searches

The root cause are usually caused by :
- too many searches : you have too many searches (or heavy dashboard)
- non optimized scheduled searches taking long to run and overlapping.
- hardware limit : the indexers and search-head have not enough cpu core to handle high search concurrency. (check limits.conf), on 6.0 the formula for historical search concurrency = 6+ 1* (number of cores)

(hint, disable the deployment monitor app if you have it)

Rukmani_Splunk
Path Finder

Thanks a lot

0 Karma

MuS
Legend

more an advice then an answer, check out this answer on search scheduling http://answers.splunk.com/answers/33717/scheduled-searches-for-summary-index-does-not-run-no-skipped...

Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...