Is there a search for indexes/sources that aren't being used...

[a212830]

Is there a way to determine what sources and/or sourcetypes AREN'T being searched? If data is coming into Splunk and nobody is really looking at that data, then I don't need to keep bringing it in. I just want to find a way to determine this.

[NullZero]

Regarding indexes which are not being used, a simple adaptation of the example already posted yields a count of searches by index. I did this over 30 days as a quick example:

index=_audit action=search user!=splunk-system-user search=* "index"
| rex "search index=\"(?<unused_index_search>\w+)"
| stats count by unused_index_search
| sort - count

This can then be used to compare to a list of indexes and simply lookup count. An extension could be to use a lookup in conjunction but this is simple.

As mentioned already, always check with the client and end users before removing anything.

[javiergn]

This is what I would do for sourcetypes:

index=_audit action=search user=* search=* "sourcetype"
| fields user, search
| rex field=search max_match=0 "sourcetype\s*=\s*(?<st_used>[\w\d_]+)"
| stats count by user, st_used
| sort limit=0 st_used

You can probably use something similar for indexes or sources.

Thanks,
J

[yannK]

It will not really show you all the data that are being searches implicitly, but will give you an idea of what is searched the most explicitly.

• explicit searches :

  sourcetype=A
  NOT sourcetype=B

• implicit searches :

  sourcetype=*
  index=B *