Splunk Search

How to overcome the subsearch limit of 10500?

aggie4life
New Member

I am working with Terabytes of data and running into a brick wall with the subsearch limit. The search that I am running is below

sourcetype=slapd_log host=server-0* "BIND"  [search sourcetype=slapd_log host=server-0* ou=orgunits OR ou=orgUnits | fields host,conn ] | stats  count by uid

Now to explain what I am trying to do. I am including an example of one event below

Oct 11 13:55:04 server-01 slapd[131027]: conn=2892910 op=0 BIND dn="uid=XXXXXXX,ou=XXXXXXXX,dc=XXXXX,dc=XXXXXX,dc=XXX" mech=SIMPLE ssf=0

I have scrubbed some sensitive information and replaced with Xs and fake server names from the search and the event.

I am looking for events where the field ou = orgunits or orgUnits. Once I found those events I need the conn and host field from that event. Once I have the conn and host information I am looking for events where the conn and host match along with having the word "BIND" in the event. Because of the way the system is designed people connect into it with a bind and then can run various queries. I only care about one type of query, but the data does not tell me who ran the query in the same event.

Now that I have these events I want to count the uid field by uid.

The only way I don't hit the 10k limit is if i run the search for less than a 3 hour time period. We have Terabytes of data. We want to get the list of uids for as far back as possible. So with splunk that will be 90 days. Does anyone have any ideas on how to accomplish this besides taking the data in 3 hours clunks and combining it manually?

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi aggie4life,

The easiest way to over come it is not to use a sub search at all.
There are other options like lookups or summary indexes and you can always check if a simple stats will do it as well.

Have a look at this answer https://answers.splunk.com/answers/129424/how-to-compare-fields-over-multiple-sourcetypes-without-jo... to learn more about it or have look at the March 2016 session of the virtual .conf you can find here http://wiki.splunk.com/Virtual_.conf.

Hope this helps ...

cheers, MuS

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...