I am working with Terabytes of data and running into a brick wall with the subsearch limit. The search that I am running is below
sourcetype=slapd_log host=server-0* "BIND" [search sourcetype=slapd_log host=server-0* ou=orgunits OR ou=orgUnits | fields host,conn ] | stats count by uid
Now to explain what I am trying to do. I am including an example of one event below
Oct 11 13:55:04 server-01 slapd[131027]: conn=2892910 op=0 BIND dn="uid=XXXXXXX,ou=XXXXXXXX,dc=XXXXX,dc=XXXXXX,dc=XXX" mech=SIMPLE ssf=0
I have scrubbed some sensitive information and replaced with Xs and fake server names from the search and the event.
I am looking for events where the field ou = orgunits or orgUnits. Once I found those events I need the conn and host field from that event. Once I have the conn and host information I am looking for events where the conn and host match along with having the word "BIND" in the event. Because of the way the system is designed people connect into it with a bind and then can run various queries. I only care about one type of query, but the data does not tell me who ran the query in the same event.
Now that I have these events I want to count the uid field by uid.
The only way I don't hit the 10k limit is if i run the search for less than a 3 hour time period. We have Terabytes of data. We want to get the list of uids for as far back as possible. So with splunk that will be 90 days. Does anyone have any ideas on how to accomplish this besides taking the data in 3 hours clunks and combining it manually?
