How to search stats count of 60,000 Session_ID's by Status without typing each Session_ID value?

[rberkheimer]

Good Afternoon,

I have a simple search. Normally this would be:

sourcetype=j_s_i Session_ID=000002b89784b98e91bd OR Session_ID=1c1b68e349f3b98f3570 OR Sesssion_ID=1c1b68e349f3b98f4d3f | stats count by Status

However, in this case I have 60K Session_ID's that I need to input into a single search query.

I have looked at inputcsv, lookup, inputlookup and outputlookup, and to be honest I just dont get it.

Some guidance on how best to proceed please?

[Ayn · Accepted solution]

Your best option (that comes to mind for me at least) would probably to use this CSV as a lookup. Past 8000 search terms or so you're not getting a performance boost out of adding more terms anyway (and there's even a hard limit of 10500 results that you can emit from a subsearch), so you might as well run a query against all data and then filter it using your lookup.

Your lookup would be something like:

Session_ID,exists
000002b89784b98e91bd,1
1c1b68e349f3b98f3570,1
1c1b68e349f3b98f4d3f,1

And then your search would look something like this:

sourcetype=j_s_i | lookup session_ids Session_ID OUTPUT exists | search exists=1 | stats count by Status

[ppablo]

Hi @rberkheimer

Do you have exactly 60,000 Session_ID's in your data, or are there more than that and you're only looking for data on those 60,000?

[rberkheimer]

Hi!

It is actually 58190 Session_IDs.

In my sourcetype=j_s_i, there are many details on each Session_ID, one of those being Status=Success or Status="some error code".

This log file for one day has 117K Session_IDs in them, and I only want to know the "| stats count by Status" for these 58190.