Are there any limitations of search string length or of count of conditions, or it is specific to cidrmatch only?

[kamlesh_vaghela]

Hello Team,

I facing an issue when executing the search on the dashboard.

Search Logic:
I have a Network KV Store lookup (My_All_Network_Lookup) with a large set of machine data with IP address column. In my dashboard, I'm fetching only those data whose cidrmatch with the particular range. Below is sample search which is working properly.

| inputlookup My_All_Network_Lookup 
| where (
                      cidrmatch("0.0.0.0/24", asset) OR 
                       cidrmatch("0.1.140.0/24", asset)
) 

In my dashboard's search, where condition is designed dynamically. So it will increase the count of conditions.
I'm getting an error "Job terminated unexpectedly" when the number of the cidrmatch condition is more than 100,000.

Like:

| inputlookup My_All_Network_Lookup 
| where (
                      cidrmatch("0.0.0.0/24", asset) OR 
                      .
                      . 
                      . 
                      . <<100,00 cidrmatch conditions >> 
                      . 
                      . 
                      . 
                      . 
                      . 
                       cidrmatch("0.1.140.0/24", asset)
)

Well, when I the final search is designed with 299954 conditions and length of the search string is around in millions.

Note : Currently I'm working with eventgen data and I'm doing load & performance testing of my app

So here my questions are:
Is that any limitation of search string length?
Is that any limitation of count of conditions Or it is specific to cidrmatch only?

Thanks

[kamlesh_vaghela · Accepted solution]

Hi Team,

To overcome with "Job terminated unexpectedly" error , i have changed my approach.

My Old approach was these:

1) Create a search which generates a cidrmatch conditions separated by OR operator. Like, cidrmatch("0.0.0.0/24", ip) OR cidrmatch("0.1.140.0/24", ip) OR ..., so on. So subnet generation search was different.

like.

MY_SUBNET_SEARCH | eval subnet_condition = " cidrmatch(\".subnet.".0/24\", ip)" | table subnet_condition //this is a sample search.

2) Put subnet_conditionthe in actual search.

like,

MY_DASHBOARD_SEARCH | where $subnet_condition$ // this search is used in dashboard panels and drilldown logics

executed search like,

MY_DASHBOARD_SEARCH | where cidrmatch("0.0.0.0/24", ip) OR cidrmatch("0.1.140.0/24", ip) //

Well, here is the problem. When there are more than 5000 cidrmatch conditions then Splunk throws an error "Job terminated unexpectedly". This 5000 conditions count I have figured my self I unable to find documented page.

So I decided to go with another approach and execute both search at a time using map command.

New Approach:
1) Create a subnet search same like 1st in above approach.
2) Use map in above search and pass 2nd search from old approach in search parameter of Map command.

The search looks like.,

MY_SUBNET_SEARCH | eval subnet = subnet.".0/24" | dedup subnet | fields subnets | map search=" MY_DASHBOARD_SEARCH ip=$subnet$"

In my case this scenario works fine.

Thanks