When I put below,
sourcetype="splunk_page_request" NOT [| inputlookup nmc_crawlers | fields ip_address]
I got a message below,
[subsearch]: Subsearch produced 12959 results, truncating to maxout 10000.
How can I solve this?
By default the subsearch result set limit is set to 10000. You can increase it in the limits.conf file. But it's not recommended to go beyond 10500. I have not tried to modify it to greater value but if its not working then need to think of something else. May be you can use Join which has a greater sub search value.
maxout =
* Maximum number of results to return from a subsearch.
* This value cannot be greater than or equal to 10500.
* Defaults to 10000.
Please refer:
http://docs.splunk.com/Documentation/Splunk/5.0.3/Admin/Limitsconf
The solution in the comments is valid, but you can also try:
sourcetype="splunk_page_request" NOT [| inputlookup nmc_crawlers | dedup ip_address |fields ip_address]
This will try to deduplicate the entries with the same IP in the nmc_crawlers lookup, hence reducing the number of returned address.
By default the subsearch result set limit is set to 10000. You can increase it in the limits.conf file. But it's not recommended to go beyond 10500. I have not tried to modify it to greater value but if its not working then need to think of something else. May be you can use Join which has a greater sub search value.
maxout =
* Maximum number of results to return from a subsearch.
* This value cannot be greater than or equal to 10500.
* Defaults to 10000.
Please refer:
http://docs.splunk.com/Documentation/Splunk/5.0.3/Admin/Limitsconf
Thank you so much!! I changed maxout to 13000 and it worked.