Splunk Search

How to search for results that are in result A, but not result B?

tdewitt_atl_rea
New Member

I have 2 logs: an error log and a success log. When an item fails (error log), it is retried. I would like to filter out the items that succeed on retry and only display items that are still failing - In other words, items that only appear in the error log.

Sample search:

index=fail_log "Processing failed" | dedup event_id | transaction event_id | rex "\"itemId\":(?P<itemIdInt>\d+)" | fields itemIdInt | search NOT [search index=success_log Order processed successfully | eval itemIdInt=substr(itemId, 4) | fields itemIdInt ] | stats count

The reason for the substring is that in the success log, the items show up as ID- and in the fail log, they are just , so we need to just pull out the actual digits.

This seems to work for most things. We have 20,000+ successes and on the order of 10-20 failures. However, only 1 or 2 of those failures have not been successful, and a simple search of:

index=success_log Order processed successfully | eval itemIdInt=substr(itemId, 4) | fields itemIdInt | search itemIdInt=<itemId_from_fail_log>

will return results, indicating that itemId has succeeded on rerun.

Is there a more formal way of removing items from result A (failures) that appear in result B (successes)

0 Karma

sundareshr
Legend

Not sure you need the transaction command. Try this

index=fail_log "Processing failed" | dedup event_id | rex "\"itemId\":(?P<itemIdInt>\d+)" | fields itemIdInt | search NOT [search index=success_log Order processed successfully | eval itemIdInt=substr(itemId, 4) | fields itemIdInt ] | stats count
0 Karma

tdewitt_atl_rea
New Member

That didn't seem to affect the results. Just curious, why would removing the transaction command potentially work?

0 Karma

Richfez
SplunkTrust
SplunkTrust

Would it be just as useful to only include items whose final state is "failed?" E.g. the last time they ran they were in state failed?

0 Karma

tdewitt_atl_rea
New Member

How would I go about that, Rich? It sounds like it could work!

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...