Splunk Search

How to search for results that are in result A, but not result B?

tdewitt_atl_rea
New Member

I have 2 logs: an error log and a success log. When an item fails (error log), it is retried. I would like to filter out the items that succeed on retry and only display items that are still failing - In other words, items that only appear in the error log.

Sample search:

index=fail_log "Processing failed" | dedup event_id | transaction event_id | rex "\"itemId\":(?P<itemIdInt>\d+)" | fields itemIdInt | search NOT [search index=success_log Order processed successfully | eval itemIdInt=substr(itemId, 4) | fields itemIdInt ] | stats count

The reason for the substring is that in the success log, the items show up as ID- and in the fail log, they are just , so we need to just pull out the actual digits.

This seems to work for most things. We have 20,000+ successes and on the order of 10-20 failures. However, only 1 or 2 of those failures have not been successful, and a simple search of:

index=success_log Order processed successfully | eval itemIdInt=substr(itemId, 4) | fields itemIdInt | search itemIdInt=<itemId_from_fail_log>

will return results, indicating that itemId has succeeded on rerun.

Is there a more formal way of removing items from result A (failures) that appear in result B (successes)

0 Karma

sundareshr
Legend

Not sure you need the transaction command. Try this

index=fail_log "Processing failed" | dedup event_id | rex "\"itemId\":(?P<itemIdInt>\d+)" | fields itemIdInt | search NOT [search index=success_log Order processed successfully | eval itemIdInt=substr(itemId, 4) | fields itemIdInt ] | stats count
0 Karma

tdewitt_atl_rea
New Member

That didn't seem to affect the results. Just curious, why would removing the transaction command potentially work?

0 Karma

Richfez
SplunkTrust
SplunkTrust

Would it be just as useful to only include items whose final state is "failed?" E.g. the last time they ran they were in state failed?

0 Karma

tdewitt_atl_rea
New Member

How would I go about that, Rich? It sounds like it could work!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...