Splunk Search

Using transactions with IronPort's potentially infinite MID rewrites

rharrisssi
Path Finder

I've seen quite a few posts about IronPort/Cisco ESA mail logs and how folks have put them together with transaction. However I see one flaw, they don't have a way to include a rewritten MID's rewritten MID. Conceptually I'm having a hard time figuring out how to approach this, any ideas would be greatly appreciated.

MID 70101307 rewritten to MID 70101309 by url-reputation-replace-action filter 'Malicious_URL'
Message finished MID 70101307 done
MID 70101309 rewritten to MID 70101311 by url-threat-protection filter 'Threat Protection'
Message finished MID 70101309 done
Message finished MID 70101311 done

As one can see there are a total of 3 MIDs here.
1. 70101307 which is the first, and root, event
2. 70101307 is rewritten to 70101309
3. 70101309 is rewritten to 70101311

These are all the same message, and I would like to combine them all into a single event. This scenario happens more often than one may assume. Unfortunately folks who are relying on | transaction mid will miss all of the rewritten MID's actions; there is a ton of juicy data there. Using a lookup populated immediately before with maps of MID to rewritten MID is another great idea, and that's how I'm successfully combining the first and second MIDs, but not sure how to capture rewritten MID's rewritten MID. The following is a light example of what I'm doing today, reduced down to the SPL that matters.

index=ironport
| transaction mid
| table _time,host,mid,rewrite_mid
| outputlookup append=false bufferlookup

index=ironport
| lookup local=true bufferlookup host,mid OUTPUT mid AS buffer_mid,rewrite_mid AS buffer_rewrite_mid
| lookup local=true bufferlookup host,mid AS rewrite_mid OUTPUTNEW mid AS buffer_mid,rewrite_mid AS rewrite_mid
| eval anchor=if(!isnull(buffer_mid) and buffer_mid>0,buffer_mid.":".buffer_rewrite_mid,mid)
| transaction anchor
0 Karma

landen99
Motivator

The proper answer is:

| sort 0 _time | eval mids=mvappend(mid,mid2,rewrite_mid) | transaction mids

other options on transaction may be added as desired, but aren't necessary if the search time frames are short.

rharrisssi
Path Finder

I am not able to explain it yet, but changing it up a bit gets me the results I'm interested in.

index=ironport MID earliest=-0d@d+8m latest=-0d@d+10m
| eval mids=mvappend(mid,rewrite_mid)
| transaction mids maxspan=600s maxpause=600s maxevents=-1 maxopentxn=-1 maxopenevents=-1
| transaction mids maxspan=600s maxpause=600s maxevents=-1 maxopentxn=-1 maxopenevents=-1

Yes the transaction command is duplicated.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...