Splunk Search

How do I correlate email events when the key-id is not defined in all events?

Log_wrangler
Builder

So I have correlated email events before where there was a UID defined as a field for all transactions of a unique email session. For example, the event containing "subject" contained a UID=xyz123 and the event containing "sender" contained a UID=xyz123, and the event containing "recipient" contained a UID=xyz123, etc...

Now I am faced with transaction-events where the event containing "subject" has a UID=abc987, but the "to" and "from" events only have "abc987" and no field name...

I can regex the UID value out of the "to" and "from" events, but I have not been successful stitching the events together to create the complete email session...

For example, If I run this query below I can get the unique ids which will be contained in all events related to a unique email session... (by session I mean Subject, To, From, etc...)

index=mail sourcetype=mail | rex field=_raw "sendmail+\S+\s(?<stitcher>[[:alnum:]]+)"|fillnull value="null"| table stitcher | WHERE stitcher!="null" 

here are some sample results

stitcher
w9FD0v3f024155
w9CCWGaF023575
w9CCAwjU026498
w9AEM7sO030350
w9ADp31g031379
w993gkLc016485
w993gjU0016459
w993UuOr000878
w9CDhH42016767
w9CDV93a026891
w9CDVAv6018597

If I search with each of theses UIDs 1 by 1, I would only get the events related to a specific email.

So I tried a number of subsearches but I am having no luck...I need to feed the list back into a search and be able to list out the subject, to, and from, by UID...

Looking for advice.
Thank you

Tags (3)
0 Karma
1 Solution

jlelli
Path Finder

If you have common unique values for different fields, would not be easier to rename them all and then apply a transaction command on the renamed field? Something like:

| rename subject as UID | rex "sendmail+\S+\s(?<UID>[[:alnum:]]+)" | transaction UID 

If the time and format of the events are always the same you can add further options to the transaction, something like
| transaction UID startswith=subject endswith=to and/or | transaction UID maxevents=3

View solution in original post

0 Karma

jlelli
Path Finder

If you have common unique values for different fields, would not be easier to rename them all and then apply a transaction command on the renamed field? Something like:

| rename subject as UID | rex "sendmail+\S+\s(?<UID>[[:alnum:]]+)" | transaction UID 

If the time and format of the events are always the same you can add further options to the transaction, something like
| transaction UID startswith=subject endswith=to and/or | transaction UID maxevents=3

0 Karma

Log_wrangler
Builder

that is an idea I will try.... thank you

0 Karma

Log_wrangler
Builder

I had to use a combo of regex and transaction. Thx

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...