Splunk Search

How to aggregate several similar fields into one and tabulate top values

shantu
Explorer

I have created several search-time field extractions to filter out Credit Card numbers from our logs:

\s+(?<CCVisaNumber>4[0-9]{12}(?:[0-9]{3})?)
\s+(?<CCMasterCardNumber>5[1-5][0-9]{14})
\s+(?<CCJCBNumber>(?:2131|1800|35\d{3})\d{11})
\s+(?<CCAmexNumber>3[47][0-9]{13})
\s+(?<CCDinersNumber>3(?:0[0-5]|[68][0-9])[0-9]{11})
\s+(?<CCDiscoverNumber>6(?:011|5[0-9]{2})[0-9]{12})

I would ideally like to aggregate all of these in one field "CreditCardNumber" so that I can then use:

host=stagexx | top CreditCardNumber

(Essentially, I want to display them in descending order of frequency)

Is there any way to do that? The rex command doesn't support this sort of OR use case (using | with quotes et al proved futile).

Instead, could I perhaps aggregate these into one field while trying to create a table? Something like assigning all of them to one type of sorts (such as CreditCardNumber) and then displaying all values in descending order of frequency?

Tags (4)
1 Solution

aweitzman
Motivator

You want to use the coalesce function of the eval command:

...all your extractions... 
| eval CCNumber=coalesce(CCVisaNumber,CCMasterCardNumber,CCJCBNumber,CCAmexNumber,CCDinersNumber,CCDiscoverNumber)
| top CCNumber

(This will only work if you only have one credit card number per event, though. If you have more than one per event, you'll need fancier footwork than this.)

View solution in original post

aweitzman
Motivator

You want to use the coalesce function of the eval command:

...all your extractions... 
| eval CCNumber=coalesce(CCVisaNumber,CCMasterCardNumber,CCJCBNumber,CCAmexNumber,CCDinersNumber,CCDiscoverNumber)
| top CCNumber

(This will only work if you only have one credit card number per event, though. If you have more than one per event, you'll need fancier footwork than this.)

shantu
Explorer

Solid.

Yeah - at most one credit card number per event, so this works quite well. Thanks for the help!

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...