Splunk Search

How to dedup a combination of two fields and get the count of unique values per host?

mvasquez2
New Member

We have devices that generate thousands of a particular entry. I created a daily search to summarize. I combined the src_int and dest_int into a single field labeled interfaces. What my boss wants is to see the total number of events per host, but only unique to the new field. The problem is he also wants to dedup the interfaces field even if the src_int and dest_int are reversed like this:
alt text

Here is the current search:

index=network  sourcetype="cisco:ios*"  (key_word="MAC_MOVE-SP-4-NOTIF" OR key_word="MAC_MOVE-SW1-4-NOTIF") 
|  eval Interfaces = src_int + "," + dest_int |  table host, Interfaces | sort host | dedup Interfaces 

What I need is to somehow dedup the "Interfaces" field even if the 2 fields that make up the eval are reversed. I then need to add a count of unique "Interfaces" entries per host after the dedup.

Thanks!

0 Karma

somesoni2
Revered Legend

Try something like this

index=network  sourcetype="cisco:ios*"  (key_word="MAC_MOVE-SP-4-NOTIF" OR key_word="MAC_MOVE-SW1-4-NOTIF") 
 |  eval Interfaces = mvsort(split(src_int + "," + dest_int,","))| nomv interfaces |  table host, Interfaces | sort host | dedup Interfaces 

mvasquez21
Explorer

Ah. I gotcha. This works though for the count. Thanks so much for your help!

0 Karma

mvasquez21
Explorer

I had tried that but it gives the same total for each line:

Host Interfaces Count
ns-s-972brus-6509c Gi7/37 47
Po246

ns-s-972brus-6509c Gi7/48 47
Po246

ns-s-972brus-6509c Gi4/25 47
Po246

ns-s-972brus-6509c Gi4/23 47
Po246

What I need is the count of the number of events for each pair of interfaces. For this search the total number of events was 47 so I would expect the count for each pair to add up to 47 total.

Thanks!

0 Karma

somesoni2
Revered Legend

I should've pointed this one out, after dedup, there will be only 1 entry available for each interface, so I guess you need the count of events what was before the dedup. Try this:

index=network  sourcetype="cisco:ios*"  (key_word="MAC_MOVE-SP-4-NOTIF" OR key_word="MAC_MOVE-SW1-4-NOTIF") 
   |  eval Interfaces = mvsort(split(src_int + "," + dest_int,","))| nomv interfaces | stats count latest(host) as host by Interfaces | sort host 
0 Karma

mvasquez2
New Member

actually it would be the # of events AFTER the dedup

0 Karma

somesoni2
Revered Legend

Try this

 index=network  sourcetype="cisco:ios*"  (key_word="MAC_MOVE-SP-4-NOTIF" OR key_word="MAC_MOVE-SW1-4-NOTIF") 
  |  eval Interfaces = mvsort(split(src_int + "," + dest_int,","))| nomv interfaces |  table host, Interfaces | sort host | dedup Interfaces  | eventstats count 
0 Karma

mvasquez2
New Member

OK. Almost there. This is awesome. All I need now is to add a column with the count of the number of events before the dedup.

thanks!

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...