Splunk Search

How do I extract this field from my sample Meraki Flow syslog events to use in search?

randombuffalo
Explorer

I am having trouble using a field that is in my log entries, but Splunk doesn't "auto-discover" it when I started indexing data. I have played around with using the "extract new fields", but can't get the field to work right. The field name is "pattern:" in the log entries. I would like to be able to search based on the value of "pattern:" in the entries. I believe the only values it can have in the logs are: 0, 1, allow all.

Here are some examples:

Dec 15 11:38:54 10.1.1.3 1 1481823534.895775102 HOST_NAME flows src=10.1.1.251 dst=23.23.23.23 mac=C4:71:FE:EE:EE:EE protocol=tcp sport=50814 dport=80 pattern: allow all

Dec 15 11:38:54 10.1.1.3 1 1481823535.012274548 HOST_NAME flows src=111.111.111.111 dst=111.111.111.111 protocol=tcp sport=10155 dport=443 pattern: 0 tcp  && dst port 443 && dst 111.111.111.111

Dec 15 11:41:07 10.1.1.3 1 1481823667.556427731 HOST_NAME flows src=111.111.111.111 dst=111.111.111.111 protocol=tcp sport=46078 dport=23 pattern: 1 all

Dec 15 11:42:00 10.1.1.3 1 1481823667.556427731 HOST_NAME flows src=111.111.111.111 dst=111.111.111.111 protocol=tcp sport=46078 dport=23 pattern: 1

As you can see the pattern: field can have text or a numeric value if that helps narrow down the issue. Any help would be greatly appreciated. I am new to Splunk and have looked around, but I might not even know the terminology to use to search properly.

0 Karma
1 Solution

tgow
Splunk Employee
Splunk Employee

Wondering if you start at the end then work backwards. Does this work:

pattern:\s?(?<pattern>[^$]+)

View solution in original post

kumargaurav
New Member

simply rexing out the field you need will work for you.You will have to just rex out your field from the field.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Give this a try

Your base search | rex "pattern: (?<pattern>(\d+|\w+\s*\w+|\w+))"

Run anywhere search with your sample data

| gentimes start=-1 | eval raw="Dec 15 11:38:54 10.1.1.3 1 1481823534.895775102 HOST_NAME flows src=10.1.1.251 dst=23.23.23.23 mac=C4:71:FE:EE:EE:EE protocol=tcp sport=50814 dport=80 pattern: allow all##Dec 15 11:38:54 10.1.1.3 1 1481823535.012274548 HOST_NAME flows src=111.111.111.111 dst=111.111.111.111 protocol=tcp sport=10155 dport=443 pattern: 0 tcp  && dst port 443 && dst 111.111.111.111##Dec 15 11:41:07 10.1.1.3 1 1481823667.556427731 HOST_NAME flows src=111.111.111.111 dst=111.111.111.111 protocol=tcp sport=46078 dport=23 pattern: 1 all##Dec 15 11:42:00 10.1.1.3 1 1481823667.556427731 HOST_NAME flows src=111.111.111.111 dst=111.111.111.111 protocol=tcp sport=46078 dport=23 pattern: 1" | table raw | makemv raw delim="##" | mvexpand raw | rename raw as _raw 
| rex "pattern: (?<pattern>(\d+|\w+\s*\w+|\w+))"

randombuffalo
Explorer

Thanks for the input but I think I have my question answered above.

0 Karma

tgow
Splunk Employee
Splunk Employee

Wondering if you start at the end then work backwards. Does this work:

pattern:\s?(?<pattern>[^$]+)

randombuffalo
Explorer

Yes that worked! I added your code to my field extraction and it looks like it is picking it up correctly now! Thanks for the help.
alt text

richgalloway
SplunkTrust
SplunkTrust

Consider this REGEX as a starting place for a parser for your events.

\w{3} \d{1,2} \d\d:\d\d:\d\d (?<ip>[^ ]+) \d (?<field>[^ ]+) (?<host_name>[^ ]+) flows src=(?<src>[^ ]+) dst=(?<dst>[^ ]+) mac=(?<mac>[^ ]+) protocol=(?<protocol>[^ ]+) sport=(?<sport>[^ ]+) dport=(?<dport>[^ ]+) pattern: (?<pattern>.*)
---
If this reply helps you, Karma would be appreciated.

randombuffalo
Explorer

Rich, the other fields are picked up fine by Splunk. Would this REGEX mess up those existing extractions? This is what Splunk gave me when I tried to go through the extraction wizard:

 ^(?:[^=\n]*=){5}\d+\s+(?P<Pattern>[^ ]+)

But that doesn't seem to work and ends up including the wrong parts of the log entry as values. As shown in this picture: alt text

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Splunk will auto-discover fields in the name=value pattern, which most of your event uses. The 'pattern' field is different, which is why Splunk is not discovering it. Do you have the ability to change the logging?

---
If this reply helps you, Karma would be appreciated.
0 Karma

randombuffalo
Explorer

Rich, yes the other fields are being picked up fine. Unfortunately for this case, Meraki does not allow changing the output of the log entries. Not sure why they decided to have that one field formatted differently.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...