Splunk Search

Field extraction regex conditional if-then-else

cdstealer
Contributor

Here are 2 events from an apache log. I have a field extraction regex which works unless the content-type contains a "charset" field.

[01/Aug/2014:07:43:48 +0100] 1150 xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx, xxx.xxx.xxx.xxx, xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx IT 200 GET www.URL.com/images/favicon.ico - "Mozilla/5.0 (Linux; U; Android 4.4.2; en-gb; HTC_Desire_610 Build/KOT49H) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30" text/plain; charset=UTF-8 2148

[01/Aug/2014:07:43:58 +0100] 293 xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx - 200 GET www.URL.com/robots.txt - "Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +go.mail.ru/help/robots)" text/javascript 118943

The regex that works on the second event is:

(?i)^[^\+]*\+\d+\]\s+(?P<bytes>[^ ]+)\s(?P<clientip>[^ ]+)\s(?P<xforward_ip>[^ ]+)\s(?P<cluster_ip>[^ ]+)\s(?P<lang>[^ ]+)\s(?P<response>[^ ]+)\s(?P<method>[^ ]+)\s(?P<uri>[^ ]+)\s(?P<referer>[^ ]+)\s"(?P<useragent>[^"]*?)"\s(?P<mime_type>[^ ]+)\s(?P<response_time>[^ ]+)

So what I’m trying to do is have the regex match "text/plain" but if it sees "; charset=UTF-8" to also match that in the same group.

So my attempt at the regex is:

(?i)^[^\\+]*\\+\\d+\\]\\s+(?P<bytes>[^ ]+)\\s(?P<clientip>[^ ]+)\\s(?P<xforward_ip>[^ ]+)\\s(?P<cluster_ip>[^ ]+)\\s(?P<lang>[^ ]+)\\s(?P<response>[^ ]+)\\s(?P<method>[^ ]+)\\s(?P<uri>[^ ]+)\\s(?P<referer>[^ ]+)\\s\"(?P<useragent>[^\"]*?)\"\\s?(\\w+\\W\\w+\\W\\s\\S+?)(?P<mime_type>[^;]+)|(?P<mime_type>[^ ]+)\\s(?P<response_time>[^ ]+)

The if-then-else statement is ?(\\w+\\W\\w+\\W\\s\\S+?)(?P<mime_type>[^;]+)|(?P<mime_type>[^ ]+) but splunk gives the error "Regex: two named subpatterns have the same name", which I understand.

Unfortunately I'm a regex noob, so this is my understanding...

?(\\w+\\W\\w+\\W\\s\\S+?) = if(condition)

(?P<mime_type>[^;]+) = then field is

|(?P<mime_type>[^ ]+) = else match

Hope that makes sense 🙂

1 Solution

somesoni2
SplunkTrust
SplunkTrust

Give this a try

(?i)^[^\+]*\+\d+\]\s+(?P<bytes>[^ ]+)\s(?P<clientip>[^ ]+)\s(?P<xforward_ip>[^ ]+)\s(?P<cluster_ip>[^ ]+)\s(?P<lang>[^ ]+)\s(?P<response>[^ ]+)\s(?P<method>[^ ]+)\s(?P<uri>[^ ]+)\s(?P<referer>[^ ]+)\s\"(?P<useragent>[^\"]*?)\"\s(?P<mime_type>(\w+\/\w+))(.*)\s(?P<response_time>\d+)

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

Give this a try

(?i)^[^\+]*\+\d+\]\s+(?P<bytes>[^ ]+)\s(?P<clientip>[^ ]+)\s(?P<xforward_ip>[^ ]+)\s(?P<cluster_ip>[^ ]+)\s(?P<lang>[^ ]+)\s(?P<response>[^ ]+)\s(?P<method>[^ ]+)\s(?P<uri>[^ ]+)\s(?P<referer>[^ ]+)\s\"(?P<useragent>[^\"]*?)\"\s(?P<mime_type>(\w+\/\w+))(.*)\s(?P<response_time>\d+)

cdstealer
Contributor

perfect! Thank you very much 🙂 Looks like I was over engineering it.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...