Splunk Search

Problems with extracted fields

pdgill314
Path Finder

I have a log that looks like this:

Feb  7 10:15:54 169.16.20.112  02/07/2013:15:15:54 GMT bs112 PPE-3 : SSLVPN ICAEND_CONNSTAT 1042 :  Source 277.50.60.70:64868 - Destination 278.52.200.57:2598 - username:domain_name John.Doe:DOMAIN - Start_time "02/07/2013:15:08:33 GMT" - End_time "02/07/2013:15:15:54 GMT" - Duration 00:07:21  - Total_bytes_send 65550 - Total_bytes_recv 3285122 - Total_compressedbytes_send 0 - Total_compressedbytes_recv 0 - Compression_ratio_send 0.00% - Compression_ratio_recv 0.00% - connectionId 58836

Out of this data I am trying to retrieve the following fields: "username:domain", "Start time", "End Time", and "Duration". I would love to be able to seperate it out so that if I had a domains named DOMAIN and OTHERDOMAIN, I could have them be on a seperate line. i.e.:

Domain   UserName   StartTime   EndTime   Duration
DOMAIN   John.Doe   Timestamp   TimeStamp 00:07:21

I have produced field extractions, but they only seem to work on the Start Time and End Time:

Start_Time = (?i) .*? "(?P<Start_time>\d+/\d+/\d+:\d+:\d+:\d+\s+\w+)(?=") 
End_Time = (?i) End_time (?P<End_time>.+?)\s+\- 
UserName = (?i) username:domain_name(?P<username:domain_name>.+?)\s+\- 
Duration = (?i) Duration (?P<Duration>[^ ]+) 

So I have no idea why only 2 out of 4 of these work when the search is ran, but they work when I generate them.

Suggestions?

Tags (1)
0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

There's no space between your domain_name and the capturing group in the third expression, that may lead to wrong results. Duration looks okay to me.

This rex works:

 ... | rex "username:domain_name\s+(?<username>[^:]+):(?<domain_name>[^ ]+)\s+-\s+Start_time\s+\"(?<start_time>[^\"]+)\"\s+-\s+End_time\s+\"(?<end_time>[^\"]+)\"\s+-\s+Duration\s+(?<duration>[^ ]+)"

domain_name  duration  end_time                 start_time               username
DOMAIN       00:07:21  02/07/2013:15:15:54 GMT  02/07/2013:15:08:33 GMT  John.Doe

It however relies on the order of fields, but you can split it up thanks to individual field labels before the value in your event.

View solution in original post

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

There's no space between your domain_name and the capturing group in the third expression, that may lead to wrong results. Duration looks okay to me.

This rex works:

 ... | rex "username:domain_name\s+(?<username>[^:]+):(?<domain_name>[^ ]+)\s+-\s+Start_time\s+\"(?<start_time>[^\"]+)\"\s+-\s+End_time\s+\"(?<end_time>[^\"]+)\"\s+-\s+Duration\s+(?<duration>[^ ]+)"

domain_name  duration  end_time                 start_time               username
DOMAIN       00:07:21  02/07/2013:15:15:54 GMT  02/07/2013:15:08:33 GMT  John.Doe

It however relies on the order of fields, but you can split it up thanks to individual field labels before the value in your event.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

In the long run I strongly recommend getting the configuration for the extraction working, else you get lots of cluttered searches with the rex. In theory you should be able to strip off the outer quotes and plonk it in a field extraction.

0 Karma

pdgill314
Path Finder

Though I think I got the search working now thanks to your help.

... | rex "username:domain_name\s+(?<username>[^:]+):(?<domain_name>[^ ]+)\s+-\s+Start_time\s+\"(?<start_time>[^\"]+)\"\s+-\s+End_time\s+\"(?<end_time>[^\"]+)\"\s+-\s+Duration\s+(?<duration>[^ ]+)" | table domain_name, username, Start_time, End_time, Duration | sort domain_name, username
0 Karma

pdgill314
Path Finder

When I attempt to go back in and modify any expressions that were created using field extraction

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

What's the expression that causes the error?

0 Karma

pdgill314
Path Finder

When I went to post, I had to recreate the Duration expression. So after I posted it was working. But when I go back to modify the domain name one.. I get this error:

Encountered the following error while trying to save: In handler 'props-extract': Regex: syntax error in subpattern name (missing terminator)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...