I have a log that looks like this:
Feb 7 10:15:54 169.16.20.112 02/07/2013:15:15:54 GMT bs112 PPE-3 : SSLVPN ICAEND_CONNSTAT 1042 : Source 277.50.60.70:64868 - Destination 278.52.200.57:2598 - username:domain_name John.Doe:DOMAIN - Start_time "02/07/2013:15:08:33 GMT" - End_time "02/07/2013:15:15:54 GMT" - Duration 00:07:21 - Total_bytes_send 65550 - Total_bytes_recv 3285122 - Total_compressedbytes_send 0 - Total_compressedbytes_recv 0 - Compression_ratio_send 0.00% - Compression_ratio_recv 0.00% - connectionId 58836
Out of this data I am trying to retrieve the following fields: "username:domain", "Start time", "End Time", and "Duration". I would love to be able to seperate it out so that if I had a domains named DOMAIN and OTHERDOMAIN, I could have them be on a seperate line. i.e.:
Domain UserName StartTime EndTime Duration
DOMAIN John.Doe Timestamp TimeStamp 00:07:21
I have produced field extractions, but they only seem to work on the Start Time and End Time:
Start_Time = (?i) .*? "(?P<Start_time>\d+/\d+/\d+:\d+:\d+:\d+\s+\w+)(?=")
End_Time = (?i) End_time (?P<End_time>.+?)\s+\-
UserName = (?i) username:domain_name(?P<username:domain_name>.+?)\s+\-
Duration = (?i) Duration (?P<Duration>[^ ]+)
So I have no idea why only 2 out of 4 of these work when the search is ran, but they work when I generate them.
Suggestions?
There's no space between your domain_name and the capturing group in the third expression, that may lead to wrong results. Duration looks okay to me.
This rex works:
... | rex "username:domain_name\s+(?<username>[^:]+):(?<domain_name>[^ ]+)\s+-\s+Start_time\s+\"(?<start_time>[^\"]+)\"\s+-\s+End_time\s+\"(?<end_time>[^\"]+)\"\s+-\s+Duration\s+(?<duration>[^ ]+)"
domain_name duration end_time start_time username
DOMAIN 00:07:21 02/07/2013:15:15:54 GMT 02/07/2013:15:08:33 GMT John.Doe
It however relies on the order of fields, but you can split it up thanks to individual field labels before the value in your event.
There's no space between your domain_name and the capturing group in the third expression, that may lead to wrong results. Duration looks okay to me.
This rex works:
... | rex "username:domain_name\s+(?<username>[^:]+):(?<domain_name>[^ ]+)\s+-\s+Start_time\s+\"(?<start_time>[^\"]+)\"\s+-\s+End_time\s+\"(?<end_time>[^\"]+)\"\s+-\s+Duration\s+(?<duration>[^ ]+)"
domain_name duration end_time start_time username
DOMAIN 00:07:21 02/07/2013:15:15:54 GMT 02/07/2013:15:08:33 GMT John.Doe
It however relies on the order of fields, but you can split it up thanks to individual field labels before the value in your event.
In the long run I strongly recommend getting the configuration for the extraction working, else you get lots of cluttered searches with the rex. In theory you should be able to strip off the outer quotes and plonk it in a field extraction.
Though I think I got the search working now thanks to your help.
... | rex "username:domain_name\s+(?<username>[^:]+):(?<domain_name>[^ ]+)\s+-\s+Start_time\s+\"(?<start_time>[^\"]+)\"\s+-\s+End_time\s+\"(?<end_time>[^\"]+)\"\s+-\s+Duration\s+(?<duration>[^ ]+)" | table domain_name, username, Start_time, End_time, Duration | sort domain_name, username
When I attempt to go back in and modify any expressions that were created using field extraction
What's the expression that causes the error?
When I went to post, I had to recreate the Duration expression. So after I posted it was working. But when I go back to modify the domain name one.. I get this error:
Encountered the following error while trying to save: In handler 'props-extract': Regex: syntax error in subpattern name (missing terminator)