How do I extract fields with Rex?

[sarit_s]

Hello
I have these events :

copy and upload completed for day:  2019-05-27 Tue May 28 12:24:40 UTC 2019
going to copy total size of:  25.1 MiB
starting time:  Tue May 28 12:24:40 UTC 2019

I want to extract from the first line the string completed (it can be also failed).
I'd like to name it Status, and StatusTime for the the first date and time.
From second line, the size and name it TotalSize.
From third line I want the date, time, and to name it StartingTime.

How do I do this?

Thanks.

[DavidHourani · Accepted solution]

Hi @sarit_s,

If you have three different events, then use the following for each of the lines.
Line 1: ...| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<StatusTime>[^\n]+)"

Line 2: ...| rex field=_raw "going\sto\scopy\stotal\ssize\sof:\s+(?<TotalSize>[^\n]+)"

Line 3: ...| rex field=_raw "starting\stime:\s+(?<StartingTime>.+)"

You could also combine them as follows:

...| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<StatusTime>[^\n]+)"|  rex field=_raw "going\sto\scopy\stotal\ssize\sof:\s+(?<TotalSize>[^\n]+)"| rex field=_raw "starting\stime:\s+(?<StartingTime>.+)"

Let me know how that works out for you.

Cheers,
David

[sarit_s]

Hi David,
this is how the results looks like :

Status StatusTime TotalSize StartingTime
Tue May 28 12:24:40 UTC 2019 Tue May 28 12:24:40 UTC 2019
Tue May 28 11:27:29 UTC 2019 Tue May 28 11:27:29 UTC 2019
completed 2019-05-27 Tue May 28 12:24:40 UTC 2019 2019-05-27 Tue May 28 12:24:40 UTC 2019 2019-05-27 Tue May 28 12:24:40 UTC 2019
completed 2019-05-27 2019-05-27 2019-05-27
total 25.1 MiB 25.1 MiB 25.1 MiB
total 5.1 MiB 5.1 MiB 5.1 MiB

[DavidHourani]

Hi @sarit_s, updated the answer, have a look and try again !

[sarit_s]

Hi @DavidHourani
Thanks !!
it is almost perfect

just the rex for StatusTime should be separeted to 2 fields :

this is the result:
2019-05-27 Tue May 28 12:24:40 UTC 2019
and i need 2019-05-27 to be one part and the rest as second part

[sarit_s]

i did something like this:

| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<RunningTime>\S+)(?<StatusTime>.+)"

is that ok ?

[DavidHourani]

Looks good, you can add the space between the two fields as well to avoid having it in StatusTime :

 | rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<RunningTime>\S+)\s(?<StatusTime>.+)"

[sarit_s]

perfect
thanks !

[DavidHourani]

This should do it :

 ...| rex field=_raw "copy\sand\supload\s(?<Status>\w+)[^:]+:\s+(?<StatusTime>[^\n]+)"|  rex field=_raw "going\sto\scopy\stotal\ssize\sof:\s+(?<TotalSize>[^\n]+)"| rex field=_raw "starting\stime:\s+(?<StartingTime>.+)"

[harsmarvania57]

Hi,

Please try below regex

<yourBaseSearch>
| rex field=_raw "^(?:[^\s]*[\s]){3}(?<Status>\w+)[^\:]*\:\s+(?<StatusTime>[^\v]*)\v[^\:]*\:\s+(?<TotalSize>[^\v]*)\v[^\:]*\:\s+(?<StartingTime>[^\v]*)"

Regex101 URL with sample data you have provided : https://regex101.com/r/a5Rbki/1

[sarit_s]

it returns empty results 😞

[harsmarvania57]

Is it multiline event? If yes then you need to provide whole sample event, part of event will not work.

[sarit_s]

each event in separet line, just as i wrote at my questions
i think it should be rex for each line

 copy and upload completed for day:  2019-05-27 Tue May 28 12:24:40 UTC 2019
 going to copy total size of:  25.1 MiB
 starting time:  Tue May 28 12:24:40 UTC 2019

[cmakepeace_nfcu]

The below SPL should show the desired end status. This can also be done within props and transforms if you wish for this to be done for all data of that specific sourcetype.

| makeresults 
| eval test="copy and upload completed for day:  2019-05-27 Tue May 28 12:24:40 UTC 2019~ 
going to copy total size of:  25.1 MiB~
starting time:  Tue May 28 12:24:40 UTC 2019" 
| fields - _time 
| rex field=test "(\w*\s){3}(?<Status>\w*)[^:]*:\s+(?<StatusTime>[^\n]*)\n[^:]*:\s+(?<TotalSize>[^\n]*)\n[^:]*:\s+(?<StartingTime>.*)"

This can also be done within props and transforms if you wish for this to be done for all data of that specific sourcetype.

[sarit_s]

it returns empty results 😞