Combining two non-correlated fields in a table

[bhhxxv]

Hi,

We have some log files created by the job. There are some lines in the logs as:

(TID#0x000013d8) 2014-Jun-24 05:11:56.928 INFO: [Source: ETC_FRW_alertLevel : Source] #52321 - Activation arguments - '{ modelId=ETC_WSH, processDate=23/6/2014 00:00:00.000000, aggUnit=default, runConfiguration=13 }'

and after some 10-15 lines there is a line as:

(TID#0x00001054) 2014-Jun-24 05:12:24.707 INFO: [Source: ETC_FRW_alertLevel : Source] #53007 - Source step 0 (Passed: 0, Filtered:0, Erroneous: 0, Containers deadlocks: 0).

I need the values corrsponding to runConfiguration i.e.13 in this case and Passed which is 0 in this case on the same line of the table.

I have made the search query but it is giving me values in two different lines as two lines are considered as two different events in Splunk. They do not even have common fileds in which we can join the 2 events.
The search query is:

| multisearch
 [search "passed" index=actim* | rex field=_raw "Passed: (?<etc_frw_passed>[0-9]+)" ]
 [search "runconfiguration" index=actim* | rex field=_raw "runConfiguration=(?<run_config>[0-9]+)"]
| table run_config etc_frw_passed _time source

Please suggest something.

Thanks.

[bhhxxv]

Hi,

There is no large gap between the two events. They occur within two minutes. The exact start time is mention in the question where the logs are pasted.

Thanks

[datasearchninja · Accepted solution]

I'm assuming these are from the same host and source log file? Without anything to exactly tie the events together, you can only guess that events that appear after each other are related. If that is ok, you could do this:

index=actim* ("passed" OR "runconfiguration") | rex field=_raw "Passed: (?<etc_frw_passed>[0-9]+)" | rex field=_raw "runConfiguration=(?<run_config>[0-9]+)"] | transaction host source maxevents=2 maxspan=60s startswith="run_config=*" endswith="etc_frw_passed=*" | table _time host source run_config etc_frw_passwd

[somesoni2]

% is wildcard character for WHERE..LIKE command. There first argument ",13,19,16,18,", each number/run_config starts and ends with comma, so when comparing this set of numbers with your run_config value, I added "%," as prefix and ",%" as suffix to it so that it will return true if run_config value matches any number in the set.

[bhhxxv]

Hi,

Can you please explain me the LIKE(",13,19,16,18,","%,".run_config.",%") statement.

Thanks

[bhhxxv]

Thanks a lot.This worked.

I didnot understood the meaning of "%,".run_config.",%" in this search.

[somesoni2]

Try this

   index=actim* ("passed" OR "runconfiguration") | rex field=_raw "Passed: (?<etc_frw_passed>[0-9]+)" | rex field=_raw "runConfiguration=(?<run_config>[0-9]+)"] | transaction host source maxevents=2 maxspan=60s startswith="run_config=*" endswith="etc_frw_passed=*" | eval run_config_name=case(LIKE(",11,21,2,5,9,41,20,25,","%,".run_config.",%"),"TAS",LIKE(",13,19,16,18,","%,".run_config.",%"),"EDW",1=1,"N/A")| table _time host source run_config run_config_name etc_frw_passwd

Note that there are commas at the start and end of the possible values (the string in case command) for run_config.

[bhhxxv]

Hi,

I have another requirement of displaying the classification of the run configuration like for run configuration values ="11,21,2,5,9,41,20,25" the name to be displayed is 'TAS' similarly for run configuration values=="13,19,16,18" name should be 'EDW'.

The sample search query i have made is :

index=actim* ("passed" OR "runconfiguration") | rex field=_raw "Passed: (?[0-9]+)" | rex field=_raw "runConfiguration=(?[0-9]+)"] | transaction host source maxevents=2 maxspan=60s startswith="run_config=" endswith="etc_frw_passed=" | eval runConfig= case(run_config=18,"EDW",run_config=11,"TAS") |table _time host source run_config etc_frw_passwd

but no expected result.

Please suggest.

Thanks

[datasearchninja]

Hi, If you could mark my answer as accepted that would be great!

The search works by building a base search that has all the events ( index=actim* ("passed" OR "runconfiguration") ). It then uses the transaction command to build a correlating transaction that (1) have exactly 2 events (2) Occur no more than 60 seconds apart (3) The first event needs to have your run_config extracted field (4) The last event needs to have your etc_frw_passed extracted field.

[bhhxxv]

Thanks Colin. This worked very well.

Yes these events are from same host and source.
it wouldbe great if you could explain me how you get that worked!!

Thanks

[martin_mueller]

Is there any other way of correlating these two lines together? For example, does such a pair happen closely together in terms of time with a reasonably large gap to the next pair?