Right now, my search looks like this:
index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?201,.*)$" | eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5) | eval ContainerID=mvindex(Msg,13) | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest | table _time ActualDest ContainerID
and the log looks like this:
2016 05 09 12:32:29.000 | SDA written: 201,64,5,1,0,8,0,0,0,0,0,16790
2016 05 09 12:32:29.000 | 5,8,04S05577
I can get the destination that I need, but the container ID, 04S05577, doesn't get listed in the table. Help?
Thanks!
Give this a try
Updated
index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?<Msg>201,.*)$" | rex "\|\s+([^,]+),([^,]+),(?<ContainerID>.+)$" | eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5) | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest | stats values(ActualDes) as ActualDest values(ContainerID) as ContainerID by _time
Update #2
Feeling positive about this one
index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?<Msg>201,.*)$" | rex max_match=2 "\|\s+(?<temp>.+)$"
| eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5) | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest
| eval temp=mvindex(temp,-1) | eval ContainerID=mvindex(split(temp,","),-1) | stats values(ActualDes) as ActualDest values(ContainerID) as ContainerID by _time
Update #3
index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?<Msg>201,.*)$" | rex max_match=2 "([\r\n]+)(?<temp>.+)" | eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5) | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest
| eval temp=mvindex(temp,-1) | eval ContainerID=mvindex(split(temp,","),-1) | stats values(ActualDes) as ActualDest values(ContainerID) as ContainerID by _time
Give this a try
Updated
index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?<Msg>201,.*)$" | rex "\|\s+([^,]+),([^,]+),(?<ContainerID>.+)$" | eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5) | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest | stats values(ActualDes) as ActualDest values(ContainerID) as ContainerID by _time
Update #2
Feeling positive about this one
index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?<Msg>201,.*)$" | rex max_match=2 "\|\s+(?<temp>.+)$"
| eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5) | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest
| eval temp=mvindex(temp,-1) | eval ContainerID=mvindex(split(temp,","),-1) | stats values(ActualDes) as ActualDest values(ContainerID) as ContainerID by _time
Update #3
index=4_ip_cnv source="*ATL*Pack*" FirstWord=SDA | rex "\s(?<Msg>201,.*)$" | rex max_match=2 "([\r\n]+)(?<temp>.+)" | eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5) | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest
| eval temp=mvindex(temp,-1) | eval ContainerID=mvindex(split(temp,","),-1) | stats values(ActualDes) as ActualDest values(ContainerID) as ContainerID by _time
What I'm thinking now is that we try to isolate by the second line and ignore the first, eliminating the firstword=SDA.
Something like:
index=4_ip_cnv source="*ATL*Pack*" | rex "\s(?201,.*)$" | rex max_match=2 "\|\s+(?.+)$"
| eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5) | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest
| eval temp=mvindex(temp,-1) | eval ContainerID=mvindex(split(temp,","),-1) | stats values(ActualDest) as ActualDest values(ContainerID) as ContainerID by _time
This gets closer to working. Now all I would have to do is eliminate any value in ContainerID that does not contain "04S"
You can add following ad the end of currently working search
...| eval ContainerID=mvfilter(match(ContainerID,"04S"))
THANKS!!! If you're interested, it ended up being:
index=4_ip_cnv source="*ATL*Pack*" | rex "\s(?201,.*)$" | rex max_match=2 "\|\s+(?.+)$"
| eval Msg=split(Msg,",") | eval ActualDest=mvindex(Msg,5) | eval ActualDest=if(like(SourceName,"%West%"),"West ","East ") . ActualDest
| eval temp=mvindex(temp,-1) | eval ContainerID=mvindex(split(temp,","),-1) | stats values(ActualDest) as ActualDest values(ContainerID) as ContainerID by _time| where like(ContainerID,"04%")| eval ContainerID=mvfilter(match(ContainerID,"04"))
Not quite. It shows
2016-05-10 06:40:14 West 4 1789
2016-05-10 06:40:15 West 4 1790
2016-05-10 06:56:11 West 1 1791
2016-05-10 06:56:12 West 1 1792
2016-05-10 06:56:33 West 1 1793
Which is the correct destination, but does not include the container ID. Rather, it includes the 11th word of the first line.
Unfortunately, not. Still does not display container ID. I think the misstep lies somewhere in rex "|\d+,\d+,(?\S+)$"
Looks like I missed taking the space after the pipe symbol. Just fixed the rex. Give that a shot.
Nope 😕 same result.
😞
Try the updated query now (made changes to regex)
That will just give me the entire first line after the 201 limit, I think.
Ok.. I probably wasn't clear in asking earlier. Does both the lines are part of single event?
e.g.
Event 1
016 05 09 12:32:29.000 | SDA written: 201,64,5,1,0,8,0,0,0,0,0,16790
2016 05 09 12:32:29.000 | 5,8,04S05577
Event2
016 05 09 12:32:29.000 | SDA written: 201,64,5,1,0,8,0,0,0,0,0,16790
2016 05 09 12:32:29.000 | 5,8,04S05577
Yes, both lines are part of a single event, they just are written down into the log in separate lines.
The sample logs that you provided is one log entry OR two? You're using 14th element in the Msg field as ContainerID, but I don't see 14 elements which are separated by comma, So what is the location of ContainerID (is it last element in Msg?).
It's in a separate line. I want to be able to include it. But you're right, I'm trying to include the 14th element, even though there isn't one. How can I include the second line to display that container ID?
04S05577 is it, in the second separated line.
To be able to group these two events together, we need to find a rule/pattern. I don't see any common field between these two events, so can time be the key using which these can be joined together (along with other metadata fields)?
Time can absolutely be used; these two messages will always show up at the exact same time.