Splunk Search

Add additional events to transaction by field

idzjuba
Engager

Hi, I need to group events where the first event begins with "Receive message" and grouped by thread id.
But then need to add event from another thread id joined by field value.
For example this is the data

2019-12-03 14:48:54,320 INFO thread1 Received message execId=trade1
2019-12-03 14:48:54,321 INFO thread1 Process message
2019-12-03 14:48:54,325 INFO thread2 Received message execId=trade2
2019-12-03 14:48:54,327 INFO thread2 Process message
2019-12-03 14:48:54,421 INFO thread1 Process again message
2019-12-03 14:48:54,427 INFO thread2 Process again message
2019-12-03 14:48:54,527 INFO sender Send message execId=trade1
2019-12-03 14:48:54,528 INFO sender Send message execId=trade2

First part I am getting with this

source="simple.txt" | rex "^[^ ]*\s+[^ ]*\s+[^ ]*\s+(?[^ ]*)" | transaction threadId startswith="Received message" maxpause=1h 

2019-12-03 14:48:54,320 INFO thread1 Received message execId=trade1
2019-12-03 14:48:54,321 INFO thread1 Process message
2019-12-03 14:48:54,421 INFO thread1 Process again message

2019-12-03 14:48:54,325 INFO thread2 Received message execId=trade2
2019-12-03 14:48:54,327 INFO thread2 Process message
2019-12-03 14:48:54,427 INFO thread2 Process again message

But cannot find the way to adjust query to add sender thread event correlated by execId to transaction.
So that in result should be

2019-12-03 14:48:54,320 INFO thread1 Received message execId=trade1
2019-12-03 14:48:54,321 INFO thread1 Process message
2019-12-03 14:48:54,421 INFO thread1 Process again message
2019-12-03 14:48:54,527 INFO sender Send message execId=trade1

2019-12-03 14:48:54,325 INFO thread2 Received message execId=trade2
2019-12-03 14:48:54,327 INFO thread2 Process message
2019-12-03 14:48:54,427 INFO thread2 Process again message
2019-12-03 14:48:54,528 INFO sender Send message execId=trade2

Tried append, join but cannot make it work.
Please advise, thanks

Tags (1)
0 Karma
1 Solution

Richfez
SplunkTrust
SplunkTrust

I pretended your regex worked well enough, and just flibbered something well enough into place.

Here's one method, using transaction after using transaction.

source="simple.txt" NOT "sender Send message" 
| rex "^[^ ]*\s+[^ ]*\s+(?<thing1>[^ ]*)\s+(?<threadId>[^ ]*)"
| transaction threadId startswith="Received message" maxpause=1h
| append [search source="simple.txt" "sender Send message"]
| transaction execId

Much of it is exactly like you had it. (I fiddled with the regex that the markup editor ate).

But you can see I initially exclude the "sender Send message" lines, and only transaction the ones that didn't have that line in them together.

THEN in line 4, I append in the "sender Send message" lines. If you run just the first four lines (take out that last one) you'll see the two groups of three, then the two orphaned lines.

So lastly, line 5 I just group those in on execId, which is parsed automatically because it's a KV pair.

Works a charm.

I do recommend converting most of these to stats so they're more performant, but I think that's another question and may not be necessary.

Happy Splunking,
Rich

View solution in original post

Richfez
SplunkTrust
SplunkTrust

I pretended your regex worked well enough, and just flibbered something well enough into place.

Here's one method, using transaction after using transaction.

source="simple.txt" NOT "sender Send message" 
| rex "^[^ ]*\s+[^ ]*\s+(?<thing1>[^ ]*)\s+(?<threadId>[^ ]*)"
| transaction threadId startswith="Received message" maxpause=1h
| append [search source="simple.txt" "sender Send message"]
| transaction execId

Much of it is exactly like you had it. (I fiddled with the regex that the markup editor ate).

But you can see I initially exclude the "sender Send message" lines, and only transaction the ones that didn't have that line in them together.

THEN in line 4, I append in the "sender Send message" lines. If you run just the first four lines (take out that last one) you'll see the two groups of three, then the two orphaned lines.

So lastly, line 5 I just group those in on execId, which is parsed automatically because it's a KV pair.

Works a charm.

I do recommend converting most of these to stats so they're more performant, but I think that's another question and may not be necessary.

Happy Splunking,
Rich

idzjuba
Engager

Thanks a lot for explanation. Works perfect

0 Karma

Richfez
SplunkTrust
SplunkTrust

Also, could you repaste your regex, and this time be sure to use the little "code" button (101010)? The editor ate your special characters.

0 Karma

Richfez
SplunkTrust
SplunkTrust

Oh, if only all the messages had "execId=" it would be so easy, and fast. 🙂

Questions - is the only messages you want sort of "appended in" like that going to be ones that follow this format?
2019-12-03 14:48:54,528 INFO sender Send message execId=trade2

And, do threads (thread1, thread2) get reused in short order and recycled? Or are those spun up for a thing, and instantly reused so they're not really useful for grouping?

(And maybe those aren't even important, I have a thought that searching for only those that aren't "sender Send message blah blah" and transactioning (or whatever) on those, then adding in the "sender Send message blah blah" ones via append, then using stats to group them.... Maybe I'll try that and write it up while waiting on these answers.)

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...