Splunk Search

Add additional events to transaction by field

idzjuba
Engager

Hi, I need to group events where the first event begins with "Receive message" and grouped by thread id.
But then need to add event from another thread id joined by field value.
For example this is the data

2019-12-03 14:48:54,320 INFO thread1 Received message execId=trade1
2019-12-03 14:48:54,321 INFO thread1 Process message
2019-12-03 14:48:54,325 INFO thread2 Received message execId=trade2
2019-12-03 14:48:54,327 INFO thread2 Process message
2019-12-03 14:48:54,421 INFO thread1 Process again message
2019-12-03 14:48:54,427 INFO thread2 Process again message
2019-12-03 14:48:54,527 INFO sender Send message execId=trade1
2019-12-03 14:48:54,528 INFO sender Send message execId=trade2

First part I am getting with this

source="simple.txt" | rex "^[^ ]*\s+[^ ]*\s+[^ ]*\s+(?[^ ]*)" | transaction threadId startswith="Received message" maxpause=1h 

2019-12-03 14:48:54,320 INFO thread1 Received message execId=trade1
2019-12-03 14:48:54,321 INFO thread1 Process message
2019-12-03 14:48:54,421 INFO thread1 Process again message

2019-12-03 14:48:54,325 INFO thread2 Received message execId=trade2
2019-12-03 14:48:54,327 INFO thread2 Process message
2019-12-03 14:48:54,427 INFO thread2 Process again message

But cannot find the way to adjust query to add sender thread event correlated by execId to transaction.
So that in result should be

2019-12-03 14:48:54,320 INFO thread1 Received message execId=trade1
2019-12-03 14:48:54,321 INFO thread1 Process message
2019-12-03 14:48:54,421 INFO thread1 Process again message
2019-12-03 14:48:54,527 INFO sender Send message execId=trade1

2019-12-03 14:48:54,325 INFO thread2 Received message execId=trade2
2019-12-03 14:48:54,327 INFO thread2 Process message
2019-12-03 14:48:54,427 INFO thread2 Process again message
2019-12-03 14:48:54,528 INFO sender Send message execId=trade2

Tried append, join but cannot make it work.
Please advise, thanks

Tags (1)
0 Karma
1 Solution

Richfez
SplunkTrust
SplunkTrust

I pretended your regex worked well enough, and just flibbered something well enough into place.

Here's one method, using transaction after using transaction.

source="simple.txt" NOT "sender Send message" 
| rex "^[^ ]*\s+[^ ]*\s+(?<thing1>[^ ]*)\s+(?<threadId>[^ ]*)"
| transaction threadId startswith="Received message" maxpause=1h
| append [search source="simple.txt" "sender Send message"]
| transaction execId

Much of it is exactly like you had it. (I fiddled with the regex that the markup editor ate).

But you can see I initially exclude the "sender Send message" lines, and only transaction the ones that didn't have that line in them together.

THEN in line 4, I append in the "sender Send message" lines. If you run just the first four lines (take out that last one) you'll see the two groups of three, then the two orphaned lines.

So lastly, line 5 I just group those in on execId, which is parsed automatically because it's a KV pair.

Works a charm.

I do recommend converting most of these to stats so they're more performant, but I think that's another question and may not be necessary.

Happy Splunking,
Rich

View solution in original post

Richfez
SplunkTrust
SplunkTrust

I pretended your regex worked well enough, and just flibbered something well enough into place.

Here's one method, using transaction after using transaction.

source="simple.txt" NOT "sender Send message" 
| rex "^[^ ]*\s+[^ ]*\s+(?<thing1>[^ ]*)\s+(?<threadId>[^ ]*)"
| transaction threadId startswith="Received message" maxpause=1h
| append [search source="simple.txt" "sender Send message"]
| transaction execId

Much of it is exactly like you had it. (I fiddled with the regex that the markup editor ate).

But you can see I initially exclude the "sender Send message" lines, and only transaction the ones that didn't have that line in them together.

THEN in line 4, I append in the "sender Send message" lines. If you run just the first four lines (take out that last one) you'll see the two groups of three, then the two orphaned lines.

So lastly, line 5 I just group those in on execId, which is parsed automatically because it's a KV pair.

Works a charm.

I do recommend converting most of these to stats so they're more performant, but I think that's another question and may not be necessary.

Happy Splunking,
Rich

idzjuba
Engager

Thanks a lot for explanation. Works perfect

0 Karma

Richfez
SplunkTrust
SplunkTrust

Also, could you repaste your regex, and this time be sure to use the little "code" button (101010)? The editor ate your special characters.

0 Karma

Richfez
SplunkTrust
SplunkTrust

Oh, if only all the messages had "execId=" it would be so easy, and fast. 🙂

Questions - is the only messages you want sort of "appended in" like that going to be ones that follow this format?
2019-12-03 14:48:54,528 INFO sender Send message execId=trade2

And, do threads (thread1, thread2) get reused in short order and recycled? Or are those spun up for a thing, and instantly reused so they're not really useful for grouping?

(And maybe those aren't even important, I have a thought that searching for only those that aren't "sender Send message blah blah" and transactioning (or whatever) on those, then adding in the "sender Send message blah blah" ones via append, then using stats to group them.... Maybe I'll try that and write it up while waiting on these answers.)

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...