Splunk Search

where in subsearch

bowa
Path Finder
sourcetype="log4j" source="*server*"
| rex field=_raw "nonce created : (?<nonce>[0-9a-z-]*)"
| transaction thread startswith="startTx" endswith="closeTx"

This first search gives transactions that have a field 'nonce' ... now i want to filter out only the transactions with a specific nonce ... i can find the nonces i am interested in with this query:

sourcetype="log4j" failed nonce
| rex field=_raw "Failed with nonce (?<nonce>[0-9a-z-]*)"
| fields + nonce

So i would like to do some sort of

| where nonce in [search {search2}] 

What is the correct syntax to do such a thing.

Do mind that this loglines that are in search2 are not part of the transaction in the first search, so i cant just filter the transactions more based on their own contence.

Tags (1)
2 Solutions

David
Splunk Employee
Splunk Employee

The right way to do it is to first have the nonce extracted in your props.conf (this simplifies the rest), such as:

[YourSourcetype]
EXTRACT-nonce-failure = Failed with nonce (?<nonce>[0-9a-z-]*)
EXTRACT-nonce-created = nonce created : (?<nonce>[0-9a-z-]*)

You can then do a subsearch first for the failure nonces, and send that to the main search:

sourcetype="log4j" source="*server*" 
   | transaction thread startswith="startTx" endswith="closeTx"
   | search [search sourcetype="log4j" failed nonce | fields nonce]

I believe that should work, at least. What is more efficient, though, assuming that the nonce failure messages also reference the thread field, is to first isolate the failure threads, and then search for the transactions where they're there:

[search sourcetype="log4j" failed nonce | fields thread] 
sourcetype="log4j" source="*server*" 
   | transaction thread startswith="startTx" endswith="closeTx"

That should give you a shortcut to a transaction that includes only the threads that have failed nonces. One bit of minutia is that depending on the number of threads that are re-used, you might need to re-run the limiting search at the end:

[search sourcetype="log4j" failed nonce | fields thread] 
sourcetype="log4j" source="*server*" 
   | transaction thread startswith="startTx" endswith="closeTx"
   | search "Failed with nonce"

The reason why it's more efficient to limit what hits the transaction command is that transaction is very slow.

Let me know if that works for you.

View solution in original post

Stephen_Sorkin
Splunk Employee
Splunk Employee

The output of a subsearch is a valid search expression that will match an event when it matches all the fields of any of the rows of the subsearch. So, if your subsearch only emits a single field, nonce, then it will yield a search expression like: nonce=row_1_nonce OR nonce=row_2_nonce OR ....

With this you can compose your search like:

sourcetype="log4j" source="*server*"
| rex field=_raw "nonce created : (?<nonce>[0-9a-z-]*)"
| transaction thread startswith="startTx" endswith="closeTx"
| search [search sourcetype="log4j" failed nonce
         | rex field=_raw "Failed with nonce (?<nonce>[0-9a-z-]*)"
         | dedup nonce
         | fields nonce]

View solution in original post

Stephen_Sorkin
Splunk Employee
Splunk Employee

The output of a subsearch is a valid search expression that will match an event when it matches all the fields of any of the rows of the subsearch. So, if your subsearch only emits a single field, nonce, then it will yield a search expression like: nonce=row_1_nonce OR nonce=row_2_nonce OR ....

With this you can compose your search like:

sourcetype="log4j" source="*server*"
| rex field=_raw "nonce created : (?<nonce>[0-9a-z-]*)"
| transaction thread startswith="startTx" endswith="closeTx"
| search [search sourcetype="log4j" failed nonce
         | rex field=_raw "Failed with nonce (?<nonce>[0-9a-z-]*)"
         | dedup nonce
         | fields nonce]

bowa
Path Finder

Perfect ! exactly what i needed. Thanks for explaining how this subsearch works like the OR matches.

0 Karma

David
Splunk Employee
Splunk Employee

The right way to do it is to first have the nonce extracted in your props.conf (this simplifies the rest), such as:

[YourSourcetype]
EXTRACT-nonce-failure = Failed with nonce (?<nonce>[0-9a-z-]*)
EXTRACT-nonce-created = nonce created : (?<nonce>[0-9a-z-]*)

You can then do a subsearch first for the failure nonces, and send that to the main search:

sourcetype="log4j" source="*server*" 
   | transaction thread startswith="startTx" endswith="closeTx"
   | search [search sourcetype="log4j" failed nonce | fields nonce]

I believe that should work, at least. What is more efficient, though, assuming that the nonce failure messages also reference the thread field, is to first isolate the failure threads, and then search for the transactions where they're there:

[search sourcetype="log4j" failed nonce | fields thread] 
sourcetype="log4j" source="*server*" 
   | transaction thread startswith="startTx" endswith="closeTx"

That should give you a shortcut to a transaction that includes only the threads that have failed nonces. One bit of minutia is that depending on the number of threads that are re-used, you might need to re-run the limiting search at the end:

[search sourcetype="log4j" failed nonce | fields thread] 
sourcetype="log4j" source="*server*" 
   | transaction thread startswith="startTx" endswith="closeTx"
   | search "Failed with nonce"

The reason why it's more efficient to limit what hits the transaction command is that transaction is very slow.

Let me know if that works for you.

bowa
Path Finder

no the thread variable is different for the failed nonce and the creation of it.

i know it won't be the most performant search, but in this case the result is more important than the timings

0 Karma

stanwin
Contributor

Will the EXTRACT strategy work for multi row output like top command which necessitates multikv?

Specifically there is custom multikv parsing logic , but extracted fields need to be referenced in search query where the EXTRACT will help..

index=main   sourcetype=JOBI    [ search index=main   sourcetype=JOBI |  tail 1 |   multikv conf=JOBI_mkv
| eval JOB  =   Subs.".".User.".".Number | return JOB ]  
| head 1
|   multikv conf=JOBI_mkv
| eval JOB  =   Subs.".".User.".".Number 

eval JOB = Subs.".".User.".".Number

concatenates 3 fields to create a primary key.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...