Splunk Search

transaction commmand with huge event base

lwalhoefer
Engager

Hi,

I've the following _raw event base:

  • line1 field1=field1Value field2=field2Value sometext: a_string

  • line2 field1=field1Value field2=field2Value sometext2 val=400

  • line3 field1=field1Value field2=field2Value sometext2 val=600

... and like to have the table that only contains events where val reaches a limit. When this limit is reached, I like to see the value behind "sometext: " (=a_string) from the event above with same field1Value and field2Value.

The resulting table should have the cols:

  • field1 | field2 | val | msg

A row should have the values:

  • field1Value | field1Value | 600 | a_string

Here's my try with the transaction command:

index=myindex "sometext" OR ("sometext2" AND val>500) 
| transaction field1 field2 
| rex field=_raw "sometext: (?<msg>.*)" 
| table field1 field2 val msg

The 2 issues are:

  • the msg field is always emtpy and seem to not extracted correctly
  • The first part of the query (up to the first pipe symbol) is returning a huge number of events (~200k) and thus the transaction seem takes an unacceptable time. Is the transaction a good way to accomplish such a resulting table? I suppose a "join" is not an option?

Any ideas?

Thanks!

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

I don't know about your extraction problem, but I might use:

index=myindex "sometext" OR ("sometext2" AND val>500) 
| rex "sometext: (?<msg>.*)"
| stats val,msg by field1,field2

This will run a lot faster if you have the data over multiple indexers, since stats will map-reduce much better than transaction.

You could do this:

index=myindex ("sometext" OR ("sometext2" AND val>500))
[ search index=myindex "sometext2" AND val>500 
  | fields + field1 field2 
  | format maxresults=10000 ] )
| rex "sometext: (?<msg>.*)"
| stats val,msg by field1,field2

but whether that is actually faster depends on your data.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...