Splunk Search

real-time search and field extraction/transformation

alexiri
Communicator

I used to have an index-time field extraction on one of my source types in order to get the error code of the message. I also had a real-time alert on that field, something like "error=ANR1234E". This worked quite nicely, whenever that particular error came up the alert action was triggered.

I've just converted this field extraction to a search-time one, as I've been told that there is no longer a performance benefit and this way its more flexible. Now, my real-time alert no longer works.

Reading the documentation on real-time alerts I see why: they're triggered before index-time. The question is, why, then, did it work when I was doing an index-time field extraction?

0 Karma
1 Solution

hazekamp
Builder

You are correct in that real-time searches grab the data before it hits the index queue, however real-time searches do have access to search time field extractions which happen in the parsing queue.

Can you successfully search for "error=ANR1234E" via non RT search. This would rule out the field extraction as the culprit?

View solution in original post

0 Karma

alexiri
Communicator

You are totally correct, I couldn't. I had the extractions defined in props.conf as EXTRACT-, changing this to REPORT- made it work correctly again. Thanks for pointing me in the right direction!

0 Karma

hazekamp
Builder

You are correct in that real-time searches grab the data before it hits the index queue, however real-time searches do have access to search time field extractions which happen in the parsing queue.

Can you successfully search for "error=ANR1234E" via non RT search. This would rule out the field extraction as the culprit?

0 Karma

alexiri
Communicator

I did, it's "error=ANR1234E".

Regardless, the question isn't about a particular search that isn't working. The question is, how is it possible that a real-time search based on an index-time field extraction actually works, given that the real-time search supposedly runs before the event is indexed?

0 Karma

Simeon
Splunk Employee
Splunk Employee

It would help if you paste your exact search.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...