Splunk Search

Routing to nullQueue based on extracted field

Paolo_Prigione
Builder

Hi, is it possible to route events to nullQueue based on the value found in a field generated by a csv lookup?

I am facing a SAP audit sequential file which:

  1. (on a LWF) is read through a scripted input, which also splits it in lines;
  2. has field extraction based on field position and length;
  3. field "user_name" is used to lookup a csv file which provides the field "user_role".

I'd need to only index log lines which have a particular value of user_role, but since that value is not present in _raw, I am not able to write the REGEX in the transforms.conf stanza.

Any idea would be greatly appreciated, thanks

hulahoop
Splunk Employee
Splunk Employee

Hi Paolo, did you ever solve this problem? If not, since you are collecting this data via a scripted input, why not add the lookup capability on the user_name field in the same script? The script could either augment _raw with the user_role, or only write events for roles are you interested in. The 2nd option saves Splunk the trouble of having to apply index-time filtering altogether and maybe some CPU cycles.

0 Karma

Paolo_Prigione
Builder

Thanks for the suggestion. At the time I didn't have the skills to set that up. I had solved the problem at search time: the volume was not that bad.

0 Karma

dwaddle
SplunkTrust
SplunkTrust

Sorry Paolo, I don't think it's possible. According to http://www.splunk.com/base/Documentation/latest/Admin/Indextimeversussearchtime lookups are applied at search time, not index time. The nullQueue routing would have to occur at index time in order to be effective.

Paolo_Prigione
Builder

Thank you Dwaddle; I actually knew it but was hoping on some creative ways to direct inputs to the parsingQueue and then to the nullQueue prior to indexing.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...