How can I tag selected sets of log records/events with a "xxx=yyy" field during indexing?

[DrFedtke]

Our situation: We have several complex conditions classifying groups of records, and we want to tag them with a new and dedicated "xxx=yyy" field (name-value pair; e.g. "XYZrelevant="yes").

In some case we want to apply 2 or 3 bigger "selects" to add the "xxx=yyy" field step-by-step to relevant records.

This added field will then be used in our real-time dashboards to easily select relevant records (namely without the need to include all the comprehensive selection). Since these dashboards are "real-time", the "xxx=yyy" cannot be added once a day, but this has to happen continuously.

How can we do that?

We know lookup tables, but their capabilities are not sufficient.

At this point it's also not clear for us which "stages" or steps exist within splunk's overall log record processing. In which of these steps could we hook in to realize the above?

Thanks for your answers.

Best
Caspar

[woodcock]

Here is an important limitation to Index-Time field extractions that is implied by the documentation, but not clearly, as I am about to do: You can only create Index-Time fields from contiguous stretches of characters that already exist inside of your raw text or metadata (e.g. source). So what you are desiring is impossible unless you preprocess your logs to insert the characters that you desire before they go to splunk to be indexed.

[somesoni2]

You might want to look at the index time field extraction.

http://docs.splunk.com/Documentation/Splunk/6.2.0/Data/Configureindex-timefieldextraction