Splunk Search

How to extract the fields using regex and props.conf at indexing time?

kiran331
Builder

Hello,

How to use Regex in props.conf to extract the fields in the below sample event with source type "syslog".

08/11/17 13:30:34 abckdefrg44 openfep[1123]: [log.c][411]: CPM ALSLLER (ID 5): pin_alarm-DOFRI 22.33.22.5.0 -c 453 1 0 -r 449 0 -v "ONLINE" "FEP" "level Group 33. status" "Status now online" &

Group: level Group 33. status
Status: Status now online

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

Please read this article. It is insightful about the use of index time field extractions:

https://answers.splunk.com/answers/396276/should-i-use-an-index-time-field-extraction.html

Splunk best practice is to do search time field extractions. There are those that are questioning this as you can see by reading the whole answer and comment on this in this article:

https://answers.splunk.com/answers/2535/search-time-vs-index-time-field-extraction.html

If you are still intent on doing index-time extraction, read this article, as it might give you some help in doing it:

https://answers.splunk.com/answers/234011/index-time-field-extraction-regexp-issue.html

And for the official documentation, see this page:

http://docs.splunk.com/Documentation/SplunkCloud/6.6.1/Data/Configureindex-timefieldextraction

And finally, this is the REGEX I would use:

REGEX="(?<Group>[^"]*)"\s+"(?<Status>[^"]*)"\s*&$

But it should go in the transforms.conf, not props.conf.

santiagoaloi
Path Finder

Assuming that there's some level of consistency in your syslog event, and "FEP" will always be there, you could do the following props.conf extract, using "FEP" as a starting point for your regex.

[syslog]
EXTRACT-Group = \"FEP\"\s\"(.*?)\"
EXTRACT-Status = \"FEP\"\s\"(?:.*?)\"\s\"(.*?)\"

Your could use regex101 to practice your regex extractions, make sure to anonymize the data before doing so, nobody knows what that site does to your sample events.

Hope it helps, shout if you need more help.

0 Karma

lfedak_splunk
Splunk Employee
Splunk Employee
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...