Splunk Search

Is there a way to generalize these in Splunk?

Nicholas_Key
Splunk Employee
Splunk Employee

I would like to know if there is a way to generalize the following EXTRACT regexes in my props.conf? The configuration looks like the following:

[activityLog]
LINE_BREAKER = [-]{8,}([\r\n]+)
SHOULD_LINEMERGE = false
EXTRACT-ComponentId = ComponentId:\t(?.*)
EXTRACT-ProcessId = ProcessId:\t(?.*)
EXTRACT-ThreadId = ThreadId:\t(?.*)
EXTRACT-ThreadName = ThreadName:\t(?.*)
EXTRACT-SourceId = SourceId:\t(?.*)
EXTRACT-ClassName = ClassName:\t(?.*)
EXTRACT-MethodName = MethodName:\t(?.*)
EXTRACT-Manufacturer = Manufacturer:\t(?.*)
EXTRACT-Product = Product:\t(?.*)
EXTRACT-Version = Version:\t(?.*)
EXTRACT-ServerName = ServerName:\t(?.*)
EXTRACT-TimeStamp = TimeStamp:\t(?.*)
EXTRACT-UnitOfWork = UnitOfWork:\t(?.*)
EXTRACT-Severity = Severity:\t(?.*)
EXTRACT-Category = Category:\t(?.*)
EXTRACT-PrimaryMessage = PrimaryMessage:\t(?.*)
EXTRACT-ExtendedMessage = ExtendedMessage:\t(?.*)

And the content have a similar pattern

ComponentId:     Application Server
ProcessId:   5316
ThreadId:    00000000
ThreadName:  P=901265:O=0:CT
SourceId:    com.ibm.ws.runtime.WsServerImpl
ClassName:   
MethodName:  
Manufacturer:    IBM
Product:     WebSphere
Version:     Platform 7.0.0.7 [BASE 7.0.0.7 cf070942.55]
ServerName:  sfeserv36Node01Cell\sfeserv36Node01\server1
TimeStamp:   2010-04-27 09:15:57.671000000
UnitOfWork:  
Severity:    3
Category:    AUDIT
PrimaryMessage:  WSVR0001I: Server server1 open for e-business
ExtendedMessage: 
Tags (2)
0 Karma
1 Solution

Simeon
Splunk Employee
Splunk Employee

The suggested extraction for that type of input would be to use a delimiter such as new line in combination with something else. See the DELIMS spec for the config file int transforms.conf:

http://www.splunk.com/base/Documentation/latest/Admin/Transformsconf

I believe you would want to use a combination of the ":" and "\n". So you specify the new line ("\n") to separate the key/value pair and the ":" to separate the key from the value... so something like:

[activity_report]
DELIMS = "\n", ":"

You will need to perform some things before this, which include moving the extraction to occur in the transforms.conf file, instead of the props.conf file. To do this, use/create a TRANSFORMS or REPORT stanza in props.conf that leverages the extraction method through transforms (scroll down to transforms configuration):

http://www.splunk.com/base/Documentation/latest/Admin/Propsconf

View solution in original post

Simeon
Splunk Employee
Splunk Employee

The suggested extraction for that type of input would be to use a delimiter such as new line in combination with something else. See the DELIMS spec for the config file int transforms.conf:

http://www.splunk.com/base/Documentation/latest/Admin/Transformsconf

I believe you would want to use a combination of the ":" and "\n". So you specify the new line ("\n") to separate the key/value pair and the ":" to separate the key from the value... so something like:

[activity_report]
DELIMS = "\n", ":"

You will need to perform some things before this, which include moving the extraction to occur in the transforms.conf file, instead of the props.conf file. To do this, use/create a TRANSFORMS or REPORT stanza in props.conf that leverages the extraction method through transforms (scroll down to transforms configuration):

http://www.splunk.com/base/Documentation/latest/Admin/Propsconf

Nicholas_Key
Splunk Employee
Splunk Employee

Thank you for your helpful hint, Simeon! It works with DELIMS = "\n", ":\t" because first, the lines are separated by a "\n" and second, the key-value pairs are separated by a ":\t".

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...