Splunk Search

Is there a way to generalize these in Splunk?

Nicholas_Key
Splunk Employee
Splunk Employee

I would like to know if there is a way to generalize the following EXTRACT regexes in my props.conf? The configuration looks like the following:

[activityLog]
LINE_BREAKER = [-]{8,}([\r\n]+)
SHOULD_LINEMERGE = false
EXTRACT-ComponentId = ComponentId:\t(?.*)
EXTRACT-ProcessId = ProcessId:\t(?.*)
EXTRACT-ThreadId = ThreadId:\t(?.*)
EXTRACT-ThreadName = ThreadName:\t(?.*)
EXTRACT-SourceId = SourceId:\t(?.*)
EXTRACT-ClassName = ClassName:\t(?.*)
EXTRACT-MethodName = MethodName:\t(?.*)
EXTRACT-Manufacturer = Manufacturer:\t(?.*)
EXTRACT-Product = Product:\t(?.*)
EXTRACT-Version = Version:\t(?.*)
EXTRACT-ServerName = ServerName:\t(?.*)
EXTRACT-TimeStamp = TimeStamp:\t(?.*)
EXTRACT-UnitOfWork = UnitOfWork:\t(?.*)
EXTRACT-Severity = Severity:\t(?.*)
EXTRACT-Category = Category:\t(?.*)
EXTRACT-PrimaryMessage = PrimaryMessage:\t(?.*)
EXTRACT-ExtendedMessage = ExtendedMessage:\t(?.*)

And the content have a similar pattern

ComponentId:     Application Server
ProcessId:   5316
ThreadId:    00000000
ThreadName:  P=901265:O=0:CT
SourceId:    com.ibm.ws.runtime.WsServerImpl
ClassName:   
MethodName:  
Manufacturer:    IBM
Product:     WebSphere
Version:     Platform 7.0.0.7 [BASE 7.0.0.7 cf070942.55]
ServerName:  sfeserv36Node01Cell\sfeserv36Node01\server1
TimeStamp:   2010-04-27 09:15:57.671000000
UnitOfWork:  
Severity:    3
Category:    AUDIT
PrimaryMessage:  WSVR0001I: Server server1 open for e-business
ExtendedMessage: 
Tags (2)
0 Karma
1 Solution

Simeon
Splunk Employee
Splunk Employee

The suggested extraction for that type of input would be to use a delimiter such as new line in combination with something else. See the DELIMS spec for the config file int transforms.conf:

http://www.splunk.com/base/Documentation/latest/Admin/Transformsconf

I believe you would want to use a combination of the ":" and "\n". So you specify the new line ("\n") to separate the key/value pair and the ":" to separate the key from the value... so something like:

[activity_report]
DELIMS = "\n", ":"

You will need to perform some things before this, which include moving the extraction to occur in the transforms.conf file, instead of the props.conf file. To do this, use/create a TRANSFORMS or REPORT stanza in props.conf that leverages the extraction method through transforms (scroll down to transforms configuration):

http://www.splunk.com/base/Documentation/latest/Admin/Propsconf

View solution in original post

Simeon
Splunk Employee
Splunk Employee

The suggested extraction for that type of input would be to use a delimiter such as new line in combination with something else. See the DELIMS spec for the config file int transforms.conf:

http://www.splunk.com/base/Documentation/latest/Admin/Transformsconf

I believe you would want to use a combination of the ":" and "\n". So you specify the new line ("\n") to separate the key/value pair and the ":" to separate the key from the value... so something like:

[activity_report]
DELIMS = "\n", ":"

You will need to perform some things before this, which include moving the extraction to occur in the transforms.conf file, instead of the props.conf file. To do this, use/create a TRANSFORMS or REPORT stanza in props.conf that leverages the extraction method through transforms (scroll down to transforms configuration):

http://www.splunk.com/base/Documentation/latest/Admin/Propsconf

Nicholas_Key
Splunk Employee
Splunk Employee

Thank you for your helpful hint, Simeon! It works with DELIMS = "\n", ":\t" because first, the lines are separated by a "\n" and second, the key-value pairs are separated by a ":\t".

0 Karma
Get Updates on the Splunk Community!

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...