I would like to know if there is a way to generalize the following EXTRACT regexes in my props.conf? The configuration looks like the following:
[activityLog] LINE_BREAKER = [-]{8,}([\r\n]+) SHOULD_LINEMERGE = false EXTRACT-ComponentId = ComponentId:\t(?.*) EXTRACT-ProcessId = ProcessId:\t(?.*) EXTRACT-ThreadId = ThreadId:\t(?.*) EXTRACT-ThreadName = ThreadName:\t(?.*) EXTRACT-SourceId = SourceId:\t(?.*) EXTRACT-ClassName = ClassName:\t(?.*) EXTRACT-MethodName = MethodName:\t(?.*) EXTRACT-Manufacturer = Manufacturer:\t(?.*) EXTRACT-Product = Product:\t(?.*) EXTRACT-Version = Version:\t(?.*) EXTRACT-ServerName = ServerName:\t(?.*) EXTRACT-TimeStamp = TimeStamp:\t(?.*) EXTRACT-UnitOfWork = UnitOfWork:\t(?.*) EXTRACT-Severity = Severity:\t(?.*) EXTRACT-Category = Category:\t(?.*) EXTRACT-PrimaryMessage = PrimaryMessage:\t(?.*) EXTRACT-ExtendedMessage = ExtendedMessage:\t(?.*)
And the content have a similar pattern
ComponentId: Application Server ProcessId: 5316 ThreadId: 00000000 ThreadName: P=901265:O=0:CT SourceId: com.ibm.ws.runtime.WsServerImpl ClassName: MethodName: Manufacturer: IBM Product: WebSphere Version: Platform 7.0.0.7 [BASE 7.0.0.7 cf070942.55] ServerName: sfeserv36Node01Cell\sfeserv36Node01\server1 TimeStamp: 2010-04-27 09:15:57.671000000 UnitOfWork: Severity: 3 Category: AUDIT PrimaryMessage: WSVR0001I: Server server1 open for e-business ExtendedMessage:
The suggested extraction for that type of input would be to use a delimiter such as new line in combination with something else. See the DELIMS spec for the config file int transforms.conf:
http://www.splunk.com/base/Documentation/latest/Admin/Transformsconf
I believe you would want to use a combination of the ":" and "\n". So you specify the new line ("\n") to separate the key/value pair and the ":" to separate the key from the value... so something like:
[activity_report]
DELIMS = "\n", ":"
You will need to perform some things before this, which include moving the extraction to occur in the transforms.conf file, instead of the props.conf file. To do this, use/create a TRANSFORMS or REPORT stanza in props.conf that leverages the extraction method through transforms (scroll down to transforms configuration):
http://www.splunk.com/base/Documentation/latest/Admin/Propsconf
The suggested extraction for that type of input would be to use a delimiter such as new line in combination with something else. See the DELIMS spec for the config file int transforms.conf:
http://www.splunk.com/base/Documentation/latest/Admin/Transformsconf
I believe you would want to use a combination of the ":" and "\n". So you specify the new line ("\n") to separate the key/value pair and the ":" to separate the key from the value... so something like:
[activity_report]
DELIMS = "\n", ":"
You will need to perform some things before this, which include moving the extraction to occur in the transforms.conf file, instead of the props.conf file. To do this, use/create a TRANSFORMS or REPORT stanza in props.conf that leverages the extraction method through transforms (scroll down to transforms configuration):
http://www.splunk.com/base/Documentation/latest/Admin/Propsconf
Thank you for your helpful hint, Simeon! It works with DELIMS = "\n", ":\t" because first, the lines are separated by a "\n" and second, the key-value pairs are separated by a ":\t".