Splunk Search

regex and BREAK_ONLY_BEFORE

Branden
Builder

I have a script that sends something like the following to stdout:

DEV#:    0    DEVICE NAME: vpath0    TYPE: 2107900    POLICY: Optimized
SERIAL: 123bac
=======================================================================
Path#         Adapter/Hard Disk       State    Mode      Select  Errors
   0             fscsi0/hidsk22       Open     NORMAL    123456       0
   1             fscsi0/hidsk29       Open     NORMAL    456789       0

DEV#:    1    DEVICE NAME: vpath1    TYPE: 2107900    POLICY: Optimized
SERIAL: 123bac
=======================================================================
Path#         Adapter/Hard Disk       State    Mode      Select  Errors
   0             fscsi0/hidsk21       Open     NORMAL    123456       0
   1             fscsi0/hidsk28       Open     NORMAL    456789       0

etc... This list goes on for as many vpaths as there are on the host.

I'd like to capture this via Splunk, but I need to split it up by vpath entry; I don't want the entire output to be one big entry.

The script runs from $PLUNK_HOME/etc/apps/fc/bin/foo.sh on the forwarder and gets indexed as sourcetype "datapath-device" I added a props.conf on the forwarder in $SPLUNK_HOME/etc/apps/fc/local with the following entry:

[datapath-device]
BREAK_ONLY_BEFORE = ^DEV#:\s+\d+\s\w+\s\w+:\s\w\s+\w+:\s\w+\s+\w+:\w+

Unfortunately, it's not breaking up the output. Instead I get one huge entry with info on all the vpaths. I'm not sure if my regular expression is wrong or if props.conf is in the wrong place.

Any suggestions are appreciated.

Thanks!

Tags (1)
1 Solution

Brian_Osburn
Builder

You need to update the props.conf in $SPLUNK_HOME/etc/system/local on the indexer, not the forwarder..

I was able to use the following entry (replace TEST with what your sourcetype is)

[test]

BREAK_ONLY_BEFORE = ^DEV#:

Edit: Also, you may have to re-index the data to pick up the changes...

Brian

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

Yeah, as Brian Osburn says, this configuration must go where-ever the parsing phase occurs. See here.

I notice there are also no timestamps, so you might want to add a line:

DATETIME_CONFIG = CURRENT

(or some other more complicated config if you need to get it from the file name)

gkanapathy
Splunk Employee
Splunk Employee

You're not going to be able to do this very well with search-time regex. What you probably need is the "multikv" command. You might need to do a change on _raw (using eval _raw=replace(_raw,... and/or rex first) to clear out the header after extracting useful information from those. Actually it might be better to post this as another question, it needs fuller answer than I have room for here.

Branden
Builder

Actually, as a follow-up question to Brian and gkanapthy...
I have those two lines as part of the output:
0 fscsi0/hdisk1 OPEN NORMAL 123456 0
1 fscsi1/hdisk2 OPEN NORMAL 345667 0

I'd like to create a regular expression that pulls out the fields from the first line, then a regular expression to pull the fields from the second line (though the fields would have slightly different names from one line to another). In other words, I'd like a path_num0 and a path_num1 field. How do I use regex to distinguish one line from the next?
Thanks!

0 Karma

Brian_Osburn
Builder

You need to update the props.conf in $SPLUNK_HOME/etc/system/local on the indexer, not the forwarder..

I was able to use the following entry (replace TEST with what your sourcetype is)

[test]

BREAK_ONLY_BEFORE = ^DEV#:

Edit: Also, you may have to re-index the data to pick up the changes...

Brian

Branden
Builder

Worked beautifully, thanks!!!

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...