Splunk Search

Need to break the events before certain word.

isha_rastogi
Path Finder

I've log file something like below,

DA FILE: /archive/attr/ABC/XYZ/20170911/file.log-new*** Files traversed: 128 - and  file format matches ***
DA FILE: /archive/attr/ABC/XYZ/20170911/file.log-newCreating the follwing directory for today's date: /mktaccess/archive/attr/abc/xyz/20170911Creating the follwing directory for today's date: /mktaccess/archive/attr/abc/xyz/20170911Creating the follwing directory for today's date: /mktaccess/archive/hello-US/abc/xyz/20170911Creating the follwing directory for today's date: /mktaccess/archive/hello-US/abc/xyz/20170911

Log format is like a paragraph.. It's the stdout and stderr output of a script
I need to break the line after ******* and before Creating. I'm writing something like below
[test]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=Creating
MUST_BREAK_AFTER=^\d{8}
TIME_FORMAT=%Y%m%d
TIME_PREFIX=(OR\w*\/)+

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

If this works for you, I can change this comment to an answer and then you can accept it so that it can be marked as solved. Thanks!

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

Your question appears to have had formatting changes that are making answering it difficult. Please highlight your example text and use the 101010 formatting option the help make the text come through properly.

0 Karma

isha_rastogi
Path Finder

@cpetterborg I've made necessary changes, please take a look

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

With the data you have, you can't break up the line into different events without loosing some data, and even that has some issues.

If you don't mind doing it at search time, instead of at index time, it can be done pretty easily. You can do something like this:

... | rex mode=sed "s/(\*\*|Creating)/|\\1/g"
| rex max_match=0 "\|(?P<part>[^\|]+)"
| mvexpand part
| table _time, part

You may add any additional fields to the table, but it worked on the example data that you provided to break the event up into multiple events for further processing.

Splunk doesn't like to split up lines like that. If the data coming in had a character (like I used the |), then you could break it up at index time, and only loose the |.

Get Updates on the Splunk Community!

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...

Splunk APM: New Product Features + Community Office Hours Recap!

Howdy Splunk Community! Over the past few months, we’ve had a lot going on in the world of Splunk Application ...

Index This | Forward, I’m heavy; backward, I’m not. What am I?

April 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...