Splunk Search

How to match the regex for LINE_BREAKER with the end of my event? Yet another one!

hvaithia
Path Finder

My event ends like this,

, "estimatedDuration": 2505189}

The no of digits in the estimated time can be a variable. How do I match this in props.conf using REGEX. I need to add it as a LINE_BREAKER.

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

why are you asking about the 'end of the event'?

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

hvaithia
Path Finder

cos I had used a combination of

SHOULD_LINEMERGE=false
TRUNCATE=200000 ----------- cos my event is greater than 10k bytes
LINE_BREAKER=\"estimatedDuration\":\s\d+}
which didn't work for the event which was above 10k bytes

Also tried the other variant as described in http://docs.splunk.com/Documentation/Splunk/6.2.3/Data/Indexmulti-lineevents

SHOULD_LINEMERGE=true
TRUNCATE=200000 ----------- cos my event is greater than 10k bytes
LINE_BREAKER=\"estimatedDuration\":\s\d+}
BREAK_ONLY_BEFORE={\"building\":
which didn't work for the event which was above 10k bytes as well

I see from the regex you gave that LINE_BREAKER should also include the place where we break/carriage return. I tried your regex as well with no luck. I have also tried the above combination with/without
MAX_EVENTS=2000
with no effect. What am I doing wrong?
PS: I am not able to write "\" (slash) marks in this comment. No idea why. Everything before 's' and 'd' in the LINE_BREAKER and BREAK_ONLY_BEFORE has a slash.

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

you need to mark the code as cold... that's why the escape slashes are being hidden...

LINE_BREAKER and BREAK_ONLY_BEFORE would not be seen together as SHOULD_LINEMERGE must be 'false' for LINE_BREAKER and 'true' for BREAK_ONLY_BEFORE

for more detail you might want to look here.
Less is More here... but without question you need to understand how LINE_BREAKER works.
So I suggest you take a look at this:
http://docs.splunk.com/Documentation/Splunk/6.2.3/Data/indexmulti-lineevents

There are instructions to help you determine exactly how your events are broken up...
You can try TRUNCATE=0 to basically turn it off while you're testing.
Leave MAX_EVENTS where it is and then check the error log to see how many lines there really are so you know where to set it... Splunk will complain and give you facts...

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

LINE_BREAKER = ([\r\n]+)\{"building":\s

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

LINE_BREAKER needs the beginning of the event. Show us the whole thing...

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

hvaithia
Path Finder

The event starts with {"building":
Sample event below
{"building": false, "changeSet": {"items": [], "kind": null}, "builtOn": "rhel6", "description": null, "artifacts": [], "timestamp": 1430241584496, "number": 13, "actions": [{"causes": [{"upstreamBuild": 14, "shortDescription": "Started by upstream project "answers" build number 14", "upstreamProject": "answers", "upstreamUrl": "job/answers/"}]}, {}, {}, {}, {}, {"highlightsData": "[{"Previous Job":"answers#14"},{"Previous Job":"answers_se"},{"Build host":"rhel6"}]", "highlightsTable": "<h4>Global Patterns</h4><b>Previous Job:</b><b>Started by <a href="/hudson/job/answers/14/">answers</a></b><b>Previous Job:</b><b>Started by <a href="/hudson/job/answers/8/">answers #8</a></b><b>Build host:</b><b>Built on <a href="/hudson/computer/rhel6/">rhel6</a></b>"}], "id": "2015-04-28_17-19-44", "keepLog": false, "url": "http://thefactory.xyz.com:9999/jenkins/job/answers/13/", "culprits": [], "result": "SUCCESS", "executor": null, "duration": 377658, "fullDisplayName": "answers", "estimatedDuration": 298415}

0 Karma
Get Updates on the Splunk Community!

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...