Getting Data In

Can I view related parts of a multiline log entry?

ripper234
Explorer

Some of the long entries my app makes are composed of multiple lines.
I would like to keep it this way (a log line can be 10+ lines sometimes, has lots of info that can't be condensed into a single line).

Is there a way, given a specific text match, to view text lines near it?

E.g. supposed that my log format looks like this:

2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Now follow a few important details.
A few more details.
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

If I search in splunk for "foo.com", and only find the first line without the details - how can I view the rest of the details?

I prefer a solution that won't force me to change the format of my log messages.

Tags (1)
0 Karma
1 Solution

emiller42
Motivator

Are the multi-line events being broken into multiple events in Splunk?

Event 1:
2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Event 2:
Now follow a few important details.
Event 3:
A few more details.
Event 4:
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

If so, you can fix that where it keeps the whole thing as one event.

In your props.conf, you want to add the following config to the appropriate stanza for this sourcetype:

LINE_BREAKER = ([\r\n]+)\d+-\d+-\d+\s\d+:\d+:\d+,\d+ 
SHOULD_LINEMERGE = false

This tells it not to automatically break events, and instead only break when it encounters a new line starting with a timestamp. (What the regex matches)

Then when new log lines are indexed, multi-line events will be kept as a single event, keeping the context you want:

Event 1:
2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Now follow a few important details.
A few more details.
Event 2:
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

View solution in original post

emiller42
Motivator

Are the multi-line events being broken into multiple events in Splunk?

Event 1:
2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Event 2:
Now follow a few important details.
Event 3:
A few more details.
Event 4:
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

If so, you can fix that where it keeps the whole thing as one event.

In your props.conf, you want to add the following config to the appropriate stanza for this sourcetype:

LINE_BREAKER = ([\r\n]+)\d+-\d+-\d+\s\d+:\d+:\d+,\d+ 
SHOULD_LINEMERGE = false

This tells it not to automatically break events, and instead only break when it encounters a new line starting with a timestamp. (What the regex matches)

Then when new log lines are indexed, multi-line events will be kept as a single event, keeping the context you want:

Event 1:
2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Now follow a few important details.
A few more details.
Event 2:
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

ripper234
Explorer

Interesting direction, I think it will do the trick. I think the lines are all separate events.

Will try it out when I get the chance - thanks for the quick answer!

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...