Getting Data In

splunk ignoring LINE_BREAKER

phemmer
Path Finder

I've configured a source type in props.conf with LINE_BREAKER = (\n+) to remove the \r from the default value. This was done so that we can send multi-line events using \r as the delimiter between lines, and \n as the delimiter between events.
The problem however is that splunk is still breaking messages into multiple events on the \r character.

Any ideas why?

Here's an example.
This screenshot is the tcpdump output of a 2-line message, where the second line should be this is line 2.

Here's the event as shown by the search app.

Here's the active sourcetype configuration:

The etc/system/local/props.conf on the indexer nodes:

[syslog]
LINE_BREAKER = (\n+)
0 Karma
1 Solution

phemmer
Path Finder

Figured it out.
We had an app which was doing transforms on syslog sources. However the regex it was using used .* to match all data, but . doesn't match carriage return (by default).
I tweaked the regex the app uses to add (?s) to set the regex flag letting it match.

View solution in original post

0 Karma

phemmer
Path Finder

Figured it out.
We had an app which was doing transforms on syslog sources. However the regex it was using used .* to match all data, but . doesn't match carriage return (by default).
I tweaked the regex the app uses to add (?s) to set the regex flag letting it match.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

The sourcetype definition in your screenshot does not match what you list under etc/system/local/props.conf on the indexer nodes!? Looks like you have defined the sourcetype in the app called "AppsBrowser", can you share the props.conf file from etc/apps/appsBrowser/local, please?

phemmer
Path Finder

@alemarzu sorry, looks like the screenshots broke. I uploaded them to an image host.

@ssievert: I don't know why that shows "AppsBrowser". I've never touched that. There is no etc/appsbrowser/local, and etc/appsbrowser/default has no props.conf.

@richgalloway: That is the default, so I just left it alone for now. However if SHOULD_LINEMERGE were causing a problem, there would be too many lines in a single event, which is the opposite of my problem.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The Admin manual says LINE_BREAKER should only be used when SHOULD_LINEMERGE=false. It also says BREAK_ONLY_BEFORE only applies if SHOULD_LINEMERGE=true so it doesn't make sense to have both LINE_BREAKER and BREAK_ONLY_BEFORE in the same sourcetype. Also, while LINE_BREAKER requires a capturing group there is no explicit requirement for a capturing group in BREAK_ONLY_BEFORE.

---
If this reply helps you, Karma would be appreciated.

alemarzu
Motivator

Hi there phemmer,

Would you mind to paste the sample data ?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...