Getting Data In

How to remove an invalid line breaker from syslog before indexing?

vlours
Explorer

Hi everyone,

I've got an application sending data to splunk, which are split over multiple lines instead to keep everything on the same line.

When I redirect my data to a file instead of splunk, I can find that the ascii code #012 is sent as part of the string.

Example:

... #012Change details       : #012filewrite#012 ...

Which are split in multiples lines in splunk:

...
9/29/16 3:25:30.000 AM  filewrite
host = xxx.xxx.xxx.xxx source = udp:3514 sourcetype = syslog
9/29/16 3:25:30.000 AM  Change details       :
host = xxx.xxx.xxx.xxx source = udp:3514 sourcetype = syslog
...

Is there any way to replace the ASCII code #012 before to index it into splunk ?

I've try to add this config in my props.conf, but it did not solved it.

[syslog]
LINE_BREAKER=#012
SHOULD_LINEMERGE=true

And also this one:

[syslog]
SEDCMD-fim = s/\#012/ /g

Thanks for your support.

0 Karma
1 Solution

vlours
Explorer

As explain by @somesoni2, the #012 is is ascii code for \n

So to solve it, I've include the following in my props.conf:
[syslog]
LINE_BREAKER=([\r\n]+)
SHOULD_LINEMERGE=true

My data are now merged correctly in one line.

View solution in original post

0 Karma

vlours
Explorer

As explain by @somesoni2, the #012 is is ascii code for \n

So to solve it, I've include the following in my props.conf:
[syslog]
LINE_BREAKER=([\r\n]+)
SHOULD_LINEMERGE=true

My data are now merged correctly in one line.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

The #012 is ascii code for \n, so that is why events are getting split (default LINE_BREAKER is ([\r\n]+)). You second configuration with SEDCMD will not work as SEDCMD executes after events are broken.

How does your data looks like when you redirect it to a file (sample entries, mask any sensitive data)? You probably have to setup correct LINE_BREAKER (assuming you're data is directly coming to indexer/heavy forwarder) to split it correctly.

vlours
Explorer

Hi somesoni2,

Actually, I did not test it with the default LINE_BREAKER, as I was fighting with my #012 value.
Setting the default value LINE_BREAKER and the SHOULD_LINEMERGE is working fine.

So, I've put the following in my props.conf:
[syslog]
LINE_BREAKER=([\r\n]+)
SHOULD_LINEMERGE=true

Thanks a lot.
Have a great day.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...