Getting Data In

How to configure Splunk to prevent line breaking events on ASCII character "#012"?

_smp_
Builder

I have syslog messages arriving at the indexer with embedded ASCII form feed characters (#012). Splunk is breaking on these characters, and I want to avoid this. How can I tell Splunk not to break on these characters?

Here is part of a message for illustration - Splunk is breaking this message into three events, one at each "#012":

Calling-Station-ID#011 value:a8-40-41-14-df-5c#012#011Attribute:CreateTime#011 value:1471359492123#012#011Attribute:DestinationIPAddress#011 value:10.51.17.211#012#011Attribute:Device Identifier#011
0 Karma

twinspop
Influencer

In props.conf on the indexer(s) explicitly define what the line break should be:

[yoursourcetype]
LINE_BREAKER = ([\r\n]+)Calling-Station-ID
SHOULD_LINEMERGE = false

Line breaker needs 1 capturing group. Anything matched in the group will not be indexed. In the case above, newline or carriage return followed by Calling-Station-ID marks the beginning of a new event. The newline/CR is discarded and not indexed.

PPape
Contributor

You could make your own sourcetype in the props.conf

there you can use the parameter
LINE_BREAKER = [regular expression]

like

[yourSourcetype]
LINE_BREAKER=\r\nCalling-Station.*
0 Karma

_smp_
Builder

Thanks for the reply. As I mentioned, the message I noted above is only part of a much larger message. The beginning of the message has a timestamp, which Splunk seems to be identifying correctly. But I have BREAK_ONLY_BEFORE_DATE = true, so I don't understand why Splunk would also break at these #012 characters. Clearly the part of the message before those characters is not a timestamp.

I am pretty green, so I opened a support case asking for some assistance and education, but support has been very unhelpful.

What makes this so complex to me is that these messages are also processed by the Cisco ISE TA. My input applies a sourcetype=syslog, but when I view the messages, they have a sourcetype=cisco:ise:syslog. Being inexperienced, I am not quite clear on how this transition happens. But I can tell you that both my syslog sourcetype and cisco:ise:syslog sourcetypes both have BREAK_ONLY_BEFORE_DATE = true.

0 Karma

PPape
Contributor

You can do it like this:

 [yourSourcetype]
 LINE_BREAKER=(\r\n+)\d{2}-\d{2}-\d{4}

If your event starts like
20-06-2016 .....
as twinspop said. LINE_BREAKER is preferred over BREAK:ONLY_BEFORE_DATE

0 Karma

twinspop
Influencer

I was only using Calling-Station-ID as an example. Using LINE_BREAKER is preferred, in my experience, over BOBD.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...