Getting Data In

How to configure Splunk to prevent line breaking events on ASCII character "#012"?

_smp_
Builder

I have syslog messages arriving at the indexer with embedded ASCII form feed characters (#012). Splunk is breaking on these characters, and I want to avoid this. How can I tell Splunk not to break on these characters?

Here is part of a message for illustration - Splunk is breaking this message into three events, one at each "#012":

Calling-Station-ID#011 value:a8-40-41-14-df-5c#012#011Attribute:CreateTime#011 value:1471359492123#012#011Attribute:DestinationIPAddress#011 value:10.51.17.211#012#011Attribute:Device Identifier#011
0 Karma

twinspop
Influencer

In props.conf on the indexer(s) explicitly define what the line break should be:

[yoursourcetype]
LINE_BREAKER = ([\r\n]+)Calling-Station-ID
SHOULD_LINEMERGE = false

Line breaker needs 1 capturing group. Anything matched in the group will not be indexed. In the case above, newline or carriage return followed by Calling-Station-ID marks the beginning of a new event. The newline/CR is discarded and not indexed.

PPape
Contributor

You could make your own sourcetype in the props.conf

there you can use the parameter
LINE_BREAKER = [regular expression]

like

[yourSourcetype]
LINE_BREAKER=\r\nCalling-Station.*
0 Karma

_smp_
Builder

Thanks for the reply. As I mentioned, the message I noted above is only part of a much larger message. The beginning of the message has a timestamp, which Splunk seems to be identifying correctly. But I have BREAK_ONLY_BEFORE_DATE = true, so I don't understand why Splunk would also break at these #012 characters. Clearly the part of the message before those characters is not a timestamp.

I am pretty green, so I opened a support case asking for some assistance and education, but support has been very unhelpful.

What makes this so complex to me is that these messages are also processed by the Cisco ISE TA. My input applies a sourcetype=syslog, but when I view the messages, they have a sourcetype=cisco:ise:syslog. Being inexperienced, I am not quite clear on how this transition happens. But I can tell you that both my syslog sourcetype and cisco:ise:syslog sourcetypes both have BREAK_ONLY_BEFORE_DATE = true.

0 Karma

PPape
Contributor

You can do it like this:

 [yourSourcetype]
 LINE_BREAKER=(\r\n+)\d{2}-\d{2}-\d{4}

If your event starts like
20-06-2016 .....
as twinspop said. LINE_BREAKER is preferred over BREAK:ONLY_BEFORE_DATE

0 Karma

twinspop
Influencer

I was only using Calling-Station-ID as an example. Using LINE_BREAKER is preferred, in my experience, over BOBD.

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...