Deployment Architecture

Splunk is not correctly splitting Syslog and JSON messages by new line

daniel_splunk
Splunk Employee
Splunk Employee

We are experiencing issues with various logging sources where messages are not being correctly split by new line and therefore are resulting in a linecount of greater than 1. This is being experienced for our indexes security-apigw (syslog) and security-carbonblack (json).

I updated the props.conf to incude the following. However, some of the event still not split correctly.
SHOULD_LINEMERGE = false
LINE_BREAKER = }}([\r\n]+)

Tags (1)
0 Karma

daniel_splunk
Splunk Employee
Splunk Employee

The “INDEXED_EXTRACTION” came out in 6.x changes a few things on how Splunk works. Its used for well known log sources (IIS, JSON, XML) to have their fields parsed out by the forwarder, moving the work from the indexer to the forwarder.

So you need to install those props on the forwarder, not the indexer. When INDEXED_EXTRACTIONs is enabled, it will pass the parsed fields into the indexers. The indexers will not apply any parsing rules for data that was processed as INDEXED_EXTRACTIONs.

You can add the following into your forwarder configuration.

props.conf
[carbonblack:json]
INDEXED_EXTRACTIONS=json

Here is the doc for more information.
http://docs.splunk.com/Documentation/Splunk/6.5.2/Data/Extractfieldsfromfileswithstructureddata

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...