Getting Data In

How can I make splunk universal forwarders treat each newline as a separate event

ilanz
New Member

Hi, my setup is two nodes, each has a Splunk Universal Forwarder which reads a logs directory and sends those logs to a central Splunk indexer.

Let's say I have a log file that looks like this:

4:01:02.316 PM 2012-03-26 16:01:02,3[CLIENT,ALL][Thread:162] INFO - {"JSON1":"hello"}

5:02:02.316 PM 2012-03-28 17:00:02,3[CLIENT,ALL][Thread:162] INFO - {"JSON2":"goodbye"}

In the case above, each line if forwarded as a separate event,

Now, let's say I change my log file format, I'll remove the prefixed timestamps to make it loo like this:

{"JSON1":"hello"}

{"JSON2":"goodbye"}



In that case, splunk treats it as a single event that is if I perform a search I actually get:

{"JSON1":"hello"}{"JSON2":"goodbye"}

How can I configure splunk correctly so it will treat each json entry in my log as a separate event, given my setup usage of universal forwarders.

Thanks beforehand!

0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

Hi, if your events are :
- really separated by a new line (in the file)
- or send to splunk one after the other (syslog by example)

then the solution is to use a sourcetype that define the events are "singleline events"
see SHOULD_LINEMERGE in http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/Indexmulti-lineevents

example

inputs.conf
[monitor:///var/log/json]
sourcetype=myjson

props.conf (on the indexer)
[myjson]
SHOULD_LINEMERGE= false

View solution in original post

0 Karma

yannK
Splunk Employee
Splunk Employee

Hi, if your events are :
- really separated by a new line (in the file)
- or send to splunk one after the other (syslog by example)

then the solution is to use a sourcetype that define the events are "singleline events"
see SHOULD_LINEMERGE in http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/Indexmulti-lineevents

example

inputs.conf
[monitor:///var/log/json]
sourcetype=myjson

props.conf (on the indexer)
[myjson]
SHOULD_LINEMERGE= false

0 Karma

ilanz
New Member

Thanks for your help!

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...