Getting Data In

events are broken in the middle of the line

mataharry
Communicator

I am monitoring with a forwarder logs file that are being written.
And sometimes the events indexed are broken in multiple events in the middle of the line :

example :
Thu Sep 19 17:13:32 PDT 2013 myhost my event and the end of my line
becomes
event1 :
Thu Sep 19 17:13:32 PDT 2013 myhost my event and
event2
the end of my line

1 Solution

yannK
Splunk Employee
Splunk Employee

If the event is but in the middle of the line, it can be a write buffer issue.
by example if the End of Line is reached when Splunk is reading the file.

To confirm, you can index the same log file separately in a test index, and see if the events are properly indexed with a static file. If this is confirmed, the option is to increase the time_before_close on the inputs.conf

see time_before_close =
* Modtime delta required before Splunk can close a file on EOF.
* Tells the system not to close files that have been updated in past seconds.
* Defaults to 3.
http://docs.splunk.com/Documentation/Splunk/latest/admin/Inputsconf

View solution in original post

yannK
Splunk Employee
Splunk Employee

If the event is but in the middle of the line, it can be a write buffer issue.
by example if the End of Line is reached when Splunk is reading the file.

To confirm, you can index the same log file separately in a test index, and see if the events are properly indexed with a static file. If this is confirmed, the option is to increase the time_before_close on the inputs.conf

see time_before_close =
* Modtime delta required before Splunk can close a file on EOF.
* Tells the system not to close files that have been updated in past seconds.
* Defaults to 3.
http://docs.splunk.com/Documentation/Splunk/latest/admin/Inputsconf

mataharry
Communicator

I confirm the application generating the log file is writing by chunks.
It doesn't happen if I index the file after.

0 Karma
Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...