Getting Data In

events are broken in the middle of the line

mataharry
Communicator

I am monitoring with a forwarder logs file that are being written.
And sometimes the events indexed are broken in multiple events in the middle of the line :

example :
Thu Sep 19 17:13:32 PDT 2013 myhost my event and the end of my line
becomes
event1 :
Thu Sep 19 17:13:32 PDT 2013 myhost my event and
event2
the end of my line

1 Solution

yannK
Splunk Employee
Splunk Employee

If the event is but in the middle of the line, it can be a write buffer issue.
by example if the End of Line is reached when Splunk is reading the file.

To confirm, you can index the same log file separately in a test index, and see if the events are properly indexed with a static file. If this is confirmed, the option is to increase the time_before_close on the inputs.conf

see time_before_close =
* Modtime delta required before Splunk can close a file on EOF.
* Tells the system not to close files that have been updated in past seconds.
* Defaults to 3.
http://docs.splunk.com/Documentation/Splunk/latest/admin/Inputsconf

View solution in original post

yannK
Splunk Employee
Splunk Employee

If the event is but in the middle of the line, it can be a write buffer issue.
by example if the End of Line is reached when Splunk is reading the file.

To confirm, you can index the same log file separately in a test index, and see if the events are properly indexed with a static file. If this is confirmed, the option is to increase the time_before_close on the inputs.conf

see time_before_close =
* Modtime delta required before Splunk can close a file on EOF.
* Tells the system not to close files that have been updated in past seconds.
* Defaults to 3.
http://docs.splunk.com/Documentation/Splunk/latest/admin/Inputsconf

mataharry
Communicator

I confirm the application generating the log file is writing by chunks.
It doesn't happen if I index the file after.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...