Getting Data In

Why are we having an indexing issue in folder monitoring from syslog server?

evelenke
Contributor

Hi Splunkers,

Please help in resolving the following issue.
We have a lot of folder monitoring from syslog server. Each folder contains logs from some of device divided by dates.
One of this logs may have long term pauses in indexing - up to 5 hours. After it starts indexing again the data for previous period is never updated so we have gaps.

alt text

File size is on average 3-4 Gb daily.

In logs I see INFO messages with information that the file has been read:

metrics.log
03-29-2018 14:44:50.336 +0300 INFO  Metrics - group=per_host_thruput, ingest_pipe=1, series="**myhost**", kbps=57.65211637507532, eps=353.7979033990437, kb=1787.2587890625, ev=10968, avg_age=157705281.7769876, max_age=157762827

splunkd.log
03-29-2018 13:37:20.064 +0300 INFO  TailReader - Batch input finished reading file='/..path/**myhost**/2018/2018-03/2018-03-29/2018-03-29_**myhost**.txt'
0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...