Getting Data In

File system monitoring of text files that are overwritten

Michael_Schyma1
Contributor

The beginning and the end of the file are often the same, but we changed the data in the middle of the file, how do we get splunk to pick up the event and re-index it?

ALTERNITIVE: (Same Question)
monitoring of text files that are overwritten and timestamp changes but the files are often the same and-or have the same beginning, end and filename. We would like to force splunk to always re-index the file if it has a new timestamp. (df monitoring of files)

Tags (1)

bmacias84
Champion

If I understand your question correctly there are two setting that might help you accomplish this task.

crcSalt (cycling reducancy checks) which is mostly for log affects log rotation. This setting is located in the inputs.conf.

CHECK_METHOD which is the file check sum configuration. This setting has three configurations: endpoint_md5 (checksum of the first and last 256 bytes), entire_md5 (checksume the entire file), and modtime (checks only modification time of time). This can be found in the props.conf.

Additional Reading:

indexing 101

log file rotation

Hope this helps.

Channu
New Member

@bmacias84 I already have the crcSalt entry in my inputs.conf file.
Should I add the CHECK_METHOD = entire_md5 in the props.conf file? Will that help?

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...