Getting Data In

Issue with monitoring one specific log file

AKG1_old1
Builder

Hi,

I am monitoring multiple files/directory under different sourcetype. For one specific log file I am getting wiered behavior.
It's not being monitored Continuously, even though file is getting updated regularly.

I am not getting any relevant error at both Splunk and forwarder side.

Whenever I install new forwarder and configure this file to read, file is being picked only once and stop updating . (It's like reading a batch file)

inputs.conf

[monitor:///net/hp707srv/hp707srv2/apps/QCST_RSAT_v3.1.42_MASTER/qcstTools/qcst_out_alerts.log]
disabled = false
host = MTE_TEST
index = mlc_live
sourcetype = MTE_ALERT
crcSalt = <Source> 
0 Karma
1 Solution

AKG1_old1
Builder

Issue is resolved by updating TIME_FORMAT In props.conf
Earlier TIME_FORMAT was not defined. but wiered thing is it was working fine initially for a month with no TIME_FORMAT . My assumption is if its not defined it takes current time bydefault.

props.conf
[MTE_ALERT]
DATETIME_CONFIG =
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Custom
disabled = false
pulldown_type = true
REPORT-MTE_ALERT = REPORT-MTE_ALERT
TIME_FORMAT = %d/%m/%Y | %H:%M:%S
TIME_PREFIX = ^

View solution in original post

0 Karma

AKG1_old1
Builder

Issue is resolved by updating TIME_FORMAT In props.conf
Earlier TIME_FORMAT was not defined. but wiered thing is it was working fine initially for a month with no TIME_FORMAT . My assumption is if its not defined it takes current time bydefault.

props.conf
[MTE_ALERT]
DATETIME_CONFIG =
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Custom
disabled = false
pulldown_type = true
REPORT-MTE_ALERT = REPORT-MTE_ALERT
TIME_FORMAT = %d/%m/%Y | %H:%M:%S
TIME_PREFIX = ^

0 Karma

lakshman239
SplunkTrust
SplunkTrust

When you define monitor stanza (the others in your inputs.conf in the UF/HF), are you ensuring that no other stanza is resolving to the above path ///net/hp707srv/hp707srv2/apps/QCST_RSAT_v3.1.42_MASTER/qcstTools/ ?

Also, how often does this file get updated and rotated? did you try crcSalt /crc checksum length?

AKG1_old1
Builder

I have tried installing fresh forwarder for monitoring only this file. After starting the forwarder full file injested in Splunk but later on it's not getting updated.

I have used crcSalt = as well but didn't work.

Around 30-50 lines are updated in one hour.

0 Karma

lakshman239
SplunkTrust
SplunkTrust

Assuming, you get new events every 1hr, are you seeing any warning/errors in splunkd.log from the time your file is first indexed to say till next 1 or 2 hrs? [ e.g file crc checksum error, file ignored, parsing error]. Also, using the metrics.log, can you check if you are constantly receiving other _internal logs from the host, so we can isolate the issue to only this specific file. I assume this is a normal text file.

AKG1_old1
Builder

@lakshman239 : Thanks for help. its got resovled.

0 Karma

Laszlo_K
Explorer

Have you considered crcSalt as described in https://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf ?

AKG1_old1
Builder

yesI have tried with

 crcSalt = <Source> 
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...