Getting Data In

timeparsing issue for sourcetype

juhisaxena28
Explorer

I have data like below:-

Log file created at: 2019/03/24 17:56:14
Running on machine: F8976GMac
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0324 17:56:14.700251 16884 server.cpp:116] - 144.77.88.67
I0324 17:56:14.747052 16884 server.cpp:118] - Listening at : 28888, 28889, 28890

and i am getting following Warnings in _internal logs:-

WARN DateParserVerbose - Accepted time (Wed Jul 6 07:27:19 2016) is suspiciously far away from the previous event's time (Fri Apr 5 07:27:19 2019), but still accepted because it was extracted by the same pattern.

Please let know what timeparsing config fix can be made here since i see the same log file prints the complete timestamp at the start only and then only time is getting printed.

0 Karma

woodcock
Esteemed Legend

Try this in props.conf on the first full instance of Splunk that handles the events (usually Heavy Forwarder or Indexer tier):

[YourSourcetypeHere]
TIME_PREFIX = "Log file created at:\s*"
TIME_FORMAT = %Y%m/%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19
LINE_BREAKER = (?!)
SHOULD_LINEMERGE = false
TRUNCATE = 99999

This makes every line in the file a single event.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi,

Please try below config on Indexer/Heavy Forwarder(whichever comes first from UF).

props.conf

[yoursourcetype]
TIME_FORMAT=%Y/%m/%d %H:%M:%S
TIME_PREFIX=created\sat\:\s
MAX_TIMESTAMP_LOOKAHEAD=19
0 Karma

juhisaxena28
Explorer

This is further breaking events since 256 char limits have exceeded.

The other events are giving warnings as they failed to parse timestamp.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

In this case you can add below settings in above props.conf or you can adjust below settings based your actual event size and length.

MAX_EVENTS = 1000
TRUNCATE = 50000
0 Karma

juhisaxena28
Explorer

Applying these settings is breaking the events because limit of 256 characters has breached. Thus the other events produced are also giving the warning of failure in parsing timestamp in MAX_TIMESTAMP_LOOKAHEAD. The two sample events created are following:-

Event1-
"Log file created at: 2019/03/24 17:56:14
Running on machine: F8976GMac
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0324 17:56:14.700251 16884 server.cpp:116] - 144.77.88.67
I0324 17:56:14.747052 16884 server.cpp:118] - Listening at : 28888, 28889, 28890"

Event2-
"I0324 18:01:01.517282 18392 client_networker.cpp:55] -> Heartbeat_Indication(419), 42 bytes.
I0324 18:01:01.517282 18392 message_sender.cpp:733] <-- Heartbeat_Indication(419), 42 bytes.
I0324 18:01:04.606200 18412 networker.cpp:504] -> User_Kernel_Status_Request(1105)
I0324 18:01:04.606200 18412 networker.cpp:523] <- (1205), 116 bytes.
I0324 18:01:06.384670 18392 client_networker.cpp:55] -> Heartbeat_Indication(419), 42 bytes."

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...