I've looked through a lot of the posts about date timestamp extraction and I think I'm decent enough at it but for the life of me I can't figure out what is going on with my logs for Crashplan. I found a post with a working example of crashplan service log props and mine matched almost exactly but still no go.
props.conf
[crashplan_service]
TIME_PREFIX = ^\[
MAX_TIMESTAMP_LOOKAHEAD = 21
TIME_FORMAT = %m.%d.%y %H:%M:%S.%3N
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true
inputs.conf
[monitor:///opt/crashplan/log/service.log.0]
source = crashplan
sourcetype = crashplan_service
index = crashplan
disabled = false
Here is one event where the day/month look to be swapped:
With this event I have no idea how it's getting the day/month:
Well I've solved my ingestion issues for CrashPlan's service log, now for the other 6. Here is the props stanza I ended up using.
[crashplan_service]
TIME_PREFIX = ^\[
BREAK_ONLY_BEFORE = (\r\n)?\[\d{2}\.\d{2}\.\d{2}\s\d{2}\:\d{2}\:\d{2}\.\d{3}
MAX_TIMESTAMP_LOOKAHEAD = 21
MAX_EVENTS = 1000
NO_BINARY_CHECK = true
TIME_FORMAT = %m.%d.%y %H:%M:%S.%3N
This allowed Splunk to ingest Crashplan configuration output (yay 600+ lines of xml), into a single event.
I would check if props.conf is located in correct place. It should be placed on the first Splunk Enterprise instance in your data flow. If it's placed on your heavy forwarder/intermediate heavy forwarder/indexer (whichever comes first), then give this configuration a try:
[crashplan_service]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\[\d+\.\d+\d\.\d+)
TIME_PREFIX = ^\[
MAX_TIMESTAMP_LOOKAHEAD = 21
TIME_FORMAT = %m.%d.%y %H:%M:%S.%3N
@somesoni2 I tried that out and it someone worked but then I found a number of multiline events that didn't play nicely with that. I ended up using EVENT_BREAKER