Getting Data In

Crashplan Service Log Date timestamp incorrect

bmorgenthaler
Path Finder

I've looked through a lot of the posts about date timestamp extraction and I think I'm decent enough at it but for the life of me I can't figure out what is going on with my logs for Crashplan. I found a post with a working example of crashplan service log props and mine matched almost exactly but still no go.

props.conf

[crashplan_service]
TIME_PREFIX = ^\[
MAX_TIMESTAMP_LOOKAHEAD = 21
TIME_FORMAT = %m.%d.%y %H:%M:%S.%3N
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true

inputs.conf

[monitor:///opt/crashplan/log/service.log.0]
source = crashplan
sourcetype = crashplan_service
index = crashplan
disabled = false

Here is one event where the day/month look to be swapped:
1st screenshot of event and incorrect timestamp

With this event I have no idea how it's getting the day/month:
2nd screenshot, wrong date but correct timestamp

0 Karma

bmorgenthaler
Path Finder

Well I've solved my ingestion issues for CrashPlan's service log, now for the other 6. Here is the props stanza I ended up using.

[crashplan_service]
TIME_PREFIX = ^\[
BREAK_ONLY_BEFORE = (\r\n)?\[\d{2}\.\d{2}\.\d{2}\s\d{2}\:\d{2}\:\d{2}\.\d{3}
MAX_TIMESTAMP_LOOKAHEAD = 21
MAX_EVENTS = 1000
NO_BINARY_CHECK = true
TIME_FORMAT = %m.%d.%y %H:%M:%S.%3N

This allowed Splunk to ingest Crashplan configuration output (yay 600+ lines of xml), into a single event.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

I would check if props.conf is located in correct place. It should be placed on the first Splunk Enterprise instance in your data flow. If it's placed on your heavy forwarder/intermediate heavy forwarder/indexer (whichever comes first), then give this configuration a try:

[crashplan_service]
 SHOULD_LINEMERGE = false
 LINE_BREAKER = ([\r\n]+)(?=\[\d+\.\d+\d\.\d+)
 TIME_PREFIX = ^\[
 MAX_TIMESTAMP_LOOKAHEAD = 21
 TIME_FORMAT = %m.%d.%y %H:%M:%S.%3N
0 Karma

bmorgenthaler
Path Finder

@somesoni2 I tried that out and it someone worked but then I found a number of multiline events that didn't play nicely with that. I ended up using EVENT_BREAKER

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...