Getting Data In

props.conf Line Breaking

Svill321
Path Finder

Hello everyone,

I have several events with different time stamps that I'm trying to breakup. The props file I'm using is as follows:

sourcetype=applogs
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=false
BREAK_ONLY_BEFORE = ^(\d{2}\:\d{2}\:\d{2}\||\d{4}\-\d{2}\-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}|\d\/\d{2}\/\d{4}\s+?\d{2}:\d{2}:\d{2}:)
disabled = false
pulldown_type = true

Alternatively, I also use:

sourcetype=applogs
SHOULD_LINEMERGE=false
LINE_BREAKER = ([\r\n]+)( \d{2}\:\d{2}\:\d{2}\||\d{4}\-\d{2}\-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}|\d\/\d{2}\/\d{4}\s+?\d{2}:\d{2}:\d{2}:)
disabled = false
pulldown_type = true

The logs look like this:

14:49:27| stuff
14:49:27|stuff
14:49:28|stuff

Another log looks like
7/17/2017 17:02:12:stuff
7/17/2017 17:02:12:stuff
7/17/2017 17:02:12:stuff

And yet another log looks like this:

7/17/2017 17:01:58.345, stuff
7/17/2017 17:01:58.355, stuff
7/17/2017 17:01:58.376, stuff

Could someone let me know what I'm doing wrong in my prop.conf files? For some reason they're not breaking up the file

0 Karma

fredzhang
New Member

Suggest use different stanzas with source::.... to do line break for each different type logs.
E.g. for each source:

[source::...source1...]
BREAK_ONLY_BEFORE_DATE = true
DATETIME_CONFIG = none
TIME_FORMAT = XXXX // set as log shows
...

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi Svill321,

here is a quick check list:

  1. What are you applying this on? source, sourcetype or host? Check for typos
  2. Where is this props.conf applied? Must be on the parsing layer, either heavyweight forwarder or indexer
  3. Check with $SPLUNK_HOME/bin/splunk btool props list if your config on the parsing instance is applied correctly
  4. your first regex does not match, try this \d{2}\:\d{2}\:\d{2}\||\d\/\d{2}\/\d{4}\s+\d{2}:\d{2}:\d{2}:|\d\/\d{2}\/\d{4}\s+\d{2}:\d{2}:\d{2}\.\d{3},
  5. BREAK_ONLY_BEFORE only works if Splunk encounters a new line that matches the regular expression.
  6. same with the second regex, try this instead `([\r\n]+)\d{2}:\d{2}:\d{2}||\d\/\d{2}\/\d{4}\s+\d{2}:\d{2}:\d{2}:|\d\/\d{2}\/\d{4}\s+\d{2}:\d{2}:\d{2}.\d{3},
  7. did you restart the parsing Splunk instance after the changes?
  8. This will only be applied to new arriving events, it is not applied to any existing already indexed events

Hope this helps to trace down the error ...

cheers, MuS

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...