How to configure props.conf using BREAK_ONLY_BEFORE?

[dturner83]

I have the following data as a text file. Each event should run from the Date field until the next date field.

I'm using a universal forwarder to send this data to a heavy forwarder and then on to the indexer. The events either all break or none break depending if I have anything or nothing in the props.conf on the heavy forwarder but I never get event breaking before Date.

Can someone help out here? It appears I need some help with the BREAK_ONLY_BEFORE option.

The following are my props.conf files for the heavy forwarder and indexer/search head.

[sampleoutput]
# your settings
BREAK_ONLY_BEFORE=^\s*Date                                           
NO_BINARY_CHECK=1
SHOULD_LINEMERGE = TRUE

Each item below is on a new line, but should be in the same event, until the Date field shows up again.

Date = 6/24/2014
Ad = item add #1
Description line 1 = Something Good
Description line 2 = Somethinggood2.
Display URL = example.com/somethinggood
Destination URL = http=//example.com
Campaign = Campaign1
Campaign type = Search Only
Campaign subtype = All features
Ad group = addgroup1
Status = disapproved
Clicks = 0
Impressions = 0
CTR = 0.00%
Avg. CPC = 0
Cost = 0
Avg. position = 0
Converted clicks = 0
Cost / converted click = 0
Click conversion rate = 0.00%

Date = 6/24/2014
Ad = item add #2
Description line 1 = Something good
Description line 2 = Something good 2
Display URL = example.com/somethingood
Destination URL = http=//example.com
Campaign = campaign2
Campaign type = Search Only
Campaign subtype = All features
Ad group = addgroup2
Status = disapproved
Clicks = 0
Impressions = 0
CTR = 0.00%
Avg. CPC = 0
Cost = 0
Avg. position = 0
Converted clicks = 0
Cost / converted click = 0
Click conversion rate = 0.00%

[martin_mueller]

This is a slightly different approach, but you should be able to use this:

LINE_BREAKER = ([\r\n]+)\s*Date
SHOULD_LINEMERGE = false

Should be much faster as well 😄

[martin_mueller]

Yeah, any indextime change to configuration files requires a restart, no matter if it's a HF or Indexer.

[dturner83]

This took care of it. I put this in the props.conf on the heavy forwarder and it didn't change, but then I restarted splunk on the heavy forwarder and it worked like a champ. That now makes me wonder if my other changes would have worked too but I'll take your faster approach 🙂

[dturner83]

Yes the sourcetype matches, the regex for ^\s*Date is something I've tried as well as ^Date and Date itself.

[MuS]

does the sourcetype match and does this regex match? you're using a regex that will match 0 or none spaces at the beginning of the string followed by Date. Did you try to use only BREAK_ONLY_BEFORE=Date ?