How to configure proper line breaking in props.conf on the universal forwarder for my sample data?

[horsefez]

Hi beloved Splunkers,

I'm currently trying to set up a data connection between one of our servers and my Splunk deployment. Unfortunately, I encountered some problems when it comes to Splunk recognizing line-endings and -beginnings.
Let's take a closer look at my current problem.

I have a data file with events that look kinda like that:

<666> this, is, the, event, number, 1,
<666> this, is, the, event, number, 2,
<666> this, is, the, event, number, 3, but, it, is, slightly, longer, than, the, others,
<666> this, is, the, event, number, 4,
<666> splunk, fast, like, a, f-18, bro,
<666> this, is, the, event, number, 6,

What you can see here is, that all those events have something in common.
Yeah, its the "< 666 >" part.

Splunk is flawless I give you that, but for some reason, it sometimes combines two single events into one.

So I was thinking that I need to configure a stanza in props.conf on the forwarder to tell splunk how to deceide when a new event starts.
I did write one, but failed.... maybe?!?

[source::/path/to/file/]
BREAK_ONLY_BEFORE = (\<\d+\>)
SHOULD_LINEMERGE = True

I would love to know if someone out there is brave enough to tell me the right solution.
Thank you for your help, bro/sis!

Regards,
pyro_wood

---

Splunk> like a F-18, bro ♥

[skoelpin · Accepted solution]

You have the correct approach, but you need to edit your props.conf on the indexer rather than the forwarder. Don't forget to restart your Splunkd Service after making these changes!

[horsefez]

Quick answer. I like it. I'll try.
Thanks skoelpin-bro!

[horsefez]

Did that, but there are still events that get combined into one event.
What statement for props.conf would you choose if you'd had events like mine.

Could you provide me with an example maybe?
Thank you!

[skoelpin]

Do the events have the same timestamp? Try hardcoding the digits in your regular expression to see if that will work. If the below stanza doesn't work then I would suggest hardcoding the host instead of the source and try that. Don't forget to restart the Splunk service, these changes won't work until the service is restarted

[SOURCE]
TIME_FORMAT = %Y-%m-%d %H:%M:%S,$3N
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE = \<666\>

[horsefez]

Kudos to you my friend!

The suggestion with the time_format did the trick. I didn't realize that the event had two dates in it. One at the start and one further in. Seems like Splunk wasn't able to determine the correct time format before or this messed up the separation of events.

Anyway....

Works!
Big Thanks! 🙂

[s2_splunk]

I would slightly modify that to say BREAK_ONLY_BEFORE = ^\<666\> to anchor the string to the beginning of the line unless you can guarantee the string doesn't occur elsewhere. Just a thought.

[skoelpin]

It looks like there is a space before <666> which would break the regular expression I provided. If you wanted to use the ^ and anchor this in, you then need to do ^\s\<666\> OR ^\s\<\d{3}\>.. Regex is super easy to learn, I'd recommend using a site like www.regex101.com to test your regex to verify it works

[horsefez]

Thank you. I'm relatively new to regular expressions so I always wondered what the ^ is for.