Splunk Search

New sourcetype linebreaking issues

gelica
Communicator

Hi,

I'm a new Splunk user and I'm trying to define a new source type.

My log file looks like:

-----------
Name1
-----------
Info1.1: one
info1.2: two
-----------
Name2
-----------
Info2.1: one
Info2.1: two
two
Info2.3: Three
etc..

I want to have a sourcetype that breaks my file before every name, but the real names aren't as nicely formatted as the ones in my example, they often consist of multiple words and sometimes some special character as well.

I manage to break before each --------- but then the names and the info gets separated. I also tried setting the LINE_BREAKER but then the pattern matching my regex is removed.

I tried some different regexes, similar to (-{2,50}[^:]+-{2,50}[^:]+:) and (?<=--)([^-:]+)(?=--) but they don't work.
It seems like the newlines aren't recognized correctly..?

Does anyone have any idea of what I'm doing wrong?

Thanks!

0 Karma
1 Solution

lguinn2
Legend

Try this in your props.conf

[yoursourcetype]
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = (?m)-{10}.*-{10}

I know that my regex is probably not what you need - but substitute whatever you used for the LINE_BREAKER. Using BREAK_ONLY_BEFORE will have a similar effect, but it doesn't remove the matching pattern.

View solution in original post

0 Karma

lguinn2
Legend

Try this in your props.conf

[yoursourcetype]
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = (?m)-{10}.*-{10}

I know that my regex is probably not what you need - but substitute whatever you used for the LINE_BREAKER. Using BREAK_ONLY_BEFORE will have a similar effect, but it doesn't remove the matching pattern.

0 Karma

lguinn2
Legend

Try replacing the word "LINE_BREAKER" with "BREAK_ONLY_BEFORE" and see what happens.

0 Karma

gelica
Communicator

I replaced the regex with ([\r\n]+)(-{2,50})([\r\n]+)([^:]+)([\r\n]+)(-{2,50}[^:]+:) and also tried removing the ([\r\n]+)'s from my regex.
I played around with LINE_BREAKER some more and got this to work in the data preview:
LINE_BREAKER = ([\r\n]+)(-{2,50})([\r\n]+)([^:]+)([\r\n]+)(-{2,50}[^:]+:)
MAX_EVENTS = 10000
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
pulldown_type = 1
For some reason the text isn't removed by this linebreaker pattern. Everything looks OK in the preview but when I add the data for real, it gets messed up..

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...