All Apps and Add-ons

Problem line break- Splunk inventing its own linebreak.

clintla
Contributor

Cant figure out why Splunk is making up its own line breaks.

LINE_BREAKER = ([\r\n]+)\s\s[A-Za-z]\S+:\s$

** the line break will always start the line with 2 spaces then a letter. Ends with
a : then space & EOL.

Group:

Authorize-log:

Copy stats: 
  Authorize-log_Dr: 
    Journal:
      Usage: 136.36GB
      Total: 138.93GB

Exchpf01:

Copy stats: 
  Exchpf01_DR: 
    Journal:
      Usage: 136.48GB
      Total: 138.93GB

======================================= (this page doesnt show the breaks/format right)

Splunk doesnt like my props though. I've tried a few things. When I try the
regular expression in Notepad++ it works great. It seems to break on numbers,
spaces.. should have 50 or so breaks but its just well into the thousands.

Tags (2)
0 Karma

clintla
Contributor

Maybe a bug.. if I remove the linebreak in props.conf & just use a rex command during the search- it finds all the break points correctly using the same Regular Expression.

0 Karma

clintla
Contributor

Think it has something to do w/ it not starting with a fresh line.

LINE_BREAKER = ([\r\n]+)\s\s[A-Za-z]\S+:\s$

its breaking many times on 2 spaces when it starts w/ 8 spaces.

Should this work? Why does splunk not use the carot well?
LINE_BREAKER = ^\s\s[A-Za-z]\S+:\s$

0 Karma

clintla
Contributor

Only have 1 props file. I've made intentional changes to it just like take
the whole line out & just have no breaks which works to so I know splunk
is looking at my new changes.

0 Karma

clintla
Contributor

An escape like this?

LINE_BREAKER = ([\r\n]+)\s\s[A-Za-z]\S+:\s$

The above change- doesnt change anything. It seems though that if your
rex command is incorrect that you would not have any line breaks which
it the part that bugs me. breaking in the middle of a number seems like
splunk has changed its mind & decided to make up its own breaks.

The "copy stats" is just a subset of the data set. its like 3 spaces in.
its part of the log & I dont want to alter the logs. All this is doing
is monitoring a .txt file that updated twice an hour. No forwarder or anything
else is being used.

0 Karma

kristian_kolb
Ultra Champion

So the first line of any event will have 2 spaces and a letter. Then some more characters followed by a colon, space, newline?

Are you sure you are editing the right props.conf (i.e. where the parsing takes place)? If you are using a full/heavy forwarder the props settings for linebreaking should go there, otherwise (lightweight/universal/no forwarder) the props.conf on the indexer is where the settings should go.

Hm have you tried to escape the colon...might help

Also, if the "copy stats:" stuff is not part of the events, you should probably take them away.

/k

0 Karma

clintla
Contributor

Copy stats will never be part of it & there are never going to be time stamps.

in my example
Authorize-log:
&
Exchpf01:

(although this page doesnt show the spaces/EOL's correctly. )

below is the description:

the line break will always start the line with 2 spaces then a letter. Ends with a ":" then space & EOL.

0 Karma

kristian_kolb
Ultra Champion

Is the "Copy Stats:" part of your events?
I am not really familiar with that type of log. If you could post more events, that might be helpful.

By the way, are there no timestamps?

If the "Copy stats:" is always in your log, you could use that as a line breaker more explicitly:

LINE_BREAKER = ([\r\n]+)\s*Copy stats

/k

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...