Getting Data In

I want to index a logfile without breaking up - regardless the content

Starlette
Contributor

Is there a config to index a full logfile regardless the content? I tried MAX_EVENTS=3000 only but it looks that this one needs a BREAK variable as well.

Tags (1)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

Use:

TRUNCATE = 9999999999999
LINE_BREAKER = (?!)

You could use:

MAX_EVENTS = 99999999
BREAK_ONLY_BEFORE = (?!)

But the former is much more efficient for Splunk to perform. (?!) is a PCRE regular expression that will always fail to match.

View solution in original post

jambajuice
Communicator

I'm trying to do the same thing. I want Splunk to index a bunch of individual rule files. My props.conf looks like this:

[nessus_plugins] TRUNCATE = 0 LINE_BREAKER = (?!) SHOULD_LINEMERGE = false

But Splunk still keeps breaking the file into chunks. I can't figure out the logic of it either. Sometimes it will split the file after 16 events or so. Other times it'll do a couple hundred lines...

Any thoughts?

Craig

0 Karma

Nicholas_Key
Splunk Employee
Splunk Employee

There is a much elegant solution to address your issue Starlette.

In your props.conf, have this stanza

[your_currently_defined_stanza]
TRUNCATE = 0
LINE_BREAKER = (?!)

The reason of using this is because we'll never know how long is long enough. So TRUNCATE = 0 tells Splunk not to break the lines at all.

Look for TRUNCATE in the following webpage http://docs.splunk.com/Documentation/Splunk/5.0/Admin/Propsconf

gkanapathy
Splunk Employee
Splunk Employee

Use:

TRUNCATE = 9999999999999
LINE_BREAKER = (?!)

You could use:

MAX_EVENTS = 99999999
BREAK_ONLY_BEFORE = (?!)

But the former is much more efficient for Splunk to perform. (?!) is a PCRE regular expression that will always fail to match.

Get Updates on the Splunk Community!

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...