Getting Data In

I want to index a logfile without breaking up - regardless the content

Starlette
Contributor

Is there a config to index a full logfile regardless the content? I tried MAX_EVENTS=3000 only but it looks that this one needs a BREAK variable as well.

Tags (1)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

Use:

TRUNCATE = 9999999999999
LINE_BREAKER = (?!)

You could use:

MAX_EVENTS = 99999999
BREAK_ONLY_BEFORE = (?!)

But the former is much more efficient for Splunk to perform. (?!) is a PCRE regular expression that will always fail to match.

View solution in original post

jambajuice
Communicator

I'm trying to do the same thing. I want Splunk to index a bunch of individual rule files. My props.conf looks like this:

[nessus_plugins] TRUNCATE = 0 LINE_BREAKER = (?!) SHOULD_LINEMERGE = false

But Splunk still keeps breaking the file into chunks. I can't figure out the logic of it either. Sometimes it will split the file after 16 events or so. Other times it'll do a couple hundred lines...

Any thoughts?

Craig

0 Karma

Nicholas_Key
Splunk Employee
Splunk Employee

There is a much elegant solution to address your issue Starlette.

In your props.conf, have this stanza

[your_currently_defined_stanza]
TRUNCATE = 0
LINE_BREAKER = (?!)

The reason of using this is because we'll never know how long is long enough. So TRUNCATE = 0 tells Splunk not to break the lines at all.

Look for TRUNCATE in the following webpage http://docs.splunk.com/Documentation/Splunk/5.0/Admin/Propsconf

gkanapathy
Splunk Employee
Splunk Employee

Use:

TRUNCATE = 9999999999999
LINE_BREAKER = (?!)

You could use:

MAX_EVENTS = 99999999
BREAK_ONLY_BEFORE = (?!)

But the former is much more efficient for Splunk to perform. (?!) is a PCRE regular expression that will always fail to match.

Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...