Splunk Search

Changing max length of field

sc0tt
Builder

I have a field that is more than 10,000 characters. I updated props.conf to include

[source::log.txt]
TRUNCATE=20000

Splunk now indexes the entire event, but the content of the long field is being ignored when doing a search. For example search | eval l = len(long_field) returns a length of 1. Where can I change the max length of a field?

Thanks

Tags (2)
0 Karma
1 Solution

Ayn
Legend

You might be hitting this limit (from limits.conf):

maxchars = <integer>
* Truncate _raw to this size and then do auto KV.
* Defaults to 10240 characters.

View solution in original post

sc0tt
Builder

Shorter fields work as expected. For example, if I count the field length for all events the max length is 9996; all the fields with a known length greater than 10,000 show as a length of 1. So it is clearly being limited to 10,000 somewhere.

0 Karma

Ayn
Legend

You might be hitting this limit (from limits.conf):

maxchars = <integer>
* Truncate _raw to this size and then do auto KV.
* Defaults to 10240 characters.

jiaminyun
Loves-to-Learn Lots

How many bytes does a character take

0 Karma

sc0tt
Builder

Thanks! That did it. I created a limits.conf file with maxchars = 20000 and it seems to be working as expected. Any known issues with increasing this value even higher? I'm seeing that some events have length > 19000.

0 Karma

pbankar
Path Finder

Hey @Ayn, is there any limit for the same?

0 Karma

Ayn
Legend

Do shorter fields with the same format work like it should? Or might this be an issue with the extraction itself?

0 Karma

sc0tt
Builder

It's a space delimited field (field=" value1 value2 value3 value4 value5 value6..etc), so just using default Splunk extraction; nothing special is being applied to the file.

0 Karma

Ayn
Legend

How is long_field extracted?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...