Getting Data In

Size limit on for an event? Part 2

alextsui
Path Finder

Hi, I searched the Splunk>answers and saw someone had asked the question before. But my situation is a little different.

Some events receive from F5 ASM thru tcp port are been split into two events. The first part of the split events all have 2094 characters in length. And a monitored input, where events are longer than 2094 characters, aren't been split. It looks like Splunk is splitting the events only if they are longer than 2094 characters and are coming in from F5 ASM (or network port). I have tried increasing the TRUNCATE and MAX_EVENT in props.conf with no luck.

Any suggestion?

Splunk version 4.1.3 on CentOS 5.4 64-bit.

Thanks, alex.

Tags (2)

gkanapathy
Splunk Employee
Splunk Employee

It's likely that your F5 is splitting events at 2048 characters because the syslog protocol limits it to that. You would probably want to use SHOULD_LINEMERGE and a BREAK_ONLY_BEFORE or BREAK_ONLY_BEFORE_DATE to assemble your "lines" back into full events. You could also try setting LINE_BREAKER so get Splunk to ignore whatever character the device is using in the stream to split lines, and provide and alternate sequence.

alextsui
Path Finder

Tried first using the SHOULD_LINEMERGE and BREAK_ONLY_BEFORE, couldn't get the split events to merge. Then change the max event size to 60k on F5 web console. Most of large size events showed as a single event, but noticed events started getting split at random length.
After using SHOULD_LINEMERGE=false, and LINE_BREAKER=(<134>) (all events from F5 ASM starts with <134> in my case) still seeing one or two events every hour with <134> at the beginning of events in Splunk search results. Shouldn't all events indexed by Splunk now be without the <134> at the beginning?

0 Karma

Lowell
Super Champion

Are your events coming over the syslog protocol?

Because syslog imposes a 2k (2048 byte) limit on the size of its log events. If this is the case then I'm guessing that splunk is configured to add a timestamp or something that amounts to 46 characters adding up to your 2094 size observation.

I know is some syslog implementations can bump this up to 4k (4096), but I think that's the max.

I could be way off here, but seems like it's a possibility

alextsui
Path Finder

Thanks for the info on the syslog's limit on event size. After checking the related settings on F5 ASM web cosole, it does provide an option for selecting the size for events. It defaults to 2k, and has 10k and 60k for selection.

0 Karma

erydberg
Splunk Employee
Splunk Employee

Can you post your settings from props.conf?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...