Getting Data In

Half event to index queue and half event to null queue

bharathkumarnec
Contributor

Hi All,

Is there a possibility to send half of the event to index queue and half of the event to null queue??

Can this be done based on the size of the event, for example only 1Kb event should be indexed out of 1MB event and rest should be sent to null queue.

Thanks in advance for your inputs!

Regards,
BK

0 Karma

woodcock
Esteemed Legend

You have 2 options:
1: Have Splunk split it into 2 events and then throw away one of them. This is silly.
2: Use SEDCMD to strip the bad part of the event and replace it with nothing. This rocks.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi bharathkumarnec,
you can use TRUNCATE in props.conf to limit the lenght of each event of a specified sourcetype.
Bye.
Giuseppe

0 Karma

bharathkumarnec
Contributor

Hi,

I have tried using Truncate in props.conf, but it did not help me out.

The logs are from kafka topic events.

Regards,
BK

0 Karma

gcusello
SplunkTrust
SplunkTrust

with TRUNCATE you define the max lenght of your events.
If you can define a point of your events (e.g. a word or some chars) that you can use in a regex you can delete all chars after this word or chars.
Bye.
Giuseppe

0 Karma

bharathkumarnec
Contributor

My data is in JSON format, will regex take care of this for JSON events??

0 Karma

gcusello
SplunkTrust
SplunkTrust

with regex you can parse every event, can you share an example of your logs?
Bye.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...