Getting Data In

How do I limit what kind of events go into Splunk to avoid daily license limit?

rung8
New Member

Hi everyone,

As the title suggests I was wondering if I can filter the logs that go into Splunk to avoid the daily volume limit. Our catalina logs show a lot of junk, and we only want the good stuff. Letting it index all of it would easily go over the limit.

I looked in the documentations, and it says that I can configure routing and filtering ONLY on a heavy forwarder, not a universal one.

If this is the case, then I should point all my uniForwarders to the heavy forwarder to do the filtering right?

Does sending traffic to nullQueue prevent the daily volume from going up? or does it still take it?

0 Karma
1 Solution

prakash007
Builder

@rung8 : you don't have to point all your forwarders to a heavy forwarder, you can filter the unwanted data via nullQueue on the indexers as well, and they are not counted against your license volume(nullQueue process happens during the parsing phase of the pipeline)
Hope this splunk docs will give you an idea...
http://docs.splunk.com/Documentation/Splunk/7.2.1/Forwarding/Routeandfilterdatad#Filter_event_data_a...
http://docs.splunk.com/Documentation/Splunk/7.2.1/Admin/Configurationparametersandthedatapipeline#Ho...

View solution in original post

kmorris_splunk
Splunk Employee
Splunk Employee

@prakash007 is correct in his comment. I would caution you when filtering out data however. Sometimes we don't know what we don't know, so be careful to be very specific on your regex when filtering. I have seen several customers who have inadvertently filtered out things that they didn't intend to.

0 Karma

sloshburch
Splunk Employee
Splunk Employee

And other's that use the searches from that data point draw conclusions on incomplete data sets...

0 Karma

prakash007
Builder

@rung8 : you don't have to point all your forwarders to a heavy forwarder, you can filter the unwanted data via nullQueue on the indexers as well, and they are not counted against your license volume(nullQueue process happens during the parsing phase of the pipeline)
Hope this splunk docs will give you an idea...
http://docs.splunk.com/Documentation/Splunk/7.2.1/Forwarding/Routeandfilterdatad#Filter_event_data_a...
http://docs.splunk.com/Documentation/Splunk/7.2.1/Admin/Configurationparametersandthedatapipeline#Ho...

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...