Getting Data In

What the best strategy to discard all temporary data while testing on some forwarders?

dsmc_adv
Path Finder

We have a clustered environment that includes heavy forwarders, universal forwarders, and forwarders under Windows. The development team sometimes do performance tests and these generate a lot of data that we don't want to be indexed. We could add a new rule on the heavy forwarders to send to null queue all events during the tests , but can this be done at forwarder or universal forwarder level? Do you think that there is a better way to achieve this ?

Thank you

0 Karma

ddrillic
Ultra Champion

You can have the data indexed into specific indexes or add a specific field which indicates that this is a performance test data. Then it's easy to "simply" delete this type of data.

0 Karma

somesoni2
Revered Legend

Have a look at this Splunk documentation to know more about event routing and filter.
http://docs.splunk.com/Documentation/Splunk/6.4.0/Forwarding/Routeandfilterdatad#Filter_event_data_a...

The send to null queue can be done on universal forwarder if it's to be done without looking into individual events (purely based on index/source/sourcetype/host). If you need to look at the event data to filter, than you need to do routing/filtering in heavy forwarder/indexer

0 Karma

dsmc_adv
Path Finder

It looks like this only can be done at hf or indexer level as I suspected, but not in universal forwarder:

"Although similar to forwarder-based routing, queue routing can be performed by an indexer, as well as a heavy forwarder"

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...