All Apps and Add-ons

Splunk Add-on for NetFlow: How to filter Netflow data and send specific fields to Indexer?

hemendralodhi
Contributor

Hello Team,

I have the Splunk Add-on for NetFlow installed on Splunk Heavy Forwarder (HF), receiving data from Netflow enabled device. I am getting the data and it is sent successfully to indexer. I want to filter fields send only specific fields for indexing. Can you let me know how to achieve this?

E.g I need to completely remove below fields from indexing. These fields are as is extracted by add-on

tcp_flag,fwd_status,src_tos

Will the below configuration work? And do I have to perform the change at HF or Indexer level?

props.conf

[netflow]
TRANSFORMS-nullQ= nullFilter

transforms.conf

[nullFilter]
REGEX = tcp_flag|fwd_status|src_tos
DEST_KEY=queue
FORMAT = nullQueue

Thanks
Hemendra

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

@hemendralodhi Now showing in a theater near you: Splunk Stream 7: The Filtered Dataset. Download now at Splunkbase [https://splunkbase.splunk.com/app/1809]. Splunk Stream captures Netflow (v5, v9, and IPFIX) and allows you to filter based on the fields being sent in the data AS WELL AS allow you to pick which fields to index. Without using a props or transforms! Read the full documentation here: http://docs.splunk.com/Documentation/StreamApp/7.0.1/DeployStreamApp/ConfigureFlowcollector

This is to be used in the place of the Netflow Add-On, as it is more robust and scalable.

woodcock
Esteemed Legend
0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

edited thanks!

0 Karma

hemendralodhi
Contributor

Thanks for the recommendation , but I may not be able to use Stream for now but look into it later.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Stream is really your only option here. You cannot simply and easily remove specific fields from the Netflow traffic using props and transforms. You can drop ENTIRE events, yes, but you cannot pull out specific fields. You need stream to do that, otherwise you will be writing some very Nasty SEDCMD settings.

0 Karma

woodcock
Esteemed Legend

Do you need to:
1) Stop specific fields from being created but allow the events that have that data to be forwarded in?
2) Stop the events themselves from coming into Splunk?

0 Karma

hemendralodhi
Contributor

Hello Woodcock,

Thanks for your response.

Fields were created by Netflow add on (installed on HF) based on the data and it has pre-defined extraction. Data comes in binary format which Add-on converts into ascii format and send to indexers.

I want to stop the events for some specific fields to send to Indexer. For e.g. I don't want tcp_flag,fwd_status,src_tos field data to be sent to indexer. There are other fields too which i want to filter out. Based on this sample , I can work them out.

Appreciate your help.

Thanks
Hemendra

0 Karma

woodcock
Esteemed Legend

See the answer by @alacercogitatus. He is spot on.

0 Karma

woodcock
Esteemed Legend

I should have added this:
3) Strip the raw events of certain bits of data but let the rest of the event string come in.

I think you mean #3 but I need you to confirm. I do not at all understand the way that you have phrased the question.

0 Karma

hemendralodhi
Contributor

Hello Woodcock,

Yes that is correct, I want to strip certain raw events (fields) from the event and then send to indexer.

Please advise.

Unfortunately , I will not be able to use Splunk Stream for now.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...