All Apps and Add-ons

How can I limit my collection of DNS data using STREAM to a single domain name.

edhealea
Path Finder

I currently have stream collecting DNS from our DNS server. I also have some DNS forwarder that I have been requested to capture any query or responses to a particular DNS name.
I though I could just turn on Stream on the DNS forwarders and write a props/transforms.conf and apply it to those UFs and it would work but it didn't. I get everything from the DNS forwarders.
This is the props and transforms that I used.
props.conf
[stream:dns]
TRANSFORMS-set= setnull, setparsing

transforms.conf
[setnull]
REGEX=^.*
DEST_KEY=queue
FORMAT=nullQueue

Stream config - Remove all data coming in

[setparsing]
REGEX=^.?(?:query|response)\":[\"(?i)(?:dns.name.here).*$
DEST_KEY=queue
FORMAT=indexQueue

Stream config - only collect data for listed url

0 Karma
1 Solution

edhealea
Path Finder

This is what I have found.
You need to call the host from within the prop.conf file not within the tansforms.conf. I couldn't find why this is true on an example @
https://answers.splunk.com/answers/75881/filtering-on-host.html

This is what I end up with and it is working as expected.

Props.conf
[stream:dns]
TRANSFORMS-null= setnull_stream_dns
[host::(servernameAA\d*)
TRANSFORMS-DMZnull= setnull_dmzdnsfwd, dmzdnsfwd_setparsing

Transforms.conf
[setnull_dmzdnsfwd]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

Stream config - Remove all DNZ DNS Forwarders

[dmzdnsfwd_setparsing]
REGEX = ^.?(?:query|response)\":[\"(?i)(?:dns.name.here).$
DEST_KEY = queue
FORMAT = indexQueue

Stream config - only collect data for listed url

[setnull_stream_dns]
REGEX = ^.?(?:query|response)\":[\"(?i)(?:^empty|.?live.com|.?microsoft.com|.?office365.com).*$
DEST_KEY = queue
FORMAT = nullQueue

Stream config

View solution in original post

0 Karma

edhealea
Path Finder

This is what I have found.
You need to call the host from within the prop.conf file not within the tansforms.conf. I couldn't find why this is true on an example @
https://answers.splunk.com/answers/75881/filtering-on-host.html

This is what I end up with and it is working as expected.

Props.conf
[stream:dns]
TRANSFORMS-null= setnull_stream_dns
[host::(servernameAA\d*)
TRANSFORMS-DMZnull= setnull_dmzdnsfwd, dmzdnsfwd_setparsing

Transforms.conf
[setnull_dmzdnsfwd]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

Stream config - Remove all DNZ DNS Forwarders

[dmzdnsfwd_setparsing]
REGEX = ^.?(?:query|response)\":[\"(?i)(?:dns.name.here).$
DEST_KEY = queue
FORMAT = indexQueue

Stream config - only collect data for listed url

[setnull_stream_dns]
REGEX = ^.?(?:query|response)\":[\"(?i)(?:^empty|.?live.com|.?microsoft.com|.?office365.com).*$
DEST_KEY = queue
FORMAT = nullQueue

Stream config

0 Karma

edhealea
Path Finder

Since the UFs don't parse data, I moved my P&T back to my HFs handling this chore. I also changed the P&T by merging it with my current TA_stream P&T. Now that I have all the my DNS Streams coming to my HFs, I have to be able to let the DNS servers send everything through but limit my DNS forwarders to only allow the single Domain queries/responses through so this is what I came up with which still doesn't work. I get everything from both the DNS servers and the DNS forwarders.
props.conf
[stream:dns]
TRANSFORMS-null=setnull_stream_dns, setparsing, setnull_fwd

transforms.conf
[setnull_fwd]
SOURCE_KEY = MetaData:Host
REGEX = ^DNS_FWD*
DEST_KEY = queue
FORMAT = nullQueue
# Stream config - Remove all DNZ DNS Forwarders

[setparsing]
REGEX=^.?(?:query|response)\":[\"(?i)(?:dns.name.here).$
DEST_KEY=queue
FORMAT=indexQueue

Stream config - only collect data for listed url

[setnull_stream_dns]
REGEX=^.?(?:query|response)\":[\"(?i)(?:^empty|.?live.com|.?microsoft.com|.?office365.com).*$
DEST_KEY=queue
FORMAT=nullQueue

Stream config

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...