All Apps and Add-ons

Splunk Add-on for ISC BIND: How to index a single data source and apply multiple sourcetypes in Hunk?

jwalzerpitt
Influencer

We use BlueCat for DNS/DHCP and we are forwarding the DNS/DHCP logs via CEF format to HDFS. I am trying to reverse engineer the Splunk Add-on for ISC BIND for Hunk, specifically the assigning of multiple sourcetypes to one index (assign events to their proper sourcetype - isc:bind:query, isc:bind:lameserver, isc:bind:network, isc:bind:transfer).

How would I go about configuring the props.conf file to assigning multiple sourcetypes to my source of BlueCat?

Thx

0 Karma

kschon_splunk
Splunk Employee
Splunk Employee

You should be able to do this with regular expressions. This page should help:
http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Advancedsourcetypeoverrides

0 Karma

jwalzerpitt
Influencer

I have my regexes ready to go, but do I list each sourcetype as follows in props.conf as:

[source::/xyz/dnslogs/...]
sourcetype = isc:bind:query

[isc:bind:query]
REPORT-1_extract_field = isc_bind_query_extract_field_0
EVAL-message_type = "Query"
EVAL-query_type = "Query"
EVAL-vendor_product = "ISC:Bind"
LOOKUP-2_look_up_extract = isc_bind_severities_lookup vendor_severity OUTPUT severity
LOOKUP-3_look_up_extract = isc_bind_category_lookup sourcetype OUTPUT vendor_category

transforms.conf
[isc_bind_query_extract_field_0]
REGEX = \s+client\s+([\w-.:]{1,100})#(\d{1,5})(?:\s+([^)]+))?:(?:\s+view\s+[^:]+:)?\s+query:\s+(?([\w-.:]{1,100}))?\s+([^\s]+)\s+([^\s]+)\s++-\s+(([\w-.:]{1,100}))$
FORMAT = vendor_severity::$1 src::$2 src_port::$3 query::$4 record_class::$5 record_type::$6 flag::$7 dest::$8

and so on?

[isc:bind:lameserver]
REPORT-1_extract_field = isc_bind_lameserver_extract_field_0
EVAL-app = "ISC:Bind"
EVAL-type = "alert"
LOOKUP-2_look_up_extract = isc_bind_severities_lookup vendor_severity OUTPUT severity
LOOKUP-3_look_up_extract = isc_bind_category_lookup sourcetype OUTPUT vendor_category

[source::/xyz/dnslogs/...]
sourcetype = isc:bind:lameserver

transforms.conf
[isc_bind_lameserver_extract_field_0]
REGEX = (?:\s+lame-servers:)?(?:\s+([^:]+):)?\s+(error\s+(([^)]+))\s+resolving\s+'([\w-.:]{1,100})/([^/]+)/([^']+)':\s+([\w-.:]{1,100})#(\d{1,5}))$
FORMAT = vendor_severity::$1 body::$2 error_type::$3 query::$4 record_type::$5 record_class::$6 dest::$7 dest_port::$8

Thx

0 Karma

kschon_splunk
Splunk Employee
Splunk Employee

(Take the following with a big grain of salt, as I haven't had a chance to test it myself)
I think what you want to do is something like the following. Choose one sourcetype to be the default for the index, i.e.:

[source::/xyz/dnslogs/...]
sourcetype = isc:bind:lameserver

Then for every other sourcetype you wish to assign, do the following:

transforms.conf
[disambiguate_isc_bind_query]
REGEX = \s+client\s+([\w-.:]{1,100})#(\d{1,5})(?:\s+([^)]+))?:(?:\s+view\s+[^:]+:)?\s+query:\s+(?([\w-.:]{1,100}))?\s+([^\s]+)\s+([^\s]+)\s++-\s+(([\w-.:]{1,100}))$
FORMAT = sourcetype:isc:bind:query
DEST_KEY = MetaData:Sourcetype

props.conf
[source::/xyz/dnslogs/...]

TRANSFORMS-isc_bind_query_transform = disambiguate_isc_bind_query

After combining all stanzas with the same header, your stanza in props should looks like:
[source::/xyz/dnslogs/...]

TRANSFORMS-isc_bind_query_transform = disambiguate_isc_bind_query
TRANSFORMS-isc_bind_network = disambiguate_isc_bind_network
......
......

0 Karma

ddrillic
Ultra Champion

In Hunk one assigns the sourcetype by the association to the file location in the HDFS.

Hunk - assigning sourcetype

Gives the example of -

[source::/user/xyz/ciscologfiles/...]
sourcetype = cisco_syslog

[source::/user/xyz/iislogfiles/...]
sourcetype = iis

Do you need to associate multiple sourcetypes for one source?

0 Karma

jwalzerpitt
Influencer

Yes, I would like to associate multiple sourcetypes to one source.

Thx

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...