Getting Data In

How to get sourcetypes to TA/apps mapping via Splunk API?

koshyk
Super Champion

As part of performance analysis, we are asked to do sourcetypes and regex analysis. The first step I wanted to see is how I can map a sourcetype to an add-on/app/TA?
We have got around 2000 sourcetypes as part of a literal search coming from various TA's, so manually it is impossible.

eg of what I'm looking for is something like below

Sourcetype,App
cisco:ise:syslog,Splunk_TA_cisco-ise
vmware:esxlog:vmkwarning,Splunk_TA_esxilogs

I know I can write scripts/grep on btool and do. but trying to think a way to do directly within Splunk

0 Karma
1 Solution

koshyk
Super Champion

Ah. Found a way. Combining conf-props, and inputs. Thanks jkat54 for the idea

| rest /services/configs/conf-props| stats count by eai:acl.app,title| search title!="source*"| rename title as sourcetype|eval endPoint="props"| append [| rest /services/configs/inputs| stats count by eai:acl.app,sourcetype| where sourcetype!=""| eval endPoint="inputs"]| stats count by eai:acl.app,sourcetype,endPoint

View solution in original post

koshyk
Super Champion

Ah. Found a way. Combining conf-props, and inputs. Thanks jkat54 for the idea

| rest /services/configs/conf-props| stats count by eai:acl.app,title| search title!="source*"| rename title as sourcetype|eval endPoint="props"| append [| rest /services/configs/inputs| stats count by eai:acl.app,sourcetype| where sourcetype!=""| eval endPoint="inputs"]| stats count by eai:acl.app,sourcetype,endPoint

jkat54
SplunkTrust
SplunkTrust

This will only work if you have the inputs on the splunk enterprise instance itself:

| rest /services/configs/inputs | table eai:acl.app sourcetype | where sourcetype!=""

If you're using forwarders to collect your data, you will need some sort of for loop and bat/powershell/bash scripts:

bash example to be run on deployment server:

for input in `find /opt/splunk/etc/deployment-apps -type f -name inputs.conf`; do echo $input; grep -i sourcetype $input; done

powershell example to be run on deployment server:

get-childitem 'C:\Program Files\Splunk\etc\deployment-apps' -Recurse -Include "inputs.conf" -WarningAction SilentlyContinue -ErrorAction SilentlyContinue|
foreach { 
    write-host $_.FullName
    select-string $_ -pattern "sourcetype"
}

# remove Warning & Error Actions above if you want to see permission issues, etc... shouldnt happen if your running powershell as administrator though.

You might also want to remove deployment-apps from the paths... which would then show everything under $splunk_home/etc and would work on any splunk server... if you want to run these on forwarders, again you will have to change the paths to match. YMMV!

koshyk
Super Champion

Thanks jkat54
but only 60% of sourcetypes are present in inputs.conf ;For example if you go into Splunk_TA_vmware, the sourcetypes are assigned within props.conf

Is there similar search for "inputs", "props" , "transforms" etc?

0 Karma

jkat54
SplunkTrust
SplunkTrust

You can change sourcetypes with props and transforms sure, but then they are dynamic based on regex, etc... So I don't know how to help you find those.

Probably some combination of the data source and the sourcetype and then the app will work.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...