Splunk Search

Splunk 6.3.0: Source type not getting applied to forwarded structured data

pbadhe
Explorer

I am using splunkforwarder-6.3.0-aa7d4b1ccb80-linux-2.6-x86_64.rpm to forward tab delimited structured data from one VM to Indexer server with splunk splunk-6.3.0-aa7d4b1ccb80-linux-2.6-x86_64.rpm installed on it.

Below is the command used to add exec:

./splunk add forward-server 1.1.1.1:9997
./splunk add exec -source "scripts/myScript.sh Device" -interval 1800 -index DeviceIndex -sourcetype DeviceSourceType

Index DeviceIndex & sourcetype DeviceSourceType are created on Indexer server 1.1.1.1

When I receive the data on 1.1.1.1, the default column Sourcetype has value DeviceSourceType for all the events.

But, I don't see the custom extracted fields defined in sourcetype DeviceSourceType in the search result.
That means the sourcetype is not getting applied to the search results.

Can anybody please let me know what is wrong? What configurations are missing?

The same configurations were working in Splunk 6.2.5.

0 Karma
1 Solution

pbadhe
Explorer

Thanks for replying. I found the issue.
My sh script outputs a structured (tab delimited) data and I defined the sourcetype accordingly.

The issue was I have not added the sourcetype definition in the inputs.conf of Universal forwarder.

After adding sourcetype definition on UF and restarting it, I can see the fields defined in sourcetype.

Still wondering though, why the sourcetype definition is required on UF since as per splunk documentation, Splunk UF do not parse the data while forwarding it.

  • sourcetype is set in the input portion of the pipleline, not the parsing portion.

View solution in original post

0 Karma

pbadhe
Explorer

Thanks for replying. I found the issue.
My sh script outputs a structured (tab delimited) data and I defined the sourcetype accordingly.

The issue was I have not added the sourcetype definition in the inputs.conf of Universal forwarder.

After adding sourcetype definition on UF and restarting it, I can see the fields defined in sourcetype.

Still wondering though, why the sourcetype definition is required on UF since as per splunk documentation, Splunk UF do not parse the data while forwarding it.

  • sourcetype is set in the input portion of the pipleline, not the parsing portion.
0 Karma

pbadhe
Explorer

Hi Rich,
Thanks for replying. I found the issue.
My sh script outputs a structured (tab delimited) data and I defined the sourcetype accordingly.

The issue was I have not added the sourcetype definition in the inputs.conf of Universal forwarder.

After adding sourcetype definition on UF and restarting it.

I can see the fields defined in sourcetype.
Still wondering though, why the sourcetype definition is required on UF since as per splunk documentation, Splunk UF do not parse the data while forwarding it.

Thanks,
Prashant

0 Karma

Richfez
SplunkTrust
SplunkTrust

The sourcetype is set in the input portion of the pipeline, not the parsing portion. These wonderful diagrams may help.

Regardless, sounds like you solved your own problem! Could you convert your comment above to an answer and mark it answered? It's completely OK to answer your own questions!

Thanks!

0 Karma

Richfez
SplunkTrust
SplunkTrust

Have you tested the various regexes (I presume) in a search to make sure they extract something?

Otherwise, could you paste in a few events, and the relevant portions of inputs.conf, props.conf and transforms.conf? Thanks!

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...