Getting Data In

Why are events that are sent to splunktcp://9816 from one Universal Forwarder to another UF going into the main index?

srinathd
Contributor

Events sent from one Universal Forwarder to another UF are going directly into the main index, even after I have specified index and sourcetype in the inputs.conf file on the receiving forwarder. How to avoid this?

inputs.conf on receiving forwarder

[splunktcp://9817]
index=test_logs
sourcetype=testlogs
0 Karma

ryandg
Communicator

I have only seen this one time and I am not sure if this will apply to your case but it's worth a shot. Where I work we have multiple departments that utilize Splunk and sometimes when the Universal Forwarder gets rolled out to a device, another company already had rolled it out. What happens then is that our deployment server takes over and sends the custom apps to the UF but it keeps the previous departments inputs and outputs.conf files. What I noticed is when this happens, sometimes the data will go to our main index because of what the other department specified in their inputs/props/transforms due to the conf files conflicting with each other.

It may be beneficial to at least check the /etc/apps folder to see if there are any additional apps directories that you do not recognize, or if you do recognize things like TA-Windows might have additional inputs.conf that you didn't originally write.

0 Karma

srinathd
Contributor

Index is correctly specified. And in the main index sourcetype is coming differently, as splunk is identifying the metadata and assigning it to the events.

0 Karma

lguinn2
Legend

What is the outputs.conf on the receiving forwarder?

0 Karma

srinathd
Contributor

This is the outputs.conf on receiving forwarder
[tcpout]
defaultGroup = splunkssl

[tcpout-server://idxr-01:9997]

[tcpout:splunkssl]
disabled = false
server = idxr-01:9997
useACK = true

0 Karma

lguinn2
Legend

Ok, now I am confused.

If receiving forwarder has the following:
inputs.conf

[splunktcp://9817]
index=test_logs
sourcetype=testlogs

outputs.conf

[tcpout]
defaultGroup = splunkssl
[tcpout-server://idxr-01:9997]
[tcpout:splunkssl]
disabled = false
server = idxr-01:9997
useACK = true

Where is the second forwarder? It looks like the receiving forwarder sends directly to the indexer...

0 Karma

srinathd
Contributor

sending forwarder has the following configuration
inputs.conf file
[default]
host = hostname
index= indexname

[monitor:///data/perfdata/*.nmon]
disabled = false

outputs.conf file

[tcpout:fwdr]
sendCookedData = false
server = lm-fwdr:9817

0 Karma

lguinn2
Legend

Does the index exist? Is it spelled exactly the same? Does the data in the main index have the correct sourcetype?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...