Getting Data In

/var/log/messages associated with index host NOT the correct source host

krussell101
Path Finder

I have two dozen UF linux systems. All of them are picking up /var/log/messages and sending it to my indexer (the one and only "splunk" host).

All of the /var/log/messages entries are indexed as coming from host=splunk.

WHY?

HELP!

Thanks.

Tags (1)
0 Karma
1 Solution

BryanBerry
Path Finder

Have any stray props.conf/transforms.conf on the indexer? Is this all of your forwarders or only some of them?

Try "splunk btool transforms list --debug > out.txt" on your indexer and grep for MetaData:Host in out.txt. It possible that there's a transform setting the host value to splunk

View solution in original post

0 Karma

BryanBerry
Path Finder

Have any stray props.conf/transforms.conf on the indexer? Is this all of your forwarders or only some of them?

Try "splunk btool transforms list --debug > out.txt" on your indexer and grep for MetaData:Host in out.txt. It possible that there's a transform setting the host value to splunk

0 Karma

BryanBerry
Path Finder

Huh, that is very odd. Was that in etc/system/local/transforms.conf or default? Did you find how that got in there? I'm curious about the cause as well.

0 Karma

krussell101
Path Finder

FOUND IT! THANK YOU!

0 Karma

krussell101
Path Finder

Any hints on where this may have snuck in? I don't recall doing any transforms at all. Will this be on my forwarders somewhere or on the indexer?

Thanks again.

0 Karma

krussell101
Path Finder

This is occurring to /var/log/messages from all of my forwarders.

transforms list output:

system [syslog-host]
system CAN_OPTIMIZE = True
system CLEAN_KEYS = True
system DEFAULT_VALUE =
system DEST_KEY = MetaData:Host
system FORMAT = host::splunk-mydomain.com

Is this taking everything of sourcetype syslog and attributing it to another host?

Cool! So now I have to unset that somehow. Thanks!

0 Karma

krussell101
Path Finder

This isn't an answer

0 Karma

krussell101
Path Finder

No. they pick up the local hostname. Which is how I want it to behave. All other files in /var/log come over with the correct hostname. This is the only file which is attributed to the incorrect host.

0 Karma

Lucas_K
Motivator

Is there a hostname defined on your UF's in /opt/splunkforwarder/etc/system/local/inputs.conf?

krussell101
Path Finder

sourcetype = syslog

0 Karma

Ayn
Legend

What sourcetype do you have for /var/log/messages?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...