- Splunk Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- Real source IP of event
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
We uncovered a problem with two forwarders using the same host-value. At first, we were baffled by the fact that Splunk was showing events that weren't in the log file itself. After looking in the _internal index, I noticed an unknown server sending events to the indexer.
The cause of the problem is that the unknown server has been created by cloning the original server, so both servers have the same host-value in their respective inputs.conf-files.
So now we have a problem where the index is contaminated with correct data from the incorrect host, using the same host-value. I would like to delete the incorrect data, but how can I find the data? The field sourceHost apparently doesn't exist outside of the _internal index.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Just want to return to this to report the conclusions.
I contacted Splunk support about this and apparently, the event is indexed using the hostname reported by the Splunk forwarder. Once it's indexed, there is no way of knowing the originating host, unless you can identify the information in the event itself, eg. source path, sourcetype, etc.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Just want to return to this to report the conclusions.
I contacted Splunk support about this and apparently, the event is indexed using the hostname reported by the Splunk forwarder. Once it's indexed, there is no way of knowing the originating host, unless you can identify the information in the event itself, eg. source path, sourcetype, etc.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So it sounds like you have duplicate events. I would run a search and then use the dedup command with a common field then just pipe to delete and that should take care of those events.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Since host2 is identifying itself as host1, it is hard to distinguish the false and irrelevant data from the real data that actually comes from host1.
So I'm wondering if there is a way of disregarding the host-field in the event and searching for the real IP address that the event originates from? Then I could delete all the incorrect events from host2. Now it seems I have to re-index.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Oops... Seems I described my problem very poorly.
No, there are no duplicate events. There are events from two hosts, host1.example.com and host2.example.com. Host2 has been created as a clone of host1. Host1 is the production host, whereas host2 has been built to be used later.
Since host2 is a clone of host1, it has this line in inputs.conf:
host = host1.example.com
Host2 is running the same software as host1, but nobody is using host2 and host2 does not have DB access. Therefore any events generated on host2 are irrelevant, but appear alarming.
Introducing Splunk Enterprise 9.2
Adoption of RUM and APM at Splunk
Routing logs with Splunk OTel Collector for Kubernetes
