Getting Data In

Clear indexing data on Splunk Indexer impact to Splunk forwarder lost connection to Splunk Indexer.

thomson12
New Member

Hi Splunk Support,

I have encounter the issue on Splunk Forwarder have lost connection to Splunk Indexer after clean indexing data on Splunk Indexer.

Scenario: I have cleaned indexing data on Splunk Indexer and check on Splunk Forwarder by command [~/bin/splunk list forward-server -auth username:password]

Result: Splunk Forwarder lost connection as below.
root@env # ~/bin/splunk list forward-server -auth username:password

Active Splunk-2-Splunk Forwards:

    Splunk_Indexer01:9997

Configured but inactive Splunk-2-Splunk Forwards:

    Splunk_Indexer02:9997

Please help. Thank you.

Tags (1)
0 Karma
1 Solution

bmacias84
Champion

The splunk clean eventdata -index ##someidex### -f command is a stop the world event requiring you to stop and start splunkd.

Are you using the auto-load-balance feature on your fowarders? When the splunkd service starts responding the fowarder should pickit up on the next interval.

[tcpout]

[tcpout-server://server1:9997]
[tcpout-server://server2:9997]
[tcpout-server://server3:9997]

[tcpout:default-autolb-group]
autoLB = true
disabled = false
autoLBFrequency = 45
maxFailuresPerInterval = 3
secsInFailureInterval = 2
connectionTimeout = 10

server = server1:9997,server2:9997, server3:9997

View solution in original post

0 Karma

thomson12
New Member

Thanks for advise, I found this message from Splunk forum "Running splunk list forward-server lists one of the servers under "Configured but inactive forwards:", but it is forwarding. (SPL-35461)" I was POC with my environment. Command "~/bin/splunk list forward-server -auth username:password" will display some Indexer inactive "Configured but inactive Splunk-2-Splunk Forwards" but forwarder still forwarding data to that server when I verify time from Search apps.

0 Karma

bmacias84
Champion

The splunk clean eventdata -index ##someidex### -f command is a stop the world event requiring you to stop and start splunkd.

Are you using the auto-load-balance feature on your fowarders? When the splunkd service starts responding the fowarder should pickit up on the next interval.

[tcpout]

[tcpout-server://server1:9997]
[tcpout-server://server2:9997]
[tcpout-server://server3:9997]

[tcpout:default-autolb-group]
autoLB = true
disabled = false
autoLBFrequency = 45
maxFailuresPerInterval = 3
secsInFailureInterval = 2
connectionTimeout = 10

server = server1:9997,server2:9997, server3:9997

0 Karma

jbsplunk
Splunk Employee
Splunk Employee

What does the TcpOutputProc component of splunkd in your splunkd.log show is happening with outgoing connections to the indexers? You should see some type of information on connections to both indexers.

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...