Getting Data In

How to troubleshoot why I received an alert saying "indexer is not reachable"?

charlesguo_2
Engager

I am pretty new to Splunk. Guess what, the consultant has left and I was supposed to take care of Splunk. I got an alert saying "indexer is not reachable". I can ssh to the server, the CPU and disk are OK.
What else can I check?

0 Karma
1 Solution

renjith_nair
Legend

It can be of different reasons but do the basic checks

  • Check if splunk process is running on indexer. Switch to your splunk user on indexer and change the directory to $SPLUK_HOME/bin where SPLUNK_HOME is the directory where your splunk installed and run ./splunk status.

  • Check if the indexer is accessible from search head on port 8089 (or the management port you used)

  • Check if the firewall is open between search head and indexer if they are not on the same subnet.

Happy Splunking!

View solution in original post

renjith_nair
Legend

It can be of different reasons but do the basic checks

  • Check if splunk process is running on indexer. Switch to your splunk user on indexer and change the directory to $SPLUK_HOME/bin where SPLUNK_HOME is the directory where your splunk installed and run ./splunk status.

  • Check if the indexer is accessible from search head on port 8089 (or the management port you used)

  • Check if the firewall is open between search head and indexer if they are not on the same subnet.

Happy Splunking!

charlesguo_2
Engager

It says it is running ok:

./splunk status
splunkd is running (PID: 3001).
splunk helpers are running (PIDs: 3002 3018 3089 3188 3294 17563 17713).

I also test the connection and it is OK. I also didn't see obvious errors from log files.

0 Karma

renjith_nair
Legend

run a search | eventcount summarize=false index=* index=_*|table server|dedup server and see if all of your indexers are listed under server. It might be a momentary n/w issue. Are you still getting the error?

Happy Splunking!
0 Karma

charlesguo_2
Engager

A server reboot solved the issue, still don't understand why.
Thanks for the help!

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...