Splunk Search

How to write a search and alert if any indexers are down?

splunker9999
Path Finder

Hi,

We have 4 indexers and we need to write a search and set up an alert if any of the indexers is down.

Can some one please advise on this type of search?

Thanks,

0 Karma

ChrisG
Splunk Employee
Splunk Employee

You probably don't need to write such a search yourself. You should start with the overview dashboard in the Distributed Management Console. It will show you your deployment topology and whether any indexers are down. If you have not configured the Distributed Management Console, see the Distributed Management Console documentation.

If you are using indexer clustering, the cluster master dashboard will also show you what indexers are up and down.

splunker9999
Path Finder

This would be useful to monitor, but we are looking for a alert to be recieved whenever indexer is down?

0 Karma

ChrisG
Splunk Employee
Splunk Employee

But you could set up an alert from the dashboard search, couldn't you?

0 Karma

splunker9999
Path Finder

We are new to the Splunk and need some assistance, Can you please help us?

0 Karma

bmacias84
Champion

The DMC has preconfigured alerts for what you want. Enable the "Search Peer Not Responding" alert.

DMC Alert - Abnormal State of Indexer Processor [edit]
One or more of your indexers is reporting an abnormal state.

DMC Alert - Critical System Physical Memory Usage [edit]
One or more instances has exceeded 90% memory usage.

DMC Alert - Expired and Soon To Expire Licenses [edit]
You have licenses that expire or will expire within two weeks.

DMC Alert - Missing forwarders [edit]
One or more forwarders are missing.

DMC Alert - Near Critical Disk Usage [edit]
You have used 80% of your disk capacity.

DMC Alert - Saturated Event-Processing Queues [edit]
One or more of your indexer queues is reporting a fill percentage, averaged over the last 15 minutes, of 90% or more.

DMC Alert - Search Peer Not Responding [edit]
One or more of your search peers is currently down.

DMC Alert - Total License Usage Near Daily Quota [edit]

Get Updates on the Splunk Community!

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...