Getting Data In

Are there alternative ways to monitor forwarders?

satkan100
Path Finder

My splunk environment we have not enable forward management so for me difficult to manage the forwarder host up & down status .

If possible to monitor any other methods? Example App or query if anyone knows please share.

0 Karma
1 Solution

adonio
Ultra Champion

hello there,

i guess there are couple different ways to achieve.
the way i approach this is by checking if splunk internal data is flowing. if it does -> all good, if it doesnt -> probably connection error or forwarder is down -> alert and check
here is a quick and dirty way to achieve it
| tstats count as event_count by host where index = _interanl
from there you can take it however you like it, the nice part about it is that |tstats takes into consideration the timepicker.
so you can schedule a report / alert
also, you can create a lookup with list of all forwarders and update it every week / day / hour etc, and then run a search that compare existing forwarders to that list

hope it helps

View solution in original post

0 Karma

aakwah
Builder

/opt/splunk/var/log/splunk/metrics.log contains information about incomming connections from forwarders, by default these events indexed under _internal index.

0 Karma

adonio
Ultra Champion

hello there,

i guess there are couple different ways to achieve.
the way i approach this is by checking if splunk internal data is flowing. if it does -> all good, if it doesnt -> probably connection error or forwarder is down -> alert and check
here is a quick and dirty way to achieve it
| tstats count as event_count by host where index = _interanl
from there you can take it however you like it, the nice part about it is that |tstats takes into consideration the timepicker.
so you can schedule a report / alert
also, you can create a lookup with list of all forwarders and update it every week / day / hour etc, and then run a search that compare existing forwarders to that list

hope it helps

0 Karma

satkan100
Path Finder

Hi

Thanks for the your update.

| tstats count AS event_count WHERE index=_internal by host from this query i am able get the details forwarder details. if any possible to create dashboard from this query forwarder on or off status?

0 Karma

adonio
Ultra Champion

the purpose of the query above is to tell you if a forwarder is not sending internal data, which might indicate that its down.
sure, set your threshold for the time you would like to be alerted on and save this search as a scheduled report.
add the report to a dashboard.
if it answered your question, please mark as answered

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...