Monitoring Splunk

Can I Add selected heavy forwrders in splunk monitoring console in forwarder section?

splbsm
Explorer

HI,

I want to be able to add only a few selected heavy forwarders in my distributed monitoring console.
so basically I want to use wildcard (or may be a text file with list of forwarders or something similar ) for hostnames of these HFs and only add these matching HF's in my Monitoring console in forwarder section..
Is this possible in splunk ?

Regards

Tags (2)
0 Karma
1 Solution

DMohn
Motivator

You cannot select to have only a few forwarders in the Monitoring Console, as this is depending on having the "full view" of everything going on in your environment.

But you could go for the following solution:

  • Add all forwarders to the MC
  • Use the dmc_forwarder_assets alert
  • limit the search results to the HFs relevant for you by creating a lookup file

A search could look like this:

| inputlookup dmc_forwarder_assets
  [|inputlookup your_hf_list.csv | return hostname]
| search status="missing"

This should give you a list of all missing HF out of your selection.

View solution in original post

DMohn
Motivator

You cannot select to have only a few forwarders in the Monitoring Console, as this is depending on having the "full view" of everything going on in your environment.

But you could go for the following solution:

  • Add all forwarders to the MC
  • Use the dmc_forwarder_assets alert
  • limit the search results to the HFs relevant for you by creating a lookup file

A search could look like this:

| inputlookup dmc_forwarder_assets
  [|inputlookup your_hf_list.csv | return hostname]
| search status="missing"

This should give you a list of all missing HF out of your selection.

splbsm
Explorer

You mean "DMC Forwarder - Build Asset Table " Alert please?

0 Karma

splbsm
Explorer

Also, I have some 60k UF's. if I add them all , will it not be a huge risk for my MC performance? Please advice.

0 Karma

DMohn
Motivator

Start by enableing the forwarder monitoring in the MC with Settings => Forwarder Monitoring Setup. You can reduce the data colletion interval if you desire.

This will enable the MC to run internal saved searches, one of which builds the forwarder asset table. This can be accessed in a regular Splunk search with | inputlookup dmc_forwarder_assets. From there on you can build your custom alert which will only cover your selected Heavy Forwarders. This is not a built-in MC alert anymore, as the standard alerts will alert for any missing forwarder. So your should leave these alerts turned off.

And no, this will not be a huge performance risk for your instance if you have sized it accordingly. Be aware that a MC used for such a large architecture has to be a standalone instance, with no other funcionalities.

0 Karma

splbsm
Explorer

yes, it is standalone MC. thanks..

0 Karma

DMohn
Motivator

If this works for you, could you please mark the answer as accepted, so others will so that there's a solution? Thanks 😉

0 Karma

chrisyounger
SplunkTrust
SplunkTrust

Hi @splbsm

If you are a customer that has a few heavy forwarders then it probably means you are large enough that you should consider having a stand-alone monitoring console. With a stand-alone monitoring console you should only add the heavy forwarders you care about as search peers. This way your less important heavy forwarders won't be displayed in the monitoring console. Once a server is defined as a search peer to the monitoring console it will be displayed. You can't filter it using a wildcard.

All the best.

splbsm
Explorer

Any Other Idea please?

0 Karma

splbsm
Explorer

Thanks for taking time for this post Chris, much appreciated.

Yes, at the moment I already have these HF's added as indexers.
But I'd like to add and see these HF's as HF in monitoring console in the forwarder section.
as you already guessed, I can NOT add all UF and HFs there because I have 1000s of UF's.

So Can I not add only a few select HF's in as HFs in monitoring console under forwarders?

Regards

0 Karma

splbsm
Explorer

Any other Idea please? anyone?

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...