I have a slightly different requirement where the missing forwarders would blow out due to constantly changing Citrix clients, so my first comment below about dismissing the "DMC Forwarder - Build Asset Table" is because its appending to the existing table.
If you check the Job Activity page on the Monitoring Console you will see two jobs that show up:
1. DMC Forwarder - Build Asset Table
2. `dmc_re_build_forwarder_assets(48m)`
The first is scheduled to run every 15 minutes and can be ignored.
The second is the job that is initiated when you "Rebuild forwarder assets" (previous 24 hours defines the 48m sparkline argument), this is what we are interested in.
If you follow the bouncing ball you can figure out what it's doing yourself, to get you started:
# grep "dmc_re_build_forwarder_assets(1)" /opt/splunk/etc/apps/splunk_monitoring_console/default/macros.conf
[dmc_re_build_forwarder_assets(1)]
I'd then recommend you create a scheduled search with " dmc_re_build_forwarder_assets(48m) ".
This can then be called via REST:
curl --silent -k -u admin https://localhost:8089/servicesNS/admin/splunk_monitoring_console/saved/searches/YOUR_SAVED_SEARCH/dispatch -d trigger_actions=1
If you're security conscious and want to put this into a then I'd recommend doing two things:
1. https://stackoverflow.com/questions/33794842/forcing-curl-to-get-a-password-from-the-environment/33818945#33818945
2. Create a dedicated user and role
The role can be tightend and given just the below:
Restrict search terms: index=_internal sourcetype=splunkd
Capabilities: admin_all_objects, output_file, search
Available search indexes: _internal
I kept running into permission issues unless I gave the "admin_all_objects" capabilitiy. Not sure if I missed something or if this is due to some Monitoring Console magic.
The capabilities above are fairly locked down, so I feel like it is a reasonable compromise.
... View more