Is there an accepted way to adjust the interval of certain stanzas remotely and on the fly?
Say, I want to casually monitor some perfmon metric most of the time but every now and again I need more accurate and timely data and so I would decrease the monitor interval.
I am thinking I will write a simple script to change this value and restart the forwarder when needed, and tie that script to an alert.
Any other recommendations?
It's either that which you mentioned "script to change... restart...", or you could use a deployment-server & app. You could also create your own Splunk app and code it with the ability to change these via the web ui.