Limit/Filter data indexed based on sourcetype volume

[kbrady]

Is there a way to limit the number of events indexed for a given sourcetype within a given amount of time?

For example, we have some windows hosts that can have highly variable data volumes. This variability occasionally leads to problems. We'd like to address this by accepting a certain amount of events, say up to 10k/minute on average, and drop any events that exceed that rate. A data size based limit might be workable if an event count limit isn't possible.

I know we can limit the KB/s sent by a forwarder. However, that's a global setting, and we'd like to do this for only specific sourcetypes. We also want to drop events when the limit is exceeded instead of queuing them for later delivery.

[kbrady]

That's what I thought, figured I'd at least ask before trying a feature request

Thank you.

[jbsplunk · Accepted solution]

The only setting like this is maxKBps, and as you've noted, it is global. You can't do rate limiting on forwarders based on source, host, sourcetype, etc.