Getting Data In

How can I set an alert for max thruput?

mamir32825
New Member

What is the search query to alert when the forwarder reaches max thruput?

0 Karma

gjanders
SplunkTrust
SplunkTrust

I have an example in this savedsearches.conf from the SplunkAdmins app that I created. My search is similar to other answers but I have used:

index=_internal "has reached maxKBps. As a result, data forwarding may be throttled" sourcetype=splunkd
| stats count(_raw) by host as countPerHost 
| where countPerHost > 1

gcusello
SplunkTrust
SplunkTrust

Hi
In Distributed Monitoring Console you can find a dashboard panel that list forwarder's connections.

| inputlookup dmc_forwarder_assets 
| eval avg_tcp_kbps = if (status == "missing", "N/A", avg_tcp_kbps) 
| fields hostname status avg_tcp_kbps
| where avg_tcp_kbps>threeshold

From this search you can find the Average kb/s and put a threeshold alert.

Bye.
Giuseppe

0 Karma

inventsekar
Ultra Champion

from an answer by @burwell at the post - Here's what I do to detect throttled forwarders. I have a scheduled search for last 4 hours (-240m to now) and then alert for any events:

 index=_internal " INFO " " throttled" NOT debug source=*splunkd.log*  | dedup host |sort host|  table host _raw

This gives me a nice table per host and I can see the hosts and what the thruput is that is getting throttled. Example output:

 foo1.host.com        10-22-2017 18:26:28.131 +0000 INFO ThruputProcessor - Current data throughput (258 kb/s) has reached maxKBps. As a result, data forwarding may be throttled. Consider increasing the value of maxKBps in limits.conf.
 foo2.host.com        10-22-2017 18:29:28.324 +0000 INFO ThruputProcessor - Current data throughput (512 kb/s) has reached maxKBps. As a result, data forwarding may be throttled. Consider increasing the value of maxKBps in limits.conf.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...