Splunk Search

How to check the universal forwarder's metrics.log to get instantaneous_kbps and average_kbps?

kteng2024
Path Finder

From Documentation:

To verify how often the forwarder is hitting this limit, check the forwarder's metrics.log. (Look for this on the forwarder because metrics.log is not forwarded by default on universal and light forwarders.)

cd $SPLUNK_HOME/var/log/splunk/metrics.log
grep "name=thruput" metrics.log

Example: The instantaneous_kbps and average_kbps are always under 256KBps.

11-19-2013 07:36:01.398 -0600 INFO  Metrics - group=thruput, name=thruput, instantaneous_kbps=251.790673,instantaneous_eps=3.934229, average_kbps=110.691774, total_k_processed=101429722, kb=7808.000000, ev=122

But when i run the grep -i "name=thruput" metrics.log , i wouldn't get any result. So, can i please know whether there is any way to check the instantaneous_kbps and average_kbps ?

0 Karma

woodcock
Esteemed Legend

Try this:
|tstats avg(tcp_KBps) AS "KBps" avg(tcp_eps) AS "Eps"
WHERE index=_internal AND sourcetype="splunkd" AND connectionType IN("cooked", "cookedSSL")
AND hostname="*"
BY hostname _time [| makeresults count=2
| streamstats count
| eval _time = _time - (100 * count)
| timechart count
| rename _* AS INV_*
| eval search = "span=" . INV_span . "s" ]
| rename hostname AS host
| timechart limit=0 useother=f first(KBps) AS KBps first(Eps) AS Eps BY host

0 Karma

woodcock
Esteemed Legend

Why bother going to the forwarder, just do this:

index=_internal source="*metrics.log" component="Metrics" host=MyHost
0 Karma

GregZillgitt
Path Finder

Can you load metrics.log into vi and search there?

I just ssh'd into one of our forwarders and it worked for me...

$ grep -i "name=thruput" metrics.log
01-26-2017 15:16:23.841 -0600 INFO  Metrics - group=thruput, name=thruput, instantaneous_kbps=0.202834, instantaneous_eps=0.257975, average_kbps=19.430535, total_k_processed=4248380.000000, kb=6.290039, ev=8.000000, load_average=2.010000
01-26-2017 15:16:54.829 -0600 INFO  Metrics - group=thruput, name=thruput, instantaneous_kbps=0.219149, instantaneous_eps=0.613138, average_kbps=19.427809, total_k_processed=4248386.000000, kb=6.791016, ev=19.000000, load_average=1.610000
01-26-2017 15:17:25.829 -0600 INFO  Metrics - group=thruput, name=thruput, instantaneous_kbps=0.209961, instantaneous_eps=0.258064, average_kbps=19.425083, total_k_processed=4248392.000000, kb=6.508789, ev=8.000000, load_average=1.500000
0 Karma
Get Updates on the Splunk Community!

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...

Updated Data Management and AWS GDI Inventory in Splunk Observability

We’re making some changes to Data Management and Infrastructure Inventory for AWS. The Data Management page, ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...