Is there an app or collection of saved searches anybody has that would monitor and graph out all parts of the TCP connection setup between a splunk forwarder and indexer? Does anybody have any favorite searches used on the metrics.log?
here's a few of them, credits go to Simeon:
Which IP addresses are connecting to Splunk as inputs and how many times is it logged in metrics.log?
index=_internal source=metrics.log tcpin_connections | stats count by sourceIp
Where is Splunk trying to forward data to?
index=_internal source=metrics.log destHost | dedup destHost
What output queues are setup?
index=_internal source=metrics.log group=queue tcpout | stats count by name
What hosts (not forwarder/tcp inputs) have logged an event to splunk in the last 10 minutes (includes rangemap
| metadata type=hosts index=netops | eval diff=now()-recentTime | where diff < 600 | convert ctime(*Time) | stats count | rangemap field=count low=800-2000 elevated=100-799 high=50-99 sever=0-49
Genti's answer is great for searching... However, there is a "deployment monitor" app that is embedded in the Splunk 4.2.x product line.
here's a few of them, credits go to Simeon:
Which IP addresses are connecting to Splunk as inputs and how many times is it logged in metrics.log?
index=_internal source=metrics.log tcpin_connections | stats count by sourceIp
Where is Splunk trying to forward data to?
index=_internal source=metrics.log destHost | dedup destHost
What output queues are setup?
index=_internal source=metrics.log group=queue tcpout | stats count by name
What hosts (not forwarder/tcp inputs) have logged an event to splunk in the last 10 minutes (includes rangemap
| metadata type=hosts index=netops | eval diff=now()-recentTime | where diff < 600 | convert ctime(*Time) | stats count | rangemap field=count low=800-2000 elevated=100-799 high=50-99 sever=0-49