Getting Data In

How can I identity forwarder data rate and index data rate (to identify a lag and prioritize logs)?

chintan_shah
Path Finder

Hi,
Is there any way where we can identify how much data the forwarder is sending and how much data is being indexed in real-time?
The problem is that I have a single forwarder that is sending data to a single indexer and its sending multiple logs i.e. 50 monitored files with different indexes. I am receiving data from a few indexes in real time whereas for some indexes I am having a lag, so I want to remove the lag and if possible give higher preferences to some logs file.

0 Karma

yannK
Splunk Employee
Splunk Employee

For forwarder lag, start to look at the metrics.log on the forwarder, if you see that it is hitting a plateau of kbps speed, it may be that you are hitting the default thuput limit.
see this article
http://docs.splunk.com/Documentation/Splunk/7.0.0/Troubleshooting/Troubleshootingeventsindexingdelay...

Also look at the timestamp, maybe is it a timezone issue.

0 Karma
Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer at Splunk .conf24 ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...