It would be useful if you could add a field to your search results that indicates for that particular source how behind it is.
input_lag = convert_to_seconds( "date/time of the last event for the source" - "current date/time")
This can be very useful it you don't have a real time feed i.e. monitor.
Suggestion:
source=somesourcename | head 5 | eval duration=(now() - [search source=somesourcename | head 1 | fields + _time | rename _time as search])
The subsearch part returns the _time of the last event in that source.
Less overhead. Let splunk work for you!
http://splunk-base.splunk.com/answers/48731/determining-logging-lag-and-device-feed-monitoring
To see the lag time of the most recent event:
source=FOO | head 1 | eval lag_time = _indextime - _time
If you are running 4.1, you can use real-time search to decorate each event with its own lag time:
source=FOO | eval lag_time = time() - _time
or generate a distribution of lag times over some time period (by choosing a real-time window from the time picker):
source=FOO | eval lag_time = round(time() - _time, 1) | chart count by lag_time
Suggestion:
source=somesourcename | head 5 | eval duration=(now() - [search source=somesourcename | head 1 | fields + _time | rename _time as search])
The subsearch part returns the _time of the last event in that source.