How to gather a span of 5 Seconds for the Max EPS/TPS for a given Day Span?

[mgrimes]

So I've posted a question a week ago regarding finding the max EPS for a timespan of a day. The query that I am using (currently from Somesoni2) is as follows:

index="eps_summary"| timechart span=1s max(count) as Total | eval Date=strftime(_time,"%m/%d/%Y") | eventstats max(Total) as max_eps by Date | where Total=max_eps

It results with the following desired format:

_time               Total     Date    max_eps
2016-07-04 21:04:09   130    07/04/2016 130
2016-07-05 00:51:46 54  07/05/2016  54

It allows me to gather and see the time that the max EPS was achieved by the day. However, I'm currently at a dilemma where I would like to easily calculate whether or not these EPS are sustained over a period of time.

For example, I would like to span or tail the events after the spike in max EPS showed seconds. I'm not sure if I would make this a separate field/column, but rather just increase the limit to show something like:

_time               Total     Date    max_eps
2016-07-04 21:04:09   130    07/04/2016 130
2016-07-04 21:04:10   125    07/04/2016 125
2016-07-04 21:04:11   100    07/04/2016 100
2016-07-04 21:04:12   10     07/04/2016 10
2016-07-04 21:04:13   75     07/04/2016 75
2016-07-04 21:04:14   70     07/04/2016 70
2016-07-04 21:04:15   90     07/04/2016 90
2016-07-05 00:51:46   54     07/05/2016 54

This is bad representation that I can already see can be modified to express a better visually appealing Statistic however I'm limited by what I know what to do. I've tried to modify the query to add a limit similar to this:

index="eps_summary"| timechart span=1s limit = 5 max(count) as Total | eval Date=strftime(_time,"%m/%d/%Y") | eventstats max(Total) as max_eps by Date | where Total=max_eps | fields

But it seems that simply adding the limit doesn't show the 5 max(count) EPS at all. Should I approach this query in a different way or what can I do to make representing the statistic information that I desire easier to read or organize?

Thanks for looking into this ahead of time.

[somesoni2 · Accepted solution]

Give this a try.
Update- fixed type on streamstats
Update-corrected query description and updated the query

This should give you 5 events after the max eps for the day, including row with max eps.

index="eps_summary"| timechart span=1s max(count) as Total | eval Date=strftime(_time,"%m/%d/%Y") | streamstats window=6 current=t max(Total) as max_eps_comm by Date  | eventstats max(Total) as max_eps_daily by Date| where max_eps_comm=max_eps_daily

[mgrimes]

Got the following error as follows:
Error in 'streamstats' command: Invalid option value. Expecting a 'boolean' for option 'current'. Instead got '6'

[mgrimes]

When changing it to a boolean value, I do get a series of statistical charts, however it appears to remain Per-Second

URL: http://imgur.com/XnRxrE6

[somesoni2]

I see so many additional columns in your snapshot. Could you post the exact search that you're trying?

[mgrimes]

Here's a URL: https://imgur.com/OMmAzEJ

I apologize, I must have kept a streamstats at the end of the search when testing. This is the query with the result.

However, shouldn't the max_eps be all the same for all the values before and after to show grouping for each max EPS?

[mgrimes]

_time                 Total                Date                    max_eps
2016-07-06 16:35:12 12  07/06/2016  12
2016-07-06 16:35:13 20  07/06/2016  20
2016-07-06 16:35:15 25  07/06/2016  25
2016-07-06 16:35:23 27  07/06/2016  27
2016-07-06 16:35:29 23  07/06/2016  23
2016-07-06 16:35:33 23  07/06/2016  23

Sample data of the query for index="eps_summary"| timechart span=1s max(count) as Total | eval Date=strftime(_time,"%m/%d/%Y") | streamstats window=6 current=t max(Total) as max_eps by Date | where Total=max_eps

[somesoni2]

Try the updated query. (and see the updated description as well)

[mgrimes]

That looks perfect, yet again I am amazed with what Splunk can do.

This is exactly what I wanted:

_time   Total   Date    max_eps_comm    max_eps_daily
2016-07-01 23:31:34 57  07/01/2016  57  57
2016-07-01 23:31:35 42  07/01/2016  57  57
2016-07-01 23:31:36 18  07/01/2016  57  57
2016-07-01 23:31:37 47  07/01/2016  57  57
2016-07-01 23:31:38 35  07/01/2016  57  57
2016-07-01 23:31:39 26  07/01/2016  57  57
2016-07-02 22:46:41 82  07/02/2016  82  82
2016-07-02 22:46:42 56  07/02/2016  82  82
2016-07-02 22:46:43 32  07/02/2016  82  82
2016-07-02 22:46:44 59  07/02/2016  82  82
2016-07-02 22:46:45 45  07/02/2016  82  82
2016-07-02 22:46:46 28  07/02/2016  82  82