How to determine time of day when web requests are slowest

[cmeredith]

I'm new to Splunk and I have a question about how to query the information I need.

I'm indexing IIS web server logs.

I would like to determine if the server is responding slowly at certain times of the day for the past few months. So, my question would be: how do I produce a graph showing average and/or max request times (time_taken) for each hour? The graph would have 24 bars (or 48 bars in pairs of 2 if showing max and avg together) representing the hours of the day and the query would control how many days that graph represents.

Thanks for your help! So far, so good with Splunk!

• Corey

[sideview · Accepted solution]

1) change field names and values below as necessary, but using either 'report builder' or the 'advanced charting' view, run the following.

sourcetype=access_combined | chart avg(request_time) max(request_time) over date_hour | sort date_hour

This will give you a chart that has distribution of hours across the x-axis and the request times on the y-axis. You probably want to set it to use a line chart if it isnt already. And the sort command is just so the table rows come out in a sensible order.

Note: If your data is very sparse such that there can sometimes be no values at all for a given date_hour, columns may dissappear entirely from the table and chart.

2) A more advanced idea but in a similar vein, is to see the same chart split up into one line per day of the week. And since here im using individual weekday values as my 'split', i can no longer graph both average and max.

sourcetype=access_combined | chart avg(request_time) over date_hour by date_wday | fields date_hour sunday monday tuesday wednesday thursday friday saturday

(the fields clause at the end is to make it correctly order the columns in the table, and the legends in the chart)

3) and here's a search that anyone with a splunk instance can run. It shows the same kind of pattern but with throughput by splunk index x weekday x hour of day.

index=_internal source=*metrics.log group=per_index_thruput | chart sum(kb) over date_hour by date_wday | fields date_hour sunday monday tuesday wednesday thursday friday saturday

[cmeredith]

Wow! Thanks, Nick. I really appreciate it!

• Corey