Refine your search:

1
1

I've got some performance data and I want to be alerted when the avg(total_requests) split by uri rises or drops by 10%. For now, you can aggregate a few hours worth of data in the average window.

asked 10 May '10, 23:59

lsheridan's gravatar image

lsheridan
111
accept rate: 0%

edited 11 May '10, 07:22

sideview's gravatar image

sideview ♦
33.3k6853


One Answer:

I think this question may need more detail. However if I make a couple assumptions I can attempt an answer. (total_requests I find to be a somewhat confusing field name so Im going to call it 'request_count')

Assuming you are starting from a search that looks like

<some search> | stats avg(request_count) over uri

which would give you output like:

uri             avg(request_count)
some/uri        12.3
some/other/uri  41.4
....

And you want your alert to fire if any of the rows rises by 10%...

And assuming you want the 'rises by 10%' to be based on a time range like comparing today to yesterday, then here's a way to do it.

<some search> | eval day = if(_time > now()-86400, "today", "yesterday") | chart avg(request_count) over uri by day

That conditional eval command puts a field called 'day' onto each event, which will be 'today' for all events today, and 'yesterday' for anything older than that.
(NOTE: timerange-wise you probably want to run this search over yesterday+today using -1d@d on the earliest side and +1d@d on the latest side)

Anyway, then the chart command after the eval gives you a table that looks like this:

uri             today      yesterday
some/uri        12.3       10.4
some/other/uri  41.4       10.5
....

Finally throw a where command on the end of that and you can filter the results down to only the uri's that actually had a 10% increase today over yesterday, like so:

<some search> | eval day = if(_time > now()-86400, "today", "yesterday") | chart avg(request_count) over uri by day | where today > (1.1 * yesterday)

And there you go. If that search returns any results, then that's bad, and you want to email those results to somebody.

(If you'd prefer the results in the email were actually the full list of URI's today vs yesterday, you could take that whole where clause off the end, and instead put it in the 'custom alerting condition' when you set up the alert. )

link

answered 11 May '10, 07:18

sideview's gravatar image

sideview ♦
33.3k6853
accept rate: 48%

Post your answer
toggle preview

Follow this question

Log In to enable email subscriptions

RSS:

Answers

Answers + Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×408
×339
×243

Asked: 10 May '10, 23:59

Seen: 920 times

Last updated: 25 May '10, 07:23

Copyright © 2005-2014 Splunk Inc. All rights reserved.