Be aware: Reviving an old thread and just skimming the comments
Also, some assumptions are made and keep in mind that the search is NOT OPTIMIZED AT ALL:
Your field userCount is already extracted (with xmlkv for example).
In this example all the test data is in the sourcetype "test4".
Assumption that alerting is wanted based on the increased amount of users.
Assumption that the total amount of userCounts in the whole hour are to be calculated.
Search is scheduled hourly and goes over the data from an hour before.
So the monster of a search that makes this happen:
sourcetype="test4" | eval currenthour=strftime(now()-3600, "%H") | eval match=if(date_wday=lower(strftime(now(), "%A")), "T","F") | eval match2=if(date_hour=currenthour,"T","F") | where match="T" and match2="T" | stats sum(userCount) by date_wday,date_hour,date_mday | stats avg(sum(userCount)) as AVERAGE | appendcols [search sourcetype=test4 | eval currenthour=strftime(now()-3600, "%H") | eval match=if(date_wday=lower(strftime(now(), "%A")), "T","F") | eval match2=if(date_hour=currenthour,"T","F") | where match="T" and match2="T" | stats latest(userCount) as LATEST]| eval PERC=(((LATEST-AVERAGE)/AVERAGE)*100) | table PERC
The basic thought of this search is to run this every hour and calculates the percentage of growth based on the average of the month. It's really hard (if even possible) to exclude the last event's field value from the average. So what is done is we take the average of the whole month and compare this with the value of the last value.
So the first three eval's are to fill some fields, like the current hour (offset by 1 hour since we go through the data from the previous hour) and two checks to filter out all the data that is not needed. The unnecessary data is filtered out with a where match="T" AND match2="T". Unnecessary data is data which is not in the same hour and week day span. Then we sum all the userCounts based on the weekday, the hour and the day of the month so we get four buckets containing the summed amount of userCounts for that specific hour on this specific weekday. Now we can do an average over these four summed values and append the summed value of the latest day. This we do with a sub search which again filters out the unnecessary data and get the latest value of sum(userCount) by weekday, the hour and the day of the month.
At this stage we are left with two values, an average of all 4 weekdays of the month and the current value of that weekday. With this we can calculate a percentage of growth for this hour, at this weekday compared with the average of this hour of this weekday of the whole month.
Now you could do some summarization or alerting based on this percentage growth, but be aware that this search uses the current hour and weekday!
Try and dissect the search and understand it on your own and see if this works for your dataset: I tested this on a months worth of self created data which only had a timestamp and a value for userCount (e.g. 17 Aug 2012 08:57:46 userCount=20)
Good luck!
... View more