Alerting

create alert when http error code from a given IP reaches a threshold

cscchen
New Member

Hi,

I'd like to create alert for the following scenario:

in http access log, we have different return codes 2xx,3xx,4xx,5xx, etc. I'd like to get an alert when within a 10 minutes windows, the returned code 4xx for an source IP is higher than 10%. The alert will include all the source IPs that meet the percentage requirement as well as the percentage of 4xx. The alert would show sth. like the following:

source ip percentage of 4xx
a.b.c.d 20%
x.x.x.x 13%

One difficulty here is that new IPs keep showing up. Would really appreciate if anyone can help.

Thanks.

Tags (2)
0 Karma

chris
Motivator

Hi,

you could use a search similar to this (the status field contains the http status code an host would contain the ip for your example):

sourcetype="access_combined" | eval status_category=substr(status,0,1)+"xx" | stats count as total count(eval(status_category="4xx")) as bad_status by host | eval Percentage=100*round(bad_status/total,2) | where Percentage>10

Schedule the search to run in realtime over a window of 10 minutes and create one alert per result.

If you need help creating the alert let me know. If you need help with the search -> post some example events and let me know if the source ip & status are extracted as fields (if you do not know what a field is yet I'll explain).

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...