Alerting

How to create an alert to include client names, error counts, and the error log events that triggered the alert?

AmitKrJash
Explorer

Hi,

I have created an alert where it checks the status of the client accessing the application. The status will be either ERROR or SUCCESS. If the status is SUCCESS, then it is fine, but if it is ERROR, it should count the number of ERROR for particular clients and will pop up an email as an alert with the list of clients and their respective ERROR counts.

This is the search:

index="abc" sourcetype="xyz" STATUS=ERROR CLIENT_ID=*|stats count by CLIENT_ID

For example lets say I have 2 clients: Amit@xyz.com and Jash@xyz.com
So the final output which I am receiving as an alert for every 1 hour are:

Client_ID Count of ERROR
Amit@xyz.com 2
Jash@xyz.com 1

So that means there are 2 errors for Amit@xyz.com and 1 error for Jash@xyz.com in the log files. So lets say the errors in the log files are:

    2015-12-29 04:05:25 ERROR - [Client ID: Amit@xyz.com] - Client is not configured properly in the database
    2015-12-29 04:06:32 ERROR - [Client ID: Amit@xyz.com] - Client is not having enough permission to access the application

2015-12-29 04:07:21 ERROR - [Client ID: Jash@xyz.com] - SOAP Fault occurred

My question is, isthere any way to email these above mentioned log file lines along with the ERROR counts for clients in the alert? So the final alert which I should receive should be something like this:

Client_ID Count of ERROR
Amit@xyz.com 2
Jash@xyz.com 1

2015-12-29 04:05:25 ERROR - [Client ID: Amit@xyz.com] - Client is not configured properly in the database
2015-12-29 04:06:32 ERROR - [Client ID: Amit@xyz.com] - Client is not having enough permission to access the application

2015-12-29 04:07:21 ERROR - [Client ID: Jash@xyz.com] - SOAP Fault occurred

Any help on this will be greatly appreciated.

0 Karma
1 Solution

jplumsdaine22
Influencer

If you're happy for them to be in line with with counts Just add the raw values into stats:

index="abc" sourcetype="xyz" STATUS=ERROR CLIENT_ID=*|stats count values(_raw)  by CLIENT_ID 

See the description for values(X) in http://docs.splunk.com/Documentation/Splunk/6.3.2/SearchReference/CommonStatsFunctions

View solution in original post

jplumsdaine22
Influencer

If you're happy for them to be in line with with counts Just add the raw values into stats:

index="abc" sourcetype="xyz" STATUS=ERROR CLIENT_ID=*|stats count values(_raw)  by CLIENT_ID 

See the description for values(X) in http://docs.splunk.com/Documentation/Splunk/6.3.2/SearchReference/CommonStatsFunctions

AmitKrJash
Explorer

@jplumsdaine22 Thanks a lot for the help.. It is working for me now. I am able to pull up the entire Error logs in the alert. Is there a way to get the source and the host details in the alert mail from where the logs are getting generated. In the alert mail body I tried something like this

Source Log= '$source$'
Server = '$host$'
but it is not working.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...