Dynamic Alerts for 300 ID,s bases on Field values coming to Splunk

[manikanthkoti]

Hi ,

We have 300 Queues which continually stores the data into Splunk every 5 mins. Each queue there is a Thresholdtime and Riskpoint and Message_in_Queue value.(Thresholdtime and Riskpoint -- Constant)

Requirement Need to Generate dynamic alerts for Queue_Names if that Queue_Name contains Message_in_Queue value continually grater then Riskpoint Value in that Threshold Time.

Example Data:

For example here Queue_Name B Contains Message_in_Queue Value as 20000 which is greater than Riskpoint continually for 5 mins.
So for B we need to raise the Alerts.

Please, anyone, help me in this case as this is a complex scenario.

[to4kawa]

UPDATE:

| makeresults 
| eval _raw="Queue_Name,Time,Message_In_Queue,ThresholdTime,Riskpoint
A,05-02-2020:01-33-40,10000,10mins,1000
B,05-02-2020:01-33-40,20000,5mins,2000
C,05-02-2020:01-33-40,25000,15mins,2050
D,05-02-2020:01-33-40,3000,5mins,150
A,05-02-2020:01-28-40,10000,10mins,1000
B,05-02-2020:01-28-40,20000,5mins,2000
C,05-02-2020:01-28-40,250,15mins,2050
D,05-02-2020:01-28-40,30,5mins,150" 
| multikv forceheader=1 
| table Queue_Name,Time,Message_In_Queue,ThresholdTime,Riskpoint 
| rename COMMENT as "this is your sample. from here, the logic" 
| reverse 
| eval time=strptime(Time,"%d-%m-%Y:%H-%M-%S") 
| streamstats dc(Message_In_Queue) as session by Queue_Name 
| eval ThresholdTime=tonumber(rtrim(ThresholdTime,"mins")) * 60 
| stats range(time) as Duration ,values(Message_In_Queue) as Message_In_Queue
    ,values(Riskpoint) as Riskpoint ,values(ThresholdTime) as ThresholdTime by Queue_Name session 
| where ThresholdTime <= Duration AND Message_In_Queue > Riskpoint

I recommend that time picker is short time range.

| makeresults 
| eval _raw="Queue_Name,Time,Message_In_Queue,ThresholdTime,Riskpoint
A,05-02-2020:01-33-40,10000,10mins,1000
B,05-02-2020:01-33-40,20000,5mins,2000
C,05-02-2020:01-33-40,25000,15mins,2050
D,05-02-2020:01-33-40,3000,5mins,150
A,05-02-2020:01-28-40,10000,10mins,1000
B,05-02-2020:01-28-40,20000,5mins,2000
C,05-02-2020:01-28-40,250,15mins,2050
D,05-02-2020:01-28-40,30,5mins,150" 
| multikv forceheader=1 
| table Queue_Name,Time,Message_In_Queue,ThresholdTime,Riskpoint 
| eval check=if(tonumber(rtrim(ThresholdTime,"mins")) * Riskpoint < Message_In_Queue,"Alert","No") 
| where check="Alert"

[manikanthkoti]

Hi Kanagawa,

Thanks for your response but this is not I want .We have to check overall data for each Queue_Name if the value is continuously greater then Rispoint in that given threshold time we need to raise the Alert.

[to4kawa]

sorry, Kanagawa is a prefecture name. That's a good place.
My answer is updated.