Searching: Did event X occur < Z minutes after event Y?

[kobailey]

Hi,

I am interested in alerting on the following scenario:

• A "generate" event occurs and a "delete" event is not seen in the next 10 min.

Right now I have something similar to the following:

index=myindex type=generate OR type=delete | get a consistant field from both event types | transaction field | search eventcount < 2 | table field 1,2,3

I am generating false positives in the following scenario:

• Alert runs over data from 1pm-2pm, the generate event happened at 1:55 and the delete at 2:01, this is valid in the 10 min window but the search as I have it will trigger an event.

Is there a better way to go about this maybe using map or a subsearch?

Thanks!

[mayurr98]

hey you can change your query to establish the results you wanted:
If you are able to achieve the same using using stats command then I suggest you to not use transaction but you can give it a try!

index=myindex type=generate OR type=delete | get a consistant field from both event types | transaction commonfield  startswith=(type="generate")  | eval Not_cleared_Within_10_mins=case(type="generate" and (duration=0 OR duration>600),"TRUE") | search Not_cleared_Within_10_mins=TRUE | table commonfield 1,2,3

If you have standard logs and for every generate event there is delete event then you should try this:

index=myindex type=generate OR type=delete | get a consistant field from both event types | transaction commonfield  startswith=(type="generate") endswith=(type="delete") | eval Not_cleared_Within_10_mins=case(type="generate" duration>600),"TRUE") | search Not_cleared_Within_10_mins=TRUE | table commonfield 1,2,3

Let me know if this helps!

[micahkemp]

You can accomplish this with streamstats. A run anywhere example of this in action is:

| makeresults | eval type="delete", _time=5000, commonfield="B"
| append [| makeresults | eval type="generate", _time=1005, commonfield="C"]
| append [| makeresults | eval type="delete", _time=1000, commonfield="A"]
| append [| makeresults | eval type="generate", _time=995, commonfield="A"]
| append [| makeresults | eval type="generate", _time=995, commonfield="B"]
| eval delete_time=if(type="delete", _time, NULL), generate_time=if(type="generate", _time, NULL)
| streamstats current=false last(delete_time) AS last_delete_time BY commonfield
| eval time_between_generate_delete=last_delete_time-generate_time
| eval is_not_cleared_within_10_mins=if(type="generate" and (isnull(time_between_generate_delete) or time_between_generate_delete>600), "TRUE", NULL)