Splunk Enterprise Security

Splunk Enterprise Security: How to set up an alert when F5 LB is "down" for more than 15 minutes without an "up" message?

wtaylor149
Explorer

I'm trying to setup a search to alert in ES when F5 LB is down for more than 15 minutes. The F5 LB only sends messages when there is a status change, i.e. the LB can't reach our vendor (MSSP).

index=f5 my_search_query_here | rex "-TCP member /Common/(?<my_vendor_host>\S+) monitor status (?<status>\w+)" | rex "\[ was (?<previous_state>[^\]]*)" | rex "mcpd\[\d+\]\:\s(?<message_code>\d+\:\d+)" | transaction my_vendor_host, status, message_code maxspan=45s maxevents=2 | rename host AS F5_Host | table _time F5_Host my_vendor_host status previous_state

Currently the search will pull both an up and down message. I'm looking to have it send an alert only when "down" message doesn't have an "up" message since the last time the search ran.

So, if the search runs every 15 minutes, search runs at top of the hour, finds a down message and based off of the host, does not find an up message in that same time frame, send an email. If that search finds a down, followed by an up message, don't send an email.

I hope I made sense. Thank you Splunk experts in advance.

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

What about the below search

<your search> |stats latest(status) as status by host|search status="down" 

This should give you the latest status of the host and then search for down which gives you all hosts which don't have "up" status as latest status and then set an alert if no of events > 0

Hope this helps

Happy Splunking!
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...