Splunk Search

How do I use IF / THEN / AND in logic in a search?

tmarlette
Motivator

I'm trying to create some logic within my search, and it requires some IF THEN AND logic, which I know Splunk has the capability to do, but I don't know how to make it work the way I'm needing it.

I have 2 different types of machines I'm searching, and I'm trying to alert on two distinct values.

example: if machines named host10* have a mount with mount=/boot, AND have drive space over 90% then alert, AND if machines named host20* have a mount with mount=/boot AND drive space over 95% alert.

Working Query:

index=nix sourcetype=df host=myHost10 * OR host=myHost20* 
| stats first(PercentUsedSpace) as pctUsed latest(Avail) as Avail by host, Filesystem, filesystem_type, Size, Used, MountedOn 
| where pctUsed > 90
| sort - pctUsed 

I thought about using |eval field=if(coalesce...) but I don't think it fits my needs here, as both host types will have a value, it's just that the value needs to be filtered differently based upon the system type. maybe a subsearch?

Any help would be appreciated.

0 Karma
1 Solution

tmarlette
Motivator

I tried all of these, but alas... there was no success. I couldn't get the filter to populate all the results I needed. In the end I simply broke this into two alerts, 1 with filters around the mount and the pctUsed, and the other around everything EXCEPT that mount, but still with the pctUsed.

View solution in original post

0 Karma

tmarlette
Motivator

I tried all of these, but alas... there was no success. I couldn't get the filter to populate all the results I needed. In the end I simply broke this into two alerts, 1 with filters around the mount and the pctUsed, and the other around everything EXCEPT that mount, but still with the pctUsed.

0 Karma

alemarzu
Motivator

Hi there @tmarlette

Try something like this.

index=nix sourcetype=df host=myHost10 * OR host=myHost20* 
 | stats first(PercentUsedSpace) as pctUsed latest(Avail) as Avail by host, Filesystem, filesystem_type, Size, Used, MountedOn 
 | eval condition=case(host="host10*" AND MountedOn="/boot" AND pctUsed> 90, "1", machine="host20*" AND MountedOn="/boot" AND pctUsed > 95, "1", 1==1, "0")
 | sort - pctUsed 
 | where condition=="1"

Hope it helps.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try this query.

 index=nix sourcetype=df (host=myHost10 * OR host=myHost20*) mount="/boot"
 | stats first(PercentUsedSpace) as pctUsed latest(Avail) as Avail by host, Filesystem, filesystem_type, Size, Used, MountedOn 
 | search (host=myHost10* pctUsed > 90) OR (host=myhost20* pctUsed>95)
 | sort - pctUsed
---
If this reply helps you, Karma would be appreciated.
0 Karma

tmarlette
Motivator

This would work, if I only had to check the /boot mount, but I need to show the results of all other mounts on the system as well. =(

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Try like this

 index=nix sourcetype=df host=myHost10 * OR host=myHost20* 
 | stats first(PercentUsedSpace) as pctUsed latest(Avail) as Avail by host, Filesystem, filesystem_type, Size, Used, MountedOn 
 | where (like(host,"myHost10%) AND pctUsed > 90) OR (like(host,"myHost20%) AND pctUsed > 95) 
 | sort - pctUsed 
0 Karma

tmarlette
Motivator

This seems so close. I just have to choose a mount on my linux systems, otherwise the data is pointless. This is what I have

 index=nix sourcetype=df host=myHost10 * OR host=myHost20* 
 | stats first(PercentUsedSpace) as pctUsed latest(Avail) as Avail by host, Filesystem, filesystem_type, Size, Used, MountedOn 
| where (like(MountedOn,"%") AND pctUsed > 90) OR (like(MountedOn,"home/work%") AND pctUsed > 95)
 | sort - pctUsed 

it's showing me weird results though. It's only showing me the machines that have 'home/work' mount ABOVE 90%.

  • Update, the last statement was my own stupidity, please ignore that. 🙂 It's just not filtering out the >95%. I think my filter is too vauge.
0 Karma

tmarlette
Motivator

So this is my filter now, I hope this explains what I'm trying to better:

  index=nix sourcetype=df host=myHost10 * OR host=myHost20*    | stats first(PercentUsedSpace) as pctUsed latest(Avail) as Avail by host, Filesystem, filesystem_type, Size, Used, MountedOn  | where (like(MountedOn,"%") AND pctUsed > 50 AND MountedOn!="/home/work*") OR (like(MountedOn,"home/work%") AND pctUsed > 95)   | sort - pctUsed

I'm looking for all mounts above 50%, and then any /home/work mounts that are above 95%, but I need them all displayed in a single table. The above query is still returning results with the /home/work below 95%.

here is an image

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Give this a try

index=nix sourcetype=df host=myHost10* OR host=myHost20*    | stats first(PercentUsedSpace) as pctUsed latest(Avail) as Avail by host, Filesystem, filesystem_type, Size, Used, MountedOn  | where (like(MountedOn,"home/work%") AND pctUsed > 95) OR   (NOT like(MountedOn,"home/work%") AND pctUsed > 50 ) | sort - pctUsed
0 Karma

tmarlette
Motivator

still got the same result set. =(

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...