All Apps and Add-ons

Filtering out nfs-mounted filesystems for DF_STORAGE data

Ranazar
Path Finder

I noticed that the DF_STORAGE data from nmon (eventtype="nmon:performance" type="DF_STORAGE") includes stats for NFS-mounted filesystems as well - and they also show up in the datamodel NMON_Data_JFS (where nodename=JFS.DF_STORAGE).

Right now I'm just filtering these out by specifying filesystem="/dev*" - is there a better way to exclude them?

I'm using v1.3.31 of the addon on the servers themselves, and v1.9.15 of the app (community edition)..

0 Karma
1 Solution

guilmxm
Influencer

Hi @Ranazar

The DF_STORAGE data collection is driven by an external data collection documented here:

https://ta-nmon.readthedocs.io/en/latest/external.html

In a few lines, this is managed by the following script:

TA-nmon/TA-nmon/bin/nmon_external_cmd/nmon_external_snap.sh

And specially this variable being set at the script execution:

DF_TABLE=`df -k -P | sed '1d' | egrep -v '\/proc$|/dev$|\/run$|^tmpfs.*\/dev.*$|^tmpfs.*\/run.*$|^tmpfs.*\/sys.*$|^tmpfs.*\/var.*$' | awk '{print $6}'`

If you run this command on one of the boxes:

df -k -P | sed '1d' | egrep -v '\/proc$|/dev$|\/run$|^tmpfs.*\/dev.*$|^tmpfs.*\/run.*$|^tmpfs.*\/sys.*$|^tmpfs.*\/var.*$'

This would provide the list of the file-systems that will be taken in charge by the data collection.

So you could update this to exclude additional file-systems, if you modify the script and push the update, this will be taken in charge at the next life cycle of the nmon binary (max 24h per server)

Can you show what the file-system looks like ?
It might makes sense for me to update this to exclude NFS mounted file-systems.

Guilhem

View solution in original post

0 Karma

guilmxm
Influencer

Hi @Ranazar

The DF_STORAGE data collection is driven by an external data collection documented here:

https://ta-nmon.readthedocs.io/en/latest/external.html

In a few lines, this is managed by the following script:

TA-nmon/TA-nmon/bin/nmon_external_cmd/nmon_external_snap.sh

And specially this variable being set at the script execution:

DF_TABLE=`df -k -P | sed '1d' | egrep -v '\/proc$|/dev$|\/run$|^tmpfs.*\/dev.*$|^tmpfs.*\/run.*$|^tmpfs.*\/sys.*$|^tmpfs.*\/var.*$' | awk '{print $6}'`

If you run this command on one of the boxes:

df -k -P | sed '1d' | egrep -v '\/proc$|/dev$|\/run$|^tmpfs.*\/dev.*$|^tmpfs.*\/run.*$|^tmpfs.*\/sys.*$|^tmpfs.*\/var.*$'

This would provide the list of the file-systems that will be taken in charge by the data collection.

So you could update this to exclude additional file-systems, if you modify the script and push the update, this will be taken in charge at the next life cycle of the nmon binary (max 24h per server)

Can you show what the file-system looks like ?
It might makes sense for me to update this to exclude NFS mounted file-systems.

Guilhem

0 Karma

Ranazar
Path Finder

Perfect, thanks! That gives me a good option to filter it out at collection type.

The filter is a bit of an interesting challenge. I only caught this because one of the common mount points - /mnt - showed up in the data, but we've got NFS filesystems on various mount points other than /mnt.

It looks like the filesystems we'd want to include always start with /dev, so a piped command like | egrep '^\/dev' would catch them all. I took a look at a Linux server (we don't have that many), and it looks like the ones which don't start with /dev are already being filtered out by your egrep -v:

$ df -k -P
Filesystem 1024-blocks Used Available Capacity Mounted on
/dev/mapper/root xxx xxx xxx 32% /
devtmpfs xxx xxx xxx 0% /dev
tmpfs xxx xxx xxx 0% /dev/shm
tmpfs xxx xxx xxx 3% /run
tmpfs xxx 0 xxx 0% /sys/fs/cgroup
/dev/mapper/lvol0 xxx xxx xxx 19% /fs1
/dev/sda1 xxx xxx xxx 26% /boot
tmpfs xxx 0 xxx 0% /run/user/1417441623
tmpfs xxx 0 xxx 0% /run/user/995`

Alternatively, I see a -T local flag in the AIX df command that takes care of it nicely; I'm not familiar enough with the Linux version though (maybe -x).

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

REGISTER NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If ...

Observability | Use Synthetic Monitoring for Website Metadata Verification

If you are on Splunk Observability Cloud, you may already have Synthetic Monitoringin your observability ...

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...