Knowledge Management

Best practices for writing log files that have variable number of fields

burwell
SplunkTrust
SplunkTrust

We are writing our own logs for disk usage and we are using key value pairs. The issue is that each host has a different number of disk partitions. So my logs look like the below.

We are not sure what we will do with the data yet. Maybe alert on conditions and maybe collect trending data.
What do people typically do in this case? Thanks.

2017-01-27 02:48:00 db_dt="2017-01-27 02:12:00" hostname=myhost1 vol1 = "/dev/sda1" capacity1 = "706G" percentfull1 = "9%" vol2 = "tmpfs" capacity2 = "7.6G" percentfull2 = "1%" 
2017-01-27 02:48:00 db_dt="2017-01-27 02:12:00" hostname=myhost2 vol1 = "/dev/sda1" capacity1 = "2.4G" percentfull1 = "84%" vol2 = "tmpfs" capacity2 = "24G" percentfull2 = "1%" vol3 = "/dev/sda3" capacity3 = "1.6T" percentfull3 = "1%" 
2017-01-27 02:48:00 db_dt="2017-01-27 02:12:00" hostname=myhost3 vol1 = "/dev/sda1" capacity1 = "474G" percentfull1 = "8%" vol2 = "tmpfs" capacity2 = "12G" percentfull2 = "4%"  vol4=/foo capacity4="3G" percentfull4="17%"
Tags (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi burwell,
I created for own customer a dashboard (with related alert) that shows all the servers disks, highlighting free space percentage, sending an alert when a disk has a free space percentage less that 10%.
I displayed each disk in a row, reporting in every row also servers informations (hostname, IP, etc...).
In this way I con see all disks, I can filter them and display the ones of only one server, and using colors, I can immediately see the ones in alert.
Putting every disk in a row, I haven't problems for different disks number in my server.
Bye.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...