Getting Data In

How to determine disk utilization of a source?

wasupchris
New Member

Hi,
Is there a way to determine how much disk space a source is using?

here is my index, source and sourcetype:

index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file 

Best Regards,
Chris

0 Karma
1 Solution

bmacias84
Champion

The is not real good way of doing this but you can make some assumptions. You can assume that every character in _raw is one byte (some unicode characters are four bytes). You then pick a time period you want to sample.

 index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file earliest=-8h@h | eval length=len(_raw) | stats avg(length) as bytesPerMessage

I don't know the size of your message but lets say every message on average 481 bytes. Now that you've done that you can use tstats find the count of those messages per day. Then we can use some eval statements with Math.

|tstats count WHERE index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file by _time span=1d | eval bytesPerDay=(481*count) | eval MB=(bytesPerDay/1024/1024)

This will get you close be is only an approximation.

View solution in original post

0 Karma

bmacias84
Champion

The is not real good way of doing this but you can make some assumptions. You can assume that every character in _raw is one byte (some unicode characters are four bytes). You then pick a time period you want to sample.

 index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file earliest=-8h@h | eval length=len(_raw) | stats avg(length) as bytesPerMessage

I don't know the size of your message but lets say every message on average 481 bytes. Now that you've done that you can use tstats find the count of those messages per day. Then we can use some eval statements with Math.

|tstats count WHERE index=network source="/opt/syslog-ng/log/hsm*.log" sourcetype=hsm:file by _time span=1d | eval bytesPerDay=(481*count) | eval MB=(bytesPerDay/1024/1024)

This will get you close be is only an approximation.

0 Karma

wasupchris
New Member

Thank you very much, bmacias84

0 Karma

aaraneta_splunk
Splunk Employee
Splunk Employee

@wasupchris - Did the answer provided by bmacias84 help provide a working solution to your question? If yes, please don't forget to resolve this post by clicking "Accept". If no, please leave a comment with more feedback. Thanks!

0 Karma

wasupchris
New Member

Anyone knows how to identify the average disk space utilization by data source.

0 Karma

aaraneta_splunk
Splunk Employee
Splunk Employee

@wasupchris - If you want to try to get some immediate help for your question, you should join the 1300+ Splunk users in our public Slack chat. People ask each other for immediate help on there daily. You can share your question/link to your post there to see if anyone can take a stab at it.

You first have to request access through www.splunk402.com/chat. Fill out the form, and once you receive the approval email from our Community Manager (usually the approval process takes a couple days), you can access Slack.com and ask for help in the #general channel or the #n00bs channel (if you're brand new to Splunk).

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...