Deployment Architecture

The minimum free disk space (5000MB) reached, solve by increasing disk space in Linux?

wuming79
Path Finder

Hi,

Recently I encountered the issue "The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch". My splunk is installed on a vm so I just tried to give more disk space to linux by just increasing the provision space. Somehow, the splunk message is no more. However, I read that I still need to do something to the linux partition table? Do I really need to do the change to partition table in linux to really solved the disk space issue for splunk?? https://www.unixmen.com/increase-disk-space-and-memory-in-linux-vmware-virtual-machines/

0 Karma

wuming79
Path Finder

So increasing the disk size from vsphere resulted in the "minimum free disk space (5000MB) reached" disappearing is a bug?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @wuming79,

why a bug?

it's a protection for your system: if you increase the dimension of your file system, you'll have more than 5 GB free so indexing will restart.

As I said, you could reduce Free Space limit to continue indexing and in he meantime, you have to enlarge file system, then you can set again Free space limit to 5GB.

In this way you will not have the pause in indexing.

In addition, you could do a Capacity Planning to understand how much you have to enlarge the file system.

Ciao.

Giuseppe

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi wuming79,
Splunk stops indexing when the minimum disk space is reached.
You can reduce minimum disk space limit (e.g. to 2 GB) in [Settings -- Server Settings -- General Settings -- Pause indexing if free disk space (in MB) falls below] but it's only a workaround while you solve the problem, at the end of activity set newly the limit to 5GB.

Anyway you have to give more space to your VM by VM-Ware console (or another VM console) and then enlarge your filesystem.
You can find in Internet how to do this, e.g. on Linux you have to

  • stop VM
  • delete old snapshots
  • give more space using VM console
  • snapshot of the new VM
  • restart VM
  • lvextend -LxxG LV
  • resize2fs LV

Bye.
Giuseppe

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...