Deployment Architecture

The minimum free disk space (5000MB) reached, solve by increasing disk space in Linux?

wuming79
Path Finder

Hi,

Recently I encountered the issue "The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch". My splunk is installed on a vm so I just tried to give more disk space to linux by just increasing the provision space. Somehow, the splunk message is no more. However, I read that I still need to do something to the linux partition table? Do I really need to do the change to partition table in linux to really solved the disk space issue for splunk?? https://www.unixmen.com/increase-disk-space-and-memory-in-linux-vmware-virtual-machines/

0 Karma

wuming79
Path Finder

So increasing the disk size from vsphere resulted in the "minimum free disk space (5000MB) reached" disappearing is a bug?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @wuming79,

why a bug?

it's a protection for your system: if you increase the dimension of your file system, you'll have more than 5 GB free so indexing will restart.

As I said, you could reduce Free Space limit to continue indexing and in he meantime, you have to enlarge file system, then you can set again Free space limit to 5GB.

In this way you will not have the pause in indexing.

In addition, you could do a Capacity Planning to understand how much you have to enlarge the file system.

Ciao.

Giuseppe

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi wuming79,
Splunk stops indexing when the minimum disk space is reached.
You can reduce minimum disk space limit (e.g. to 2 GB) in [Settings -- Server Settings -- General Settings -- Pause indexing if free disk space (in MB) falls below] but it's only a workaround while you solve the problem, at the end of activity set newly the limit to 5GB.

Anyway you have to give more space to your VM by VM-Ware console (or another VM console) and then enlarge your filesystem.
You can find in Internet how to do this, e.g. on Linux you have to

  • stop VM
  • delete old snapshots
  • give more space using VM console
  • snapshot of the new VM
  • restart VM
  • lvextend -LxxG LV
  • resize2fs LV

Bye.
Giuseppe

Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...