Getting Data In

Why am I unable to delete indexes from the Splunk Web? Why do I have to restart Splunk when I create a new Index from Splunk Web?

dshakespeare_sp
Splunk Employee
Splunk Employee

Customer reported several issue with Index Management using the Splunk Web:
- Unable to create new Indexes from Settings > Indexes > New Index (Splunk Web reports that a restart is required)
- Unable to create new Indexes from Data Inputs > Files & Directories > New (Splunk Web reports that a restart is required)
- Unable to delete Indexes from Settings > Indexes > Delete (Splunk Web will not accept any input apart from "Cancel")

1 Solution

dshakespeare_sp
Splunk Employee
Splunk Employee

On investigation TWO issues were found.

  • The "need a restart" type messages seen in the GUI suggested that the indexes.conf on disk had been manually changed while Splunk was running and Splunk had not been restarted so that the running version in memory matched the disk version.

This issue was supported by the following error in splunkd.log:

"unable to reload indexer after adding: reason='reload is not safe since a path has been deleted or modified for an index, or an index has been disabled. You must restart the Splunk Server, for your changes to take effect.'. Restart required." messages seen in splunkd.log.
  • splunkd.log also reported the error

    ERROR IndexConfig - stanza=osstats Required parameter=homePath not configured

Using "splunk btool indexes list --debug to locate the osstats stanza it was observed that in $SPLUNK_HOME/etc/system/local/inputs.conf, the following was declared

[osstats]
disabled = 1

Even though the stanza is disabled Splunk needs a homePATH coldPATH and thawedPATH to tell Splunk where the data files are. Without these stanzas Splunk sees the indexes.conf as invalid and may have problem configuring changes.

The stanza was changed to read

[osstats]
coldPath = $SPLUNK_DB/osstats/colddb
homePath = $SPLUNK_DB/osstats/db
thawedPath = $SPLUNK_DB/osstats/thaweddb
disabled = 1 

On restarting Splunk, the delete index functionality was returned to normal operation

View solution in original post

dshakespeare_sp
Splunk Employee
Splunk Employee

On investigation TWO issues were found.

  • The "need a restart" type messages seen in the GUI suggested that the indexes.conf on disk had been manually changed while Splunk was running and Splunk had not been restarted so that the running version in memory matched the disk version.

This issue was supported by the following error in splunkd.log:

"unable to reload indexer after adding: reason='reload is not safe since a path has been deleted or modified for an index, or an index has been disabled. You must restart the Splunk Server, for your changes to take effect.'. Restart required." messages seen in splunkd.log.
  • splunkd.log also reported the error

    ERROR IndexConfig - stanza=osstats Required parameter=homePath not configured

Using "splunk btool indexes list --debug to locate the osstats stanza it was observed that in $SPLUNK_HOME/etc/system/local/inputs.conf, the following was declared

[osstats]
disabled = 1

Even though the stanza is disabled Splunk needs a homePATH coldPATH and thawedPATH to tell Splunk where the data files are. Without these stanzas Splunk sees the indexes.conf as invalid and may have problem configuring changes.

The stanza was changed to read

[osstats]
coldPath = $SPLUNK_DB/osstats/colddb
homePath = $SPLUNK_DB/osstats/db
thawedPath = $SPLUNK_DB/osstats/thaweddb
disabled = 1 

On restarting Splunk, the delete index functionality was returned to normal operation

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...