Deployment Architecture

Change location of introspection index

BrendanCO
Path Finder

Hello. I'd like to change the location of this disk hogging index. I've read through some other posts on this and it refers to an indexes.conf that doesn't reside where they say it does. Here are the ones I have:

find . -name "indexes.conf"

./opt/splunk/etc/master-apps/_cluster/default/indexes.conf
./opt/splunk/etc/system/default/indexes.conf
./opt/splunk/etc/system/local/indexes.conf
./opt/splunk/etc/apps/sample_app/default/indexes.conf
./opt/splunk/etc/apps/SplunkLightForwarder/default/indexes.conf

The one that has the type of information I'm looking for (location to where it write) is ./opt/splunk/etc/apps/sample_app/default/indexes.conf, contents are:

Version 6.5.3

Creates a sample index for sample data.

[sample]
homePath = $SPLUNK_DB/sample/db
coldPath = $SPLUNK_DB/sample/colddb
thawedPath = $SPLUNK_DB/sample/thaweddb

That doesn't seem like the path I'm looking for. Can anyone help point me in the right direction, please? I feel like this should be configurable in the GUI but can't find anything there on that.

Thanks in advance.

Tags (1)
0 Karma

adonio
Ultra Champion

not sure why would you like to change the location of that index but in the case you need to, you can edit the path as you posted in your question. create a new inputs.conf in /opt/splunk/etc/system/local (this is highest precedence in splunk file structure)
in that file, indicate where you would like the introspection index to be:
[_introspection]
homePath = path/to/index/_introspection/db
coldPath = path/to/index/_introspection/colddb
thawedPath = path/to/index/_introspection/thaweddb
more to read here: https://docs.splunk.com/Documentation/Splunk/6.5.3/Admin/Indexesconf
you can find where $SPLUNK_DB is pointing to by navigating to settings -> server settings -> General Settings -> scroll down to "path to indexes field"

0 Karma

BrendanCO
Path Finder

Thanks Adonio! The reasoning behind my wanting to change the location is simply disk space on the primary filesystem is growing day by day. It's now at 72% use and grows by an entire percent per day. What's odd is that I put an ln -s for the dispatch folder to go to the new filesystem and my utilization hasn't changed in days on the target filesystem. Only on the primary.

Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 7.8G 5.5G 2.2G 72% /
devtmpfs 3.9G 68K 3.9G 1% /dev
tmpfs 3.9G 0 3.9G 0% /dev/shm
/dev/xvdb1 40G 12G 27G 30% /splunkdata

So the /dev/xvda1 filesystem is where /opt/splunk resides. The /dev/xvdb1 filesystem is where things are supposed to go but has remained at 30% use for this entire week. So, something isn't working right! This is what prompted me to want to move indexes, unless you advise against that!
I need to get my hands around this before I run out of space on /dev/xvda1. Once I get that set up correctly, then I can start looking at how to manage log retention and not be such a bother on this board... 🙂

0 Karma

adonio
Ultra Champion

look at the last part of my answer and at your indexes configurations, if they contain $SPLUNK_DB the index location reffers to twhere $SPLUNK__DB points. you can change $SPLUNK_DB
if you would like to move all indexes to new file system, follow this link: https://docs.splunk.com/Documentation/Splunk/6.5.3/Indexer/Moveanindex

BrendanCO
Path Finder

Using your instructions, I was able to move all my indexes to my new filesystem. It is reflecting the new location in the GUI as well. Thanks again, Adonio. As always, a great help.

0 Karma

adonio
Ultra Champion

is it a clustered environment or single indexer?

0 Karma

BrendanCO
Path Finder

Single indexer

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...