Splunk Search

What is the quickest and safest way to move indexed data from one location to another?

acontarciego
Explorer

What's the quickest and safest way to move indexed data from one location to another? I have data that is currently stored in the default $SPLUNK_HOME/var/lib/splunk location and I need to move it to a different directory going forward.

If I don't move the existing data, will Splunk be able to search the data that's in the old location in addition to those that will be written in the new directory?

Thanks!

Labels (1)
Tags (1)
1 Solution

Steve_G_
Splunk Employee
Splunk Employee

There's a topic in our doc wiki that provides procedures for moving indexes:

http://docs.splunk.com/Documentation/Splunk/5.0/Indexer/Moveanindex

View solution in original post

Sumit
New Member

You may take a look at below to see the detailed info on changing Splunk DB location and copying the data from existing directory to new.

https://pandeysumitsolutions.blogspot.com/2023/05/changing-splunk-db-location-if-our.html

Blog link - http://sumitpandey.co.in 

0 Karma

chandu245
Explorer

This is what I followed to get this migrated.

Stop the splunk : service splunk stop
Run the below command : rsync -av /opt/splunk/var/lib/ /splunk_data_vol/
Take a backup of /opt/splunk/etc/splunk-launch.conf
Override the new $SPLUNK_DB path /splunk_data_vol/splunk under /opt/splunk/etc/splunk-launch.conf
Start the splunk : service splunk start

splunk_kk
Path Finder

If I'm moving my default SPLUNK_DB location to a new location, can I make a change in the splunk-launch.conf like this:

SPLUNK_HOME=/opt/splunk
SPLUNK_DB=/apps/mysplunkdb

iben
Engager

Here's the error I'm getting

Search not executed: The minimum free disk space (5000MB) reached for /opt/splunk/var/run/splunk/dispatch. user=admin.

Here are the changes I made to move the splunk DB from original location to new place to increase space.

`[root@splunk]# cat etc/splunk-launch.conf

Version 6.0

Modify the following line to suit the location of your Splunk install.

If unset, Splunk will use the parent of the directory this configuration

file was found in

SPLUNK_HOME=/opt/splunk

By default, Splunk stores its indexes under SPLUNK_HOME in the

var/lib/splunk subdirectory. This can be overridden

here:

SPLUNK_DB=/opt/splunk/var/lib/splunk

SPLUNK_DB=/home/splunk

Splunkd daemon name

SPLUNK_SERVER_NAME=splunkd

Splunkweb daemon name

SPLUNK_WEB_NAME=splunkweb
`

BEFORE: My root partition was almost full but the home partition has a lot of free space

[root@splunk]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_centos6-lv_root
17G 15G 949M 94% /
tmpfs 939M 0 939M 0% /dev/shm
/dev/sda1 477M 96M 356M 22% /boot
/dev/sdb1 158G 42G 108G 29% /home

So i used this command to copy the files from old location on root partition to new location on home partition


[root@splunk]# splunk stop
[root@splunk]# cp -R var/lib/splunk/ /home/splunk/
[root@splunk]# rm -fR /opt/splunk/var/lib/splunk/
[root@splunk]# splunk start

AFTER: Verify splunk is working properly and error message about space is gone.

[root@splunk]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_centos6-lv_root
17G 9.5G 6.0G 62% /
tmpfs 939M 0 939M 0% /dev/shm
/dev/sda1 477M 96M 356M 22% /boot
/dev/sdb1 158G 42G 108G 29% /home

bmcnally
New Member

I think $SPLUNK_DB on Linux by default is: /opt/splunk/var/lib/splunk. The referenced link does not make that clear.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

it's pretty simple. Stop Splunk, move the data, change the indexes.conf file to point to the new location. If you're moving not just one index, but the entire $SPLUNK_DB directory, you can instead edit the splunk-launch.conf file and modify the SPLUNK_DB setting. Then start Splunk up again.

Steve_G_
Splunk Employee
Splunk Employee

There's a topic in our doc wiki that provides procedures for moving indexes:

http://docs.splunk.com/Documentation/Splunk/5.0/Indexer/Moveanindex

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...