Splunk Search

Is it possible to divide an index to two indexes?

appleman
Contributor

Hello there,

I just wonder if I can divide an index into two indexes.
e.g, Divide the data in index=main to index=production and index=text

Thank you.


既にIndex化されたデータ(Index1)を、途中で二つのindex(Index1とIndex2)に分けることは可能でしょうか。

例:index=mainに入っているデータを、途中でindex=production と index=testに分ける

Tags (2)
1 Solution

jbsplunk
Splunk Employee
Splunk Employee

If you wanted to split the main index in half, here is what you'd do with an out of the box install:

  1. Create two new indexes called 'text and 'production'.
  2. Go into the /opt/splunk/var/lib/splunk/defaultdb/db and /opt/splunk/var/lib/splunk/defaultdb/colddb. This is where the actual data sits inside of buckets using the format described here.
  3. Move the data, using whatever logic you'd like, to the same location in the destination index so that half resides in one index and half the other. Ensure not to introduce any bucket id conflicts, otherwise the destination indexes will be disabled, as per this splunk answer.
  4. Restart the Indexer and you should now be able to search the data in the new indexes.

View solution in original post

jbsplunk
Splunk Employee
Splunk Employee

If you wanted to split the main index in half, here is what you'd do with an out of the box install:

  1. Create two new indexes called 'text and 'production'.
  2. Go into the /opt/splunk/var/lib/splunk/defaultdb/db and /opt/splunk/var/lib/splunk/defaultdb/colddb. This is where the actual data sits inside of buckets using the format described here.
  3. Move the data, using whatever logic you'd like, to the same location in the destination index so that half resides in one index and half the other. Ensure not to introduce any bucket id conflicts, otherwise the destination indexes will be disabled, as per this splunk answer.
  4. Restart the Indexer and you should now be able to search the data in the new indexes.

jbsplunk
Splunk Employee
Splunk Employee

Well, my suggestion was to move, not copy, the data. If you move it, you're not making duplicates because each index would only have a single copy of it, with half the buckets in test and half in production. Of course, you would need to adjust your inputs and/or props/transforms to ensure the data inputs you'd like are routing to production.

0 Karma

appleman
Contributor

I understand that it is not possible to re-index the data which has been already indexed. I guess I should delete the data and create new indexes to divide them.

0 Karma

lukejadamec
Super Champion

This solution creates a duplicate of the index in each of the new indexes, it does not separate them. Data that has been indexed cannot be altered.

You will have problems going forward, because all new data will be placed into Main unless you redirect the data to the desired index.

0 Karma

appleman
Contributor

Thank you.
So, it's possible to divide the index which has already indexed once, right? I'm bit confused since linu1988 below says it's not possible.....

0 Karma

linu1988
Champion

Hello,
You could create two separate indexes. Use COLLECT command to pass that info into your required indices. But it will not be possible to divide the index further once it is created and data is in them. Before passing the events to the new indexes please read the documentation.

_http://docs.splunk.com/Documentation/Splunk/6.0.1/searchreference/collect
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...