Getting Data In

Export/Reindex 500GB

ckillg
Path Finder

I'm trying to figure out the best way to send most of a 500GB index back through the Heavy Forwarder -> Indexer Cluster chain to correct hostname and sourcetype issues.

Any suggestions?

0 Karma
1 Solution

gfuente
Motivator

Hello

If you have a reasonable number of index/sourcetype/host combinations you can run an indivual export by combination through the api:

Example curl command from the HF (to export the file locally to the HF):

sudo curl -k -u admin:password https://splunksh:8089/services/search/jobs/export --data-urlencode search='search index=myindex sourcetype=mysourcetype host=myhost earliest=0 latest=now' -d output_mode=raw -o myindex_myhost_mysourcetype.txt

Then, create monitor stanzas in the HF with the appropiate index, sourcetype and host values for each file. You could also create transforms to get those meta data from the file name, but that will be more complex, for doing it just once

Hope this helps

Regards

View solution in original post

gfuente
Motivator

Hello

If you have a reasonable number of index/sourcetype/host combinations you can run an indivual export by combination through the api:

Example curl command from the HF (to export the file locally to the HF):

sudo curl -k -u admin:password https://splunksh:8089/services/search/jobs/export --data-urlencode search='search index=myindex sourcetype=mysourcetype host=myhost earliest=0 latest=now' -d output_mode=raw -o myindex_myhost_mysourcetype.txt

Then, create monitor stanzas in the HF with the appropiate index, sourcetype and host values for each file. You could also create transforms to get those meta data from the file name, but that will be more complex, for doing it just once

Hope this helps

Regards

gfuente
Motivator

Forgot to say, that you need to ensure that the HF has the appropiate props configurations for the sourctypes you want to reindex.

0 Karma

ckillg
Path Finder

This is perfect. Thank you!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...