Getting Data In

How to recreate partial index data with metadata on different Splunk installation?

deepdive100
Loves-to-Learn Everything

I have a Splunk container for development (Dev).  I want to import a slice of data from one index of my production Splunk (Prod) to this container so I can write searches against that data exactly as it appears in Prod. 

Using Export on Prod and Import on Dev is not producing my desired outcome.  Doing this as a single file with a single indexing is creating logs that are indexing the container hostname as the host not the host of the data itself.  The data in the Prod index is of varying sourcetypes so the import is also only creating the sourcetype of the import file, not tha sourcetype from the data itself. 

I'm looking at possibly using the  EventGen app but not sure if this will do what I'm trying to do.

Is what I'm doing possible?  I do not want the entire prod index. I do not want to rsync or otherwise go to the backend to move data.  

EDIT: I modified the title, it seems I want the raw data and metadata to all come over in one package?

Labels (2)
Tags (2)
0 Karma

deepdive100
Loves-to-Learn Everything

So it seems the way forward for me is to write some scripts to pull down `index=app host=each_host sourcetype=each_sourcetype` for a specific time block, export them with the hostname in the title and import each with the hostname widget set to the filename.  One script of API calls with the variables on the hosts and sourcetype should do it.  Will try it out and update here

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

basically you could try to copy those from prod node. Here is an old post about it https://community.splunk.com/t5/Installation/How-to-migrate-indexes-to-new-indexer-instance/m-p/5280...

You should change needed configurations after copy as you want this to be a different host  also you should copy only needed indexes or remove those after rsync.

r. Ismo

 

0 Karma
Get Updates on the Splunk Community!

Modern way of developing distributed application using OTel

Recently, I had the opportunity to work on a complex microservice using Spring boot and Quarkus to develop a ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had 3 releases of new security content via the Enterprise Security ...

Archived Metrics Now Available for APAC and EMEA realms

We’re excited to announce the launch of Archived Metrics in Splunk Infrastructure Monitoring for our customers ...