Getting Data In

indexes.conf - Can multiple indexers save to same "cold" directory?

zindain24
Path Finder

We have two indexers and a single search head:
Splunk1
Splunk2
Splunkweb

Both Splunk1 and Splunk2 have their own independant SAN drives for Hot and Warm storage. For Cold storage, Splunk1 and Splunk2 share a NAS drive with enough space for extended retention.

Can I have Splunk1 and Splunk2 write to the same "Cold" directory for each index? Or should they be different directories for each indexer?

For example - Same Index:

Splunk1 and Splunk2: $SPLUNK_HOME/etc/system/local/indexes.conf

coldPath = /n01/data/spl/internetdb/colddb
coldPath = /n01/data/spl/databasedb/colddb

OR - Different Path:

Splunk1: $SPLUNK_HOME/etc/system/local/indexes.conf

coldPath = /n01/data/spl/internetdb1/colddb
coldPath = /n01/data/spl/databasedb1/colddb

Splunk2: $SPLUNK_HOME/etc/system/local/indexes.conf

coldPath = /n01/data/spl/internetdb2/colddb
coldPath = /n01/data/spl/databasedb2/colddb

Thanks

0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

No they have to use different folders for homePath and coldPath.

This is because :

  • first, each buckets need to have an unique ID per index. otherwise the indexer will detect a duplicate and stop until manually fixed.
  • second, having multiple splunkd process maintaining the retention/rotation rules the same folders will lead to locks and errors.

View solution in original post

yannK
Splunk Employee
Splunk Employee

No they have to use different folders for homePath and coldPath.

This is because :

  • first, each buckets need to have an unique ID per index. otherwise the indexer will detect a duplicate and stop until manually fixed.
  • second, having multiple splunkd process maintaining the retention/rotation rules the same folders will lead to locks and errors.

yannK
Splunk Employee
Splunk Employee

thank you for the addendum mister Sowings.

0 Karma

sowings
Splunk Employee
Splunk Employee

They could use different paths to the same storage partition. The issues Yann describe would arise if both indexers were attempting to use the same target directory as the location for their cold buckets.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...