Getting Data In

Can anyone provide guidance on my plan to configure cold storage in indexes.conf?

ccsfdave
Builder

So, I got the 150TB cold, but they are mounted into /mnt/splunk1/cold and /mnt/splunk2/cold. I figured that may cause issues with the indexers, so I made symlinks to /opt/splunk/var/lib/splunk/cold on each of the indexers to prevent issues with which indexer Splunk wants to write to.

I am now thinking about changing the indexes.conf and adding to the volume stanza:

# One Volume for Cold
[volume:cold]
path = /opt/splunk/var/lib/splunk/cold
# 150000GB (150TB)
maxVolumeDataSizeMB = 150000000

Then changing the cold locations from:
coldPath = volume:primary/defaultdb/colddb
to
coldPath = volume:cold/defaultdb/colddb

The ES definitions are:
coldPath = $SPLUNK_DB/audit_summarydb/colddb

I would like to change that too, similar to above:
coldPath = volume:cold/audit_summarydb/colddb

Thoughts? Guidance?

0 Karma
1 Solution

ccsfdave
Builder
  1. I changed the coldPath on all my indexes to volume:cold
  2. I created a /opt/splunk/etc/system/local/indexes.conf on my SH and Indexers

maxWarmDBCount = 50
maxHotSpanSecs = 2592000

Anything else I should or shouldn't have done?

View solution in original post

0 Karma

ccsfdave
Builder
  1. I changed the coldPath on all my indexes to volume:cold
  2. I created a /opt/splunk/etc/system/local/indexes.conf on my SH and Indexers

maxWarmDBCount = 50
maxHotSpanSecs = 2592000

Anything else I should or shouldn't have done?

0 Karma

ccsfdave
Builder

Well, the above worked for me. In our case we have 675GB SSD RAID 1 each on two indexers and they were full with the default settings. I finally got the 150TB of spinning drives mounted in as cold but nothing was rolling over to it. So I did a search of my data to see how far it went back. Not sure this was scientific in anyway but I decided to 1/3 the default settings above with the end results being .

The end result was I brought my hot drives to 60% and 72% utilization. So we will go forward with this config until I get more hot drives.

Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...