Deployment Architecture

Can I, Should I, Change the default rotation from Warm To Cold Buckets

hartfoml
Motivator

The default for rotation from warm to cold is 300. I am retaining about 1 years worth of data in all indexes and most of that data is kept in warm buckets I have about 13.22 TB of "homepath" data and 9.08 TB of "coldpath" data. If I change the default for warm to cold rotation from 300 to 150 I will move about 6.5 TB into cold storage. this will allow me to put the cold buckets on slower SAN space.

My question is,

What will happen when I start up splunk with this new rotation policy?

Will splunk 6.3 chock when trying to move 6.5 TB of data from a fast SAN to a Slower SAN?

I have been asked to do this as a cost saving to the service.

1 Solution

somesoni2
Revered Legend

When you restart your splunk instance, Splunk should start rolling older Warm buckets into Cold bucket, keeping only 150 (latest) Warm buckets in warm bucket directory.
I would not say it would choke but it would take some time and will show high CPU usage based on the amount of data and write speed of your slower SAN for cold bucket storage.

View solution in original post

somesoni2
Revered Legend

When you restart your splunk instance, Splunk should start rolling older Warm buckets into Cold bucket, keeping only 150 (latest) Warm buckets in warm bucket directory.
I would not say it would choke but it would take some time and will show high CPU usage based on the amount of data and write speed of your slower SAN for cold bucket storage.

somesoni2
Revered Legend

Another suggestion would be to do it in 2-3 steps, change it to 250, then 200 and then final 150. Check its performance after changing it to 250, if it doesn't affect searching and indexing , then you can directly reduce it to 150 from 250.

hartfoml
Motivator

Thanks @Somesoni2 you always have a wise and guiding response. I will try the phased approach you suggested.

Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

REGISTER NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If ...

Observability | Use Synthetic Monitoring for Website Metadata Verification

If you are on Splunk Observability Cloud, you may already have Synthetic Monitoringin your observability ...

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...