Splunk Search

Is it possible to mount HOT to a ram disk for performance?

daniel333
Builder

All,

Just day dreaming here a little as I read the indexes.conf file documentation a bit. I was thinking, assuming you're willing to risk the data loss wouldn't it make sense to house your Hot buckets to a RAM disk, then roll from HOT to warm to actual disk?

Crazy?

https://www.jamescoyle.net/how-to/943-create-a-ram-disk-in-linux

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

HOT and WARM are on the same filesystem; you cannot separate that. The only real difference between HOT and WARM buckets is that HOT buckets are open for R/W whereas WARM buckets are read-only.
Splunk really doesn't care how you implement the HOT/WARM storage as long as it is exposed as a supported filesystem.
You will very definitely want to very carefully assess your failure scenarios with RAM disk.

jkat54
SplunkTrust
SplunkTrust

I concur with the above.

Also consider the limitation of maybe at most 512gb available...

Would be great for a very specific use case though!

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

An unscheduled reboot will loose all the data from a RAMdisk. and since some hot buckets stay around a long time, you would loose all that data. You'd also loose the warm buckets, too, since they have to reside on the same FS.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

True.

You will very definitely want to very carefully assess your failure scenarios with RAM disk.

In other words: Don't do this without Splunk index replication. Spread your indexers across multiple racks/switches/PDUs/etc.
Even after doing anything you can think of, accept the remaining risk of losing data due to catastrophic failures.

gjanders
SplunkTrust
SplunkTrust

Also consider what will happen when a server requires a reboot for maintenance purposes.

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

REGISTER NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If ...

Observability | Use Synthetic Monitoring for Website Metadata Verification

If you are on Splunk Observability Cloud, you may already have Synthetic Monitoringin your observability ...

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...