Deployment Architecture

Recommended filesystem for hot/warm buckets

stwong
Communicator

Hi,

We're setting up Splunk 6.4 on RedHat Enterprise Linux 7.
We intend to use ext4 but there is no native snapshot support (we learnt that for hot buckets, backup on snapshot is required). Then we're testing btrfs. However, we got system crash when doing some bonnie++ test on btrfs filesystem on SSD.

Would like to know:
- any recommended filesystem for hot/warm bucket
- can we take the random seek result of bonnie++ as IOPS value (read some post about this topic and would like to know if there is any update on this topic).

Thanks a lot.
Rgds

0 Karma

jrodman
Splunk Employee
Splunk Employee

For what it's worth, a lot of people I trust are now quite bullish on ext4 with lvm2 snapshots as a completely legit solution. I don't have direct experience.

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

Here is a list of supported file systems for the latest Splunk Version:

http://docs.splunk.com/Documentation/Splunk/6.4.2/Installation/Systemrequirements#Supported_file_sys...

If you're looking at snapshot capability, ZFS might be the better choice. It's pretty robust and I've encountered deployments on ZFS doing snapshots to fulfill back requirements. (Although backing up of buckets in a clustered environment is questionable.....)

0 Karma

jkat54
SplunkTrust
SplunkTrust

Yes, random seek in bonnie++ is the iops. You might find this varies greatly between bonnie++ versions however, and thus I recomend fio for these tests instead. I know Splunk prefers bonnie++ however, so if you're getting reliable results, please continue to use bonnie++ at your leisure.

As for filesystems, you might get next to compile on RHEL:
http://next3.sourceforge.net/

0 Karma

stwong
Communicator

Will give next a try. Thanks.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...