Hello,
I just downloaded splunk today to try it out on a few of our servers, but found out very quickly that it doesn't support btrfs:
Filesystem type is not supported: buf.f_type = 0x9123683e
The output of locktest
looks like this:
~/splunk]% bin/locktest
Could not create a lock in the SPLUNK_DB directory.
Filesystem type is not supported: buf.f_type = 0x9123683e
If supporting this filesystem type is important to you, please file an Enhancement Request with Splunk Support with the fs info number listed.
~/splunk]% ls $SPLUNK_DB
audit/ authDb/ blockSignature/ defaultdb/ fishbucket/ hashDb/ historydb/ _internaldb/ sample/ summarydb/ test.ijKHJ9 test.R0jT0h test.T65SU0
Output of strace
:
~/splunk]% strace bin/locktest
execve("bin/locktest", ["bin/locktest"], [/* 32 vars */]) = 0
brk(0) = 0x245b000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8ad5120000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=27312, ...}) = 0
mmap(NULL, 27312, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f8ad5119000
close(3) = 0
open("/lib64/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\355\1\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1832712, ...}) = 0
mmap(NULL, 3664040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f8ad4b84000
mprotect(0x7f8ad4cf9000, 2097152, PROT_NONE) = 0
mmap(0x7f8ad4ef9000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x175000) = 0x7f8ad4ef9000
mmap(0x7f8ad4efe000, 18600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f8ad4efe000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8ad5118000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8ad5117000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8ad5116000
arch_prctl(ARCH_SET_FS, 0x7f8ad5117700) = 0
mprotect(0x7f8ad4ef9000, 16384, PROT_READ) = 0
mprotect(0x7f8ad5121000, 4096, PROT_READ) = 0
munmap(0x7f8ad5119000, 27312) = 0
stat("/home/xx/splunk/var/lib/splunk", {st_mode=S_IFDIR|0711, st_size=258, ...}) = 0
umask(0777) = 022
umask(066) = 0777
gettimeofday({1290061806, 661323}, NULL) = 0
getpid() = 15124
open("/home/xx/splunk/var/lib/splunk/test.dHc3Nt", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
statfs("/home/xx/splunk/var/lib/splunk/test.dHc3Nt", {f_type=0x9123683e, f_bsize=4096, f_blocks=2228224, f_bfree=1430113, f_bavail=1128033, f_files=0, f_ffree=0, f_fsid={196237592, 245698777}, f_namelen=255, f_frsize=4096}) = 0
statfs("/home/xx/splunk/var/lib/splunk/test.dHc3Nt", {f_type=0x9123683e, f_bsize=4096, f_blocks=2228224, f_bfree=1430113, f_bavail=1128033, f_files=0, f_ffree=0, f_fsid={196237592, 245698777}, f_namelen=255, f_frsize=4096}) = 0
statfs("/home/xx/splunk/var/lib/splunk", {f_type=0x9123683e, f_bsize=4096, f_blocks=2228224, f_bfree=1430113, f_bavail=1128033, f_files=0, f_ffree=0, f_fsid={196237592, 245698777}, f_namelen=255, f_frsize=4096}) = 0
write(2, "Could not create a lock in the S"..., 52Could not create a lock in the SPLUNK_DB directory.) = 52
statfs("/home/xx/splunk/var/lib/splunk/test.dHc3Nt", {f_type=0x9123683e, f_bsize=4096, f_blocks=2228224, f_bfree=1430113, f_bavail=1128033, f_files=0, f_ffree=0, f_fsid={196237592, 245698777}, f_namelen=255, f_frsize=4096}) = 0
statfs("/home/xx/splunk/var/lib/splunk/test.dHc3Nt", {f_type=0x9123683e, f_bsize=4096, f_blocks=2228224, f_bfree=1430113, f_bavail=1128033, f_files=0, f_ffree=0, f_fsid={196237592, 245698777}, f_namelen=255, f_frsize=4096}) = 0
write(2, "Filesystem type is not supported"..., 201Filesystem type is not supported: buf.f_type = 0x9123683e
If supporting this filesystem type is important to you, please file an Enhancement Request with Splunk Support with the fs info number listed.) = 201
exit_group(9)
Given the change of state, I unaccepted my prior answer (that was years old) and marked ogdin's as correct.
Splunk v6.2 released. Still aborts with unhelpful message when running on top of btrfs.
Not sure why a workaround solution to this was removed from this thread but I am re-adding it again.
As of Splunk 6.1.3 (build 220630) there is still no support for BTRFS.
A workaround which I recommend for non production environments is to create a blockfile formatted to ext4 and mount this at /opt/splunk/var.
A 10G file can be created with the following on Fedora 20 (likely the same for most Linux flavours)
dd if=/dev/zero of=/opt/splunk-var-blockfile bs=1M count=10240
mkfs.ext4 /opt/splunk-var-blockfile
mount /opt/splunk-var-blockfile /opt/splunk/var
In order for the file to be mounted on reboot edit /etc/fstab and add the following line
/opt/splunk-var-blockfile /opt/splunk/var/ auto loop 0 0
xfs is the default filesystem in RHEL 7, not btrfs so splunk will work fine on it
thanks for the answer, that's a good news.
sorry my mistake, I was effectively looking at performance and support for xfs
any news about btrfs support with splunk 6.1 or why it's not supported at this time ?
Btrfs is the default file system for the just released "RedHat Enterprise 7" and io oriented benchmark tests seems to indicate it could be a good choice for filesystem with lots of data such as splunk.
I just encountered this issue too as I was about to go on a work trip. I installed Splunk on my Fedora 18 laptop and lo and behold it doesnt work.
Still no solution for this after 3 years?
I don't know precisely why Splunk threw an unsupported message about your btrfs - but it is usually very picky about the underlying filesystem's locking semantics. What does the Splunk 'locktest' command say about your btrfs?
Thanks for this it appears to work and is acceptable for a development environment but I would have hoped that Splunk could support btrfs at least in version 6.
It is now the default filesystem for many Linux distributions.
And no btrfs support in 6.0?
Can someone please give us an update of when it might be available?
July 2013 and splunk 5.0.3 - still no btrfs support.
On fedora 18 I've hit this problem as well, my workaround (that doesn't work see additional comment below) might be helpful to some people.
I just replace the locktest (/opt/splunk/bin/locktest) with a bash script which does an 'exit 0'.
Edit: Ok, this worked for a little while and then resulted in the indexer blocking and lots of weird things happening.
Perfect, thanks! I'm sure running ext3 on top of btrfs isn't terribly quick, but on my laptop it'll fly. Where'd I put that imaginary SSD now???
Jan 2013, need brtfs support
December 2012 and Splunk 5.0.1 - still no btrfs support.
it's now April 2012 and Splunk 4.3.1 - still no official btrfs support ;(
Hi Henry/Archie(?),
first create a file as big as you think you will need for splunk data
dd if=/dev/zero of=/mydata/splunkfs.img bs=1M count=5000
Then format it with ext4 (sans journal, which is kind of moot in this case):
mkfs.ext4 -O ^has_journal /mydata/splunkfs.img
Add a line to /etc/fstab like this:
/mydata/splunkfs.img /opt/splunk/var/ auto loop 0 0
Stop splunk:
/opt/splunk/bin/splunk stop
Then just something like
cd /opt/splunk
mkdir vartmp
mv var/* vartmp
mount var
mv vartmp/* var
/opt/splunk/bin/splunk start
"As a workaround I've loopback-mounted a file formatted with ext4 to /opt/splunk/var and copied the contents of the original directory there. Seems to work just fine so far."
Please provide a "how to" for this workaround as I am new to linux and dont completely follow how this is achieved. Any help appreciated. I created a loopback file system but I am confused about recreating the var directory as an ext4 filesystem since this did not work when following this procedure..http://www.walkernews.net/2007/07/01/create-linux-loopback-file-system-on-disk-file/#comment-15959
Regards, Henry
Since btrfs is planned to become the default in Fedora 17 and others will surely follow, I think splunk should start to support it as soon as possible.
As a workaround I've loopback-mounted a file formatted with ext4 to /opt/splunk/var and copied the contents of the original directory there. Seems to work just fine so far.