I am implementing Revealing the Magic
on Splunk v7.2.4 from here:
https://static.rainfocus.com/splunk/splunkconf18/sess/15230307008970013eU6/finalPDF/FN1303_Revealing...
I cannot get the ZST
compression working. First of all, I notice that they wrote index.conf
on page 16 when I assume that they meant indexes.conf
.
I created an app just for this, which I assume the authors did, too. In this app, I have tried:
1: Using journalCompression=zst
all by itself to override the default value, but this did not work. 2: Using a stanza header for each index ( e.g. [_audit]
), each with that same journalCompression=zst
line beneath it, but this did not work either!
For the latter, if I btool
like this:
/opt/splunk/bin/splunk btool indexes list | egrep "journalCompression|zst|^["
Then I get this (which indicates it is OK):
[_audit]
journalCompression = zst
[_internal]
journalCompression = zst
[_introspection]
journalCompression = zst
[_telemetry]
journalCompression = zst
[_thefishbucket]
journalCompression = zst
[car_data]
journalCompression = zst
[cim_modactions]
journalCompression = zst
[default]
journalCompression = zst
[firedalerts]
journalCompression = zst
[history]
journalCompression = zst
[main]
journalCompression = zst
[os]
journalCompression = zst
[power_of_spl]
journalCompression = zst
[provider-family:hadoop]
[splexamples]
journalCompression = zst
[splexamples_downloadcount]
journalCompression = zst
[splexamples_mysummary]
journalCompression = zst
[splunklogger]
journalCompression = zst
[summary]
journalCompression = zst
[volume:_splunk_summaries]
journalCompression = zst
[whois]
journalCompression = zst
But after restart, when I run this:
find /opt/splunk/var/lib/splunk -name "*.zst"
It returns nothing, so the feature is clearly not active.
Not surprisingly, running this returns nothing:
/opt/splunk/bin/splunk btool check
On another 3-node Index cluster, I actually DO get errors trying to apply the bundle:
( /opt/splunk/bin/splunk show cluster-bundle-status
😞
master
cluster_status=None
active_bundle
checksum=6BC53BF8B9FA9F10A38818E85CA2226C
timestamp=1548996573 (in localtime=Thu Jan 31 23:49:33 2019)
latest_bundle
checksum=6BC53BF8B9FA9F10A38818E85CA2226C
timestamp=1548996573 (in localtime=Thu Jan 31 23:49:33 2019)
last_validated_bundle
checksum=143308AF52A5F9606F4C60557CA30794
last_validation_succeeded=0
timestamp=1550442117 (in localtime=Sun Feb 17 17:21:57 2019)
invalid_bundle
checksum=143308AF52A5F9606F4C60557CA30794
timestamp=1550442117 (in localtime=Sun Feb 17 17:21:57 2019)
bundle_path=/opt/splunk/var/run/splunk/cluster/remote-bundle/d48fa52e996bba0be686541559e3ea2b-1550442117.bundle
<bundle_validation_errors on master>
last_check_restart_bundle
last_check_restart_result=restart not required
checksum=
timestamp=0 (in localtime=Wed Dec 31 19:00:00 1969)
<bundle_validation_errors on peer>
[Critical] stanza=_audit parameter=journalCompression Value supplied='zst' is illegal; default='gzip'
[Critical] stanza=_internal parameter=journalCompression Value supplied='zst' is illegal; default='gzip'
...
aze-spl-idx01 A43CE47D-0B1B-4697-A1F2-6B2B1A1977E0 site1
active_bundle=6BC53BF8B9FA9F10A38818E85CA2226C
latest_bundle=6BC53BF8B9FA9F10A38818E85CA2226C
last_validated_bundle=143308AF52A5F9606F4C60557CA30794
last_bundle_validation_status=failure
restart_required_apply_bundle=0
status=Up
aze-spl-idx02 B6DD0A86-368E-4BC3-BF1F-43B9BF0F3504 site1
active_bundle=6BC53BF8B9FA9F10A38818E85CA2226C
latest_bundle=6BC53BF8B9FA9F10A38818E85CA2226C
last_validated_bundle=143308AF52A5F9606F4C60557CA30794
last_bundle_validation_status=failure
restart_required_apply_bundle=0
status=Up
aze-spl-idx03 EEBD9627-49E6-4C7B-B843-FC98BC9D5223 site1
active_bundle=6BC53BF8B9FA9F10A38818E85CA2226C
latest_bundle=6BC53BF8B9FA9F10A38818E85CA2226C
last_validated_bundle=143308AF52A5F9606F4C60557CA30794
last_bundle_validation_status=failure
restart_required_apply_bundle=0
If you have gotten this feature to work, please share what version of splunk and a minimal sample of the working file.
I used the following all by itself in indexes.conf and it worked for me:
journalCompression = zstd
version 7.2.1
I used the following all by itself in indexes.conf and it worked for me:
journalCompression = zstd
version 7.2.1
I cannot believe it but that is it. THANK YOU SO MUCH!!!!!!!
journalCompression = zst is invalid, the correct spelling in the original post is "zstd"
I should have checked the documentation, which is correct.