About Marlan

Marlan · ‎07-06-2021

Thank you codebuilder. Yes, we are trying to replace the node manager and I follow the procedure laid out in the document link you provided. The replacement worked, it was just that the two nodes did not behave exactly the same way. In the mean time, after a number of reboots and without having changed configuration, they are now in agreement on this matter.

Marlan · ‎06-30-2021

Thanks for you reply. Our indexes are fully replicated with a search factor of 2, so even if one indexer is down, data should be searchable after primary buckets have been re-assigned to the running indexer. A searchable rolling restart should do precisely that in advance of restarting the indexers, one at a time. I cannot see why it should not work in our system. https://docs.splunk.com/Documentation/Splunk/8.1.2/Indexer/Userollingrestart Unfortunately, the documentation offers no insight into the check that results in: "Ready for searchable rolling restart NO"

Marlan · ‎06-21-2021

We have a 8.1.2 deployment with 2 clustered indexers and a search head cluster. We have two master nodes, one is operational, the other is on stand-by and they are configured identically. Let's call them master1 and master2. I want to make searchable rolling restart work in our system. When I run splunk apply cluster-bundle on master1 when it is active, it works. While when I make master2 the active one, I get this answer: splunk apply cluster-bundle Warning: Under some circumstances, this command will initiate a rolling restart of all peers. This depends on the contents of the configuration bundle. For details, refer to the documentation. Do you wish to continue? [y/n]: y Encountered some errors while applying the bundle. Request rejected. Searchable rolling restart cannot be guaranteed if num of peers <= 2. We are puzzled why it works on one master node but not on the other when they are identically configured. Everything looks fine in the Monitoring Console whether master1 is active or master2 is. They have the same configuration parameters for searchable_* and for rolling_restart (checked with splunk btool server list --debug) Any suggestion about where to look for the problem?

Marlan · ‎06-21-2021

We have a 8.1.2 dployment with 2 clustered indexers and a search head cluster. We have two master nodes, one is operational, the other is on stand-by. When I run splunk show cluster-status --verbose on the master node I get all positive answers except for this one: "Ready for searchable rolling restart NO". It is not explained in the documentation what is behind this check. Does anyone know what could be the matter? splunk show cluster-status --verbose Pre-flight check successful .................. YES Replication factor met ............... YES Search factor met .................... YES All data is searchable ............... YES All peers are up ..................... YES CM version is compatible ............. YES No fixup tasks in progress ........... YES Splunk version peer count { 8.1.2: 2 } Ready for searchable rolling restart NO Indexing Ready YES idx14 6XYZ1234-9877-4911-A73E-XYZ12345FC43 default Searchable YES Status Up Bucket Count=314 Splunk Version=8.1.2 idx13 DXYZ1234-8765-4DA5-BD4F-XYZ12345B8DF default Searchable YES Status Up Bucket Count=316 Splunk Version=8.1.2 _audit Number of non-site aware buckets=0 Number of buckets=124 Size=41491951 Searchable YES Replicated copies tracker 122/124 122/124 Searchable copies tracker 122/124 122/124 _internal Number of non-site aware buckets=0 Number of buckets=115 Size=3672105501 Searchable YES Replicated copies tracker 115/115 115/115 Searchable copies tracker 115/115 115/115 _telemetry Number of non-site aware buckets=0 Number of buckets=18 Size=67083 Searchable YES Replicated copies tracker 18/18 18/18 Searchable copies tracker 18/18 18/18 main Number of non-site aware buckets=0 Number of buckets=57 Size=6295363 Searchable YES Replicated copies tracker 57/57 57/57 Searchable copies tracker 57/57 57/57

Marlan · ‎02-23-2021

The solution was to modify this line in pamScripted.py Replace in line 60 output = proc.communicate( infoIn['password'] ) with output = proc.communicate( bytes(infoIn['password'],'utf8') )

Marlan · ‎02-22-2021

We have setup a distributed sandbox system with release 8.1.2. We have configured scripted authentication on our search head based on the PAM scripts located in $SPLUNK_HOME/share/splunk/authScriptSamples. We are using the userMapping.py, pamScripted.py and a compiled version of pamauth.c in our setup. We have made some minor modifications to pamScripted.py to make sure that the script only returns users that are in userMapping.py instead of everyone in /etc/passwd, otherwise we have just followed the configuration guidelines in Securing the Splunk Platform manual. When a user tries to login we get errors in splunkd.log like these TypeError: memoryview: a bytes-like object is required, not 'str' The full sequence of error messages are ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/pamscripts/pamScripted.py userLogin': usage: Traceback (most recent call last): ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/pamscripts/pamScripted.py userLogin': File "/opt/splunk/pamscripts/pamScripted.py", line 177, in <module> ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/pamscripts/pamScripted.py userLogin': userLogin( dictIn ) ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/pamscripts/pamScripted.py userLogin': File "/opt/splunk/pamscripts/pamScripted.py", line 60, in userLogin ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/pamscripts/pamScripted.py userLogin': output = proc.communicate( infoIn['password'] ) ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/pamscripts/pamScripted.py userLogin': File "/opt/splunk/lib/python3.7/subprocess.py", line 964, in communicate ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/pamscripts/pamScripted.py userLogin': stdout, stderr = self._communicate(input, endtime, timeout) ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/pamscripts/pamScripted.py userLogin': File "/opt/splunk/lib/python3.7/subprocess.py", line 1695, in _communicate ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/pamscripts/pamScripted.py userLogin': input_view = memoryview(self._input) ERROR ScriptRunner - stderr from '/opt/splunk/bin/python3.7 /opt/splunk/pamscripts/pamScripted.py userLogin': TypeError: memoryview: a bytes-like object is required, not 'str' Seems to me that the pamScripted.py script has not been changed to work under python 3.7. Did anyone manage to make it work in earlier versions of release 8?

Posts	6
Solutions	1
Karma Given	0
Karma Received	0
Member Since	‎02-22-2021

Online Status	Offline
Date Last Visited	‎07-07-2021 03:08 AM

Searchable rolling restart works on master node an...

Why do we get "not Ready for searchable rolling re...

Scripted PAM authentication in 8.1.2

Re: Searchable rolling restart works on master nod...

Re: Why do we get not Ready for searchable rolling...

Searchable rolling restart works on master node an...

Why do we get "not Ready for searchable rolling re...

Re: Scripted PAM authentication in 8.1.2

Scripted PAM authentication in 8.1.2