Along the lines of this question - http://splunk-base.splunk.com/answers/11577/splunk-overwrites-outputsconf-and-inputsconf-on-reboot, but I've tried a few additional things I'd like to note. To reiterate the issue - I am trying to enable SSL in my outputs.conf for one of my forwarders. I am using the default certs and the default password "password". I've changed the location of my outputs.conf quite a few times, trying these paths: 1) ./etc/system/local/outputs.conf 2) ./etc/apps/forwarder/local/outputs.conf 3) ./etc/apps/forwarder/default/outputs.conf No matter which of these paths I choose, I continue to run into this pattern: 1) Update outputs.conf to have a sslPassword of "password" 2) Use btool to check outputs and note the sslPassword: ./bin/splunk cmd btool outputs list --debug forwarder [tcpout] system autoLB = true forwarder defaultGroup = splunk system forwardedindex.0.whitelist = .* system forwardedindex.1.blacklist = _.* system forwardedindex.2.whitelist = _audit system forwardedindex.filter.disable = false system maxQueueSize = 500KB forwarder [tcpout-server://server.domain:8002] forwarder sslCertPath = $SPLUNK_HOME/etc/auth/server.pem forwarder sslPassword = password forwarder sslRootCAPath = $SPLUNK_HOME/etc/auth/cacert.pem forwarder [tcpout:splunk] forwarder compressed = true forwarder disabled = false forwarder server = server.domain:8002 3) Restart splunkd - ./bin/splunk restart splunkd 4) Use btool to check outputs again and note the sslPassword again: ./bin/splunk cmd btool outputs list --debug forwarder [tcpout] system autoLB = true forwarder defaultGroup = splunk system forwardedindex.0.whitelist = .* system forwardedindex.1.blacklist = _.* system forwardedindex.2.whitelist = _audit system forwardedindex.filter.disable = false system maxQueueSize = 500KB forwarder [tcpout-server://server.domain:8002] forwarder sslCertPath = $SPLUNK_HOME/etc/auth/server.pem forwarder sslPassword = $1$ZMTWcdnuueG6 forwarder sslRootCAPath = $SPLUNK_HOME/etc/auth/cacert.pem forwarder [tcpout:splunk] forwarder compressed = true forwarder disabled = false forwarder server = server.domain:8002 The sslPassword has been hashed, just as it mentions here - http://www.splunk.com/wiki/Community:Splunk2Splunk_SSL_DefaultCerts in the note for #2. Note that the server certificate pass phrase will be hashed and stored in $SPLUNK_HOME/etc/system/local/outputs.conf, overwriting the clear-text value of "sslPassword" if it was defined there. If "sslPassword" was defined in clear-text in an outputs.conf located in an app, it will not be hashed there and will still be present in clear text in that location. This doesn't matter too much in this case since the pass phrase for the default server certificate is well known. The note claims that the password will not be hashed if located in an app. But it is hashed when I used any of the locations above, where I consider paths 2 and 3 to be an app path. I don't have an issue with the hashing, but I feel that it has to do with the SSL error I am getting: ERROR SSLCommon - Can't read key file $SPLUNK_HOME/etc/auth/server.pem errno=101077092 error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt. When I check the password of the server.pem file using openssl: openssl rsa -in /logs/splunk_forwarder/etc/auth/server.pem -text Enter pass phrase for /logs/splunk_forwarder/etc/auth/server.pem: I enter "password" It works. So either, the hashing needs to stop or needs to work. ... View more

I see how to update the colors for a saved search that displays as part of a dashboard, but do not see how to modify the colors of a saved search shown in the display report builder. I'd like to do something similar to the "charting.seriesColors" color property. Thoughts? ... View more

I'd like to understand how external-cmd transforms are being called - for example: Are the results cached, i.e., are calls with the same parameters called multiple times or just the once? Is the CSV stdin sent all at once or in chunks? I have a state lookup defined like this in my transforms.conf: [stateLookup] external_cmd = state.py node eventtype state fields_list = node, eventtype, state My query looks like this: tag=statechange NOT jobstart | sort _time | search node="nodename" | head 8 | lookup stateLookup node eventtype We have server nodes (node) and an eventtype (derived from things like node up and node down) such that a node up would return the node to a user "state". So state is what we want as the output, but only if the state has "changed", this requires a previous state to be tracked. Is there a good way to do this? We were using streamstats which seems to work perfectly, but we thought we'd try tracking previous state in the script as to eliminate a step for our users. Debug statements have led me to believe that the same node with the same eventtype does not get called again (1 - cached), likewise I see the python script being called many times (2 - chunked CSV data, i.e., not sent ALL at once to the script). The reason that (1) is a concern is that we are tracking previous states in a global hash so that we can tell if the state changed or not based on the node and eventtype. So if the call to the script is cached, we will not get accurate results. It's just curious that I am not seeing a "row" in the CSV stdin for each line I have showing in my query. And the reason that (2) is a concern is because when the script is called again, the global hash (dictionary) is lost and therefore the node/eventtype will not be able to find its previous state. Any ideas? Comments? Thoughts? Thank you. ... View more