Just checking we are planning on upgrading a lightweight forwarder to a universal forwarder. the plan is install the universal forwarder create the /opt/splunkforwarder/old_splunk.seed Set a deployment server in /opt/splunkforwarder/etc/system/local/deploymentclient.conf start the universal forwarder. What I am wondering, will the checkpoint file be honored, if I start the server then wait for the deployment server to send apps the universal forwarder. The app contains the indexes and inputs etc for each of the feeds that the server needs. Thanks ... View more

I tried copying over the audit keys from a 4.2 box to a 4.3 box and I am getting Last few lines of stderr (may contain info on assertion failure, but also could be old): 2012-05-31 15:20:17.658 -0400 splunkd started (build 123586) terminate called after throwing an instance of 'RSAException' what(): Encrypt/Decrypt Exception: error:0906D06C:PEM routines:PEM_read_bio:no start line 2012-05-31 15:21:34.142 -0400 splunkd started (build 123586) 2012-05-31 15:26:22.825 -0400 Interrupt signal received 2012-05-31 15:26:59.047 -0400 splunkd started (build 123586) terminate called after throwing an instance of 'RSAException' what(): Encrypt/Decrypt Exception: error:0906D06C:PEM routines:PEM_read_bio:no start line 2012-05-31 15:28:55.387 -0400 splunkd started (build 123586) terminate called after throwing an instance of 'RSAException' what(): Encrypt/Decrypt Exception: error:0906D06C:PEM routines:PEM_read_bio:no start line ... View more

I can't seem to get this working, logs are getting confused about the time format since it is Day Month Year. the log is in /var/log/holly/alarms.20120307-001.log 07/03/12 17:33:54;test2 more props.conf [source::/var/log/holly/*alarms*"] TIME_PREFIX = ^ TIME_FORMAT = %d/%m/%y %H:%M:%S; ... View more

I was running a cold to frozen script that moved the forzen files into a separate directory per index. /opt/splunk/bin/compressedExport.sh ( cd "$1" && gzip *.tsidx ) SPATH=`echo $1 |sed 's/^\/var\/splunk\/lib\/splunk\/\(.*\/\)db.*$/\1/'` mkdir -p /var/splunk/archive/$SPATH cp -r "$1" /var/splunk/archive/$SPATH #replace this with your archive directory So I would end up with the archives in /var/splunk/archive/index1 /var/splunk/archive/index2 etc Is there an easy way to do something similar with the new coldtofrozen script. I tried setting coldToFrozenDir = /var/splunk/archive/ but all the archive files end up in the root of the /var/splunk/archive/ directory. Can I define coldToFrozenDir per index? ... View more

I have a very simple example [serverClass:ash-etl] whitelist.0=server1 [serverClass:ash-etl:app:etl] stateOnClient=enabled restartSplunkd=True [serverClass:ash-etl:app:fwd_to_splunk] stateOnClient=enabled restartSplunkd=True When I change the verison in the app [ui] is_visible = 0 label = etl [launcher] author=Ian description=Etl version=1.5 The client downloads it but I get a ton of errors in the logs 01-09-2012 17:38:27.834 -0500 INFO DeployedApplication - Downloaded url: 10.20.2.28:8089/services/streams/deployment?name=default:ash-etl:etl to file: /opt/splunk/var/run/ash-etl/etl-1326148643.bundle 01-09-2012 17:38:27.835 -0500 WARN DeployedApplication - Installing app: etl to location: /opt/splunk/etc/apps/etl 01-09-2012 17:38:27.839 -0500 ERROR IniFile - Cannot open ini file for parsing: No such file or directory 01-09-2012 17:38:27.839 -0500 ERROR ConfObjectManagerDB - Cannot initialize: /opt/splunk/etc/apps/etl/metadata/local.meta 01-09-2012 17:38:27.858 -0500 ERROR IniFile - Cannot open ini file for parsing: No such file or directory 01-09-2012 17:38:27.858 -0500 ERROR ConfObjectManagerDB - Cannot load to modify: /opt/splunk/etc/apps/etl/metadata/local.meta 01-09-2012 17:38:27.858 -0500 ERROR DeployedApplication - Failed to install app : /opt/splunk/etc/apps/etl. Cannot update application info: /nobody/etl/app/install/state = enabled: Metadata could not be written: /nobody/etl/app/install/state: { }, removable: yes 01-09-2012 17:38:27.858 -0500 WARN DeployedServerClass - There was a problem installing app: etl for server class: ash-etl 01-09-2012 17:38:27.861 -0500 ERROR IniFile - Cannot open ini file for parsing: No such file or directory 01-09-2012 17:38:27.861 -0500 ERROR ConfObjectManagerDB - Cannot initialize: /opt/splunk/etc/apps/etl/metadata/local.meta 01-09-2012 17:38:27.924 -0500 WARN DeployedApplication - Installing app: etl to location: /opt/splunk/etc/apps/etl 01-09-2012 17:38:27.929 -0500 ERROR IniFile - Cannot open ini file for parsing: No such file or directory 01-09-2012 17:38:27.929 -0500 ERROR ConfObjectManagerDB - Cannot initialize: /opt/splunk/etc/apps/etl/metadata/local.meta 01-09-2012 17:38:27.948 -0500 ERROR IniFile - Cannot open ini file for parsing: No such file or directory 01-09-2012 17:38:27.948 -0500 ERROR ConfObjectManagerDB - Cannot load to modify: /opt/splunk/etc/apps/etl/metadata/local.meta 01-09-2012 17:38:27.948 -0500 ERROR DeployedApplication - Failed to install app : /opt/splunk/etc/apps/etl. Cannot update application info: /nobody/etl/app/install/state = enabled: Metadata could not be written: /nobody/etl/app/install/state: { }, removable: yes 01-09-2012 17:38:27.948 -0500 WARN DeployedServerClass - There was a problem installing app: etl for server class: ash-etl 01-09-2012 17:38:27.951 -0500 ERROR IniFile - Cannot open ini file for parsing: No such file or directory 01-09-2012 17:38:27.951 -0500 ERROR ConfObjectManagerDB - Cannot initialize: /opt/splunk/etc/apps/etl/metadata/local.meta I never see splunk reloading even though it has restartSplunkd=True, and the directory has been updated and has the new app version in the app.conf I am running a fairly new version of the splunk 4.2 ... View more

At the back of my mind I remember reading something about reducing the need to do restarts when deploying apps. Can someone point me to the documentation about that? Does the deployment client automatically know to do a restart? For example do you need to do a restart if you add a new index? When do you need to do restartSplunkd=true Most of my apps are simple input apps that have indexes, inputs and props and thats about it. ... View more

My plan was create a search head with a basic app for each department. Users are allowed to save search results in in the department app. What I am seeing is the the saved searched are deleted after updating the app on the deployment servers. I created a very simple app test2 test2: bin default test2/bin: README test2/default: app.conf Then saved a search, changed it to be owned by the app, so I had a test2/local/savedsearch.conf, but after updating the version in default/app.conf and redeploying this file disappears. So unless I am doing something wrong, the deployment server process doesn't honor the app/local or the metadata/local.meta files and deletes the whole app directory, rather than just update the files in the directory ... View more

I would like to have a serverclass for all linux machines that excludes one machine. I have tried the following but it doesn't appear to work, the app fwd_to_splunk still gets deployed to the server. Any suggestions. [global] blacklist.0=* [serverClass:alllinux] machineTypes=linux-* blacklist.0=host1 [serverClass:alllinux:app:fwd_to_splunk] stateOnClient=enabled restartSplunkd=true ... View more

We have hit our limit and I am trying to work out the source of the overage. For today if I run index="_internal" source="*metrics.log" per_index_thruput | timechart span=1d sum(kb) by series The sum total don't make sense to me, It gives 35.30685103 GB and splunk is reporting 17,035 MB indexed, when I look at the license information. How do I get an accurate report that tells me what has been indexed, verse what splunk counts as indexed data. ... View more

Currently links to reports that are emailed out contain just the hostname, not the fully qualified machine name. Our build standard is command host should return just the hostname and host -f returns hostname.fqdn.com. I have set serverName in /opt/splunk/etc/system/local/server.conf to hostname.fqdn.com but the links still come out using hostname ... View more