Playing with the Windows App, I realized I was sending the wrong type of data to my linux indexer. I was sending perfmon data when I wanted to send WMI data. I've successfully installed a wmi.conf file and am collecting that data (thank you, MarioM). But when I remove the perfmon scripts from my inputs.conf and restart splunk, it just keeps sending the perfdata. The contents of my inputs.conf file, are pretty basic.
[default]
host = DOLLAR
That's it. I've also tried rebooting, no change. What am I missing?
There are many places for an inputs.conf file to reside. In fact, an infinite number.
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Aboutconfigurationfiles
Probably your config was in %SPLUNK_HOME%\etc\apps\windows\local\
or etc\apps\search\local\
.
I thought of this, and have searched through all inputs.conf file in the $SPLUNK_HOME\etc dirs. The splunkperfom entry appears in two places, etc\system\local and etc\system\defaults. It appears in etc\systems\defaults not matter how I install the splunk fowarder, and it is always with disabled = 0, so I don't think that's it. The other entry is where I've manually disabled it.
So, I have a workaround, I suppose, but I'd like to understand how this works so I know what/where to edit for changes in the future.
BTW, if I completely uninstall and reinstall splunk, it stops sending perfdata, and the local/inputs.conf file looks the same, so apparently it gets set somewhere during the install, but not in local.
Also, it appears that placing this in local\inputs.conf
[script://$SPLUNK_HOME\bin\scripts\splunk-perfmon.path]
interval = 10000000
source = PerformanceMonitor
sourcetype = PerformanceMonitor
disabled = 1
queue = winparsing
persistentQueueSize=50MB
Prevents it from sending the data. But this doesn't exist in a client where I didn't check the perf option checkboxes.