Forwarder doesn't show in list of forwarders

[bpournader]

The situation:
- Our storage is full and the last time data received is a couple of months ago.
- I'm new here and other guys who implemented Splunk in our network are gone.
- I want to add some windows forwarders to our Splunk environment, I installed the forwarder exe and used "splunk install app /stormforwarder_.spl -auth " to add credentials. Based on documentations, it should work, but no new forwarder is shown in the input ==> forwarders in Splunk Storm.

The Question:
- Does anybody know whether the problem of not showing up new in forwarders is because of full storage or it should be another problem?!

For now, It's very unlikely to free storage to check it myself, I would really appreciate if somebody can help?

[grijhwani]

Very likely. I would fix the major issue - of space - before worrying about fiddling with the behaviour. If your log space has filled up and is a distinct partition for the purpose, you just have the problem of catching up. If, however, the logs are not in a segregated partition you have the problem of a server in a potentially sick state. (Filling up the system partition of any server is a dangerous game.)

[bpournader]

It's a brand new machine.
I added the credentials via "install app" command of Splunk and compared the output and input files of new system with old systems and they are identical.

[ericsix]

If you do a $splunk_home/bin/splunk list forward-server, what is the output? And check the splunkd.log in $splunk_home/var/log/splunk. Very good information in there...

[bpournader]

Thanks, I did the "list forward-server", the funny thing is in a certain machine, sometimes it says its configured but inactive and sometimes it's active, I also ran "netstat" and it seems the connection between forwarder and splunk storm server is established.
My problem now is in the Splunk Storm UI, it doesn't show new clients!

FYI: there is a chance the problem is not having free space.

[aholzer]

Are you upgrading or are you installing a brand new forwarder on a new machine?

Check your outputs.conf on your forwarder [$splunk_home/etc/apps/<appname>/local or $splunk_home/etc/system/local]? You have to point it to your indexer(s) so that it knows where to send data.