Getting Data In

How to activate forward-server?

jangid
Builder

I am struggling to send data from remote machine to Splunk server due to lack of quality documentation.

can anyone tell me how to activate forward server?

C:\Program Files\SplunkUniversalForwarder\bin>splunk.exe list forward-server
Active forwards:
None
Configured but inactive forwards:
169.254.131.1:9997

Thanks

1 Solution

lguinn2
Legend

Here is my quick checklist:

  • Have you enabled receiving on the indexer(s)? [at least, on the indexer running on 169.254.131.1]
  • Is the port open that you are using to receive? [9997 in your case]
  • Can you ping from the forwarder to the indexer? This is a very basic connectivity test, as ping does not use TCP, or the port you have selected
  • Is there a firewall between the forwarder and the indexer that could be blocking the port?
  • Is outputs.conf configured correctly on the forwarder?
  • Is inputs.conf configured correctly?
  • Have you restarted the forwarder? (You should try that)

There is no command to "activate" a forwarder. What the Universal Forwarder is telling you is "I am configured to send data to the indexer at 169.254.131.1:9997, but I am unable to do so".

This is usually a problem with a firewall or a mis-configured outputs.conf

Hope this helps!

Finally, you could read the following:

http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Aboutforwardingandreceivingdata

http://docs.splunk.com/Documentation/Splunk/6.0/Data/Usingforwardingagents

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf

And search for "splunk inactive forwarder" in your favorite search engine, or on answers.splunk.com

View solution in original post

shtychkn
Engager

I am only posting this because none of the above answers helped me figure out that my config was actually fine to begin with.

When you use the splunk list forward-server is only shows you forward servers that are active in the instant you ran the command, so unless you are actively pushing data, none of your forward servers are going to show up as being "active". Once you are pushing data, you can also incorporate the watch command to auto refresh the command and catch an active forward: watch -n 5 splunk list forward-server,Only posting this because none of the above options helped me discover that my config was fine:

A forward server will only show up as "active" using splunk list forward-server command if it is actively forwarding data. Run the command a few times and chances are you will see it move to the active state. You can also incorporate the watch command to auto refresh the command: watch -n 5 splunk list forward-server

jackal713
Path Finder

Trying to get splunk setup and actually working has been a very frustrating experience. I have not found any guides that gives you step by step in the same document. I start reading one, it tells me, go read this other thing. I go read that, and it tells me go read something else.

No where in any of the things I read did it say anything about opening firewall ports on the indexer. But that was the problem. Opened port 9997 TCP and now it is active. But when I go to the "add data" page and click forwarders it still shows none. It also says, "There are currently no forwarders configured as deployment clients to this instance." GRRRRR! Then what have I been setting up for the last 4 hours?

If it is required then why didn't the installer do it? Or at least ask, if I want it to do it for me. Or even give a "Hey don't forget..." message. Based on my experience I will be recommending that we look at a different solution that is easier to implement. I DO NOT want to repeat this experience at our other 2 sites.

gonghaokaka
New Member

*For me issue was related to the Windows Firewall at the indexer (Splunk enterprise) server blocked the forwarder server's connection *

The Error found in the forwarder server splunkd.log

ERROR TcpInputProc - Error encountered for connection from ... timeout

To fix it, simply allow the splunkd.exe (located at splunk_home\bin) to communicate through window firewall.

Tim

0 Karma

jhl226116
Explorer

I already checked and disabled firewall in Ubuntu but still unable to activate forwards. How come there's no proper solution detailed out anywhere? I'm stuck with the same issue.

#./splunk list forward-server
Active forwards: 
None
Configured but inactive forwards: 
10.10.50.49:9997
0 Karma

jyuan_splunk
Splunk Employee
Splunk Employee

Try telnet 169.254.131.1 9997 to check connection, if failed, check firewall. I solved this problem in this way.

Strunk
Explorer

For me, I had updated my outputs.conf with the correct hostname and port number but had not reloaded the universal forwarder to implement the changes. ./splunk restart worked for me.

lguinn2
Legend

Here is my quick checklist:

  • Have you enabled receiving on the indexer(s)? [at least, on the indexer running on 169.254.131.1]
  • Is the port open that you are using to receive? [9997 in your case]
  • Can you ping from the forwarder to the indexer? This is a very basic connectivity test, as ping does not use TCP, or the port you have selected
  • Is there a firewall between the forwarder and the indexer that could be blocking the port?
  • Is outputs.conf configured correctly on the forwarder?
  • Is inputs.conf configured correctly?
  • Have you restarted the forwarder? (You should try that)

There is no command to "activate" a forwarder. What the Universal Forwarder is telling you is "I am configured to send data to the indexer at 169.254.131.1:9997, but I am unable to do so".

This is usually a problem with a firewall or a mis-configured outputs.conf

Hope this helps!

Finally, you could read the following:

http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Aboutforwardingandreceivingdata

http://docs.splunk.com/Documentation/Splunk/6.0/Data/Usingforwardingagents

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf

And search for "splunk inactive forwarder" in your favorite search engine, or on answers.splunk.com

s50600822
Engager

Do you know if there's a log somewhere that say what the forwarder is stuck with? Or the only way is manually check each thing?

0 Karma

lguinn2
Legend

Here is a great article on the Splunk wiki: Troubleshooting Monitor Inputs

http://wiki.splunk.com/Community:Troubleshooting_Monitor_Inputs

[I would skip the first part of the page on setting DEBUG mode; the other suggestions on the page are generally both easier and more useful.]

And as a first step, I would log onto the forwarder and give the following command

splunk list monitor

which will tell you which files Splunk is reading. A quick peek at splunkd.log may be helpful, too; you can even search it on the indexer with the following command:

index=_internal source=*splunkd.log

0 Karma

lguinn2
Legend

And one more thing - check the actual files/scripts/ports that are monitored by your inputs.conf.

Do the files/directories exist? Is there data in the files/directories?

Do the scripts run properly and generate results on stdout?

Is any data arriving on the ports you are monitoring?

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...