Push logs from rsyslog into splunk

[bhavya_shah]

I was able to setup rsyslog to push logs into splunk but issue is only /var/log/messages are pushed to splunk but i have many more logs such as /logs/server-logs/servername/* on rsyslog server that I want to push it to splunk. Is there a way to push it?

[bhavya_shah · Accepted solution]

http://splunk-base.splunk.com/answers/95281/having-issues-with-universal-forwarder

http://splunk-base.splunk.com/answers/95503/how-to-setup-universal-forwarder-on-linux

[grijhwani]

In its simplest form you just need something like the following stanza in the inputs.conf on the rsyslog server. (I assume, from the mention of other logs already being pushed, you have installed a light forwarder instance at the very least.)

[monitor:///logs/server-logs/]
host_segment = 3
sourcetype = syslog
index = syslog
disabled = false

As splunk try the following:

bin/splunk btool inputs list --debug

This should show you your complete inputs configuration.

To interrogate what is or is not being consumed, point this at your indexer:

https://{yoursplunkserver}:8089/services/admin/inputstatus/TailingProcessor:FileStatus

It will show you what is and what is not being processed, and why.

[bhavya_shah]

Firewall issue has been fixed but in splunkd.log I am seeing following error:

07-15-2013 04:09:36.647 -0700 INFO TailingProcessor - Could not send data to output queue (parsingQueue), retrying...
07-15-2013 04:09:38.935 -0700 INFO TailingProcessor - ...continuing.

07-15-2013 04:10:00.879 -0700 INFO BatchReader - Removed from queue file=

07-15-2013 09:27:38.824 -0700 INFO WatchedFile - Will begin reading at offset

[grijhwani]

In that case you seem to have firewall or routing issues.

[bhavya_shah]

I am still not getting logs but I have more issue.

/opt/splunkforwarder/bin/splunk list forward-server

Active forwards:
None
Configured but inactive forwards:
servername:9997

Here is the exact error on my forwarder on splunkd.log

07-11-2013 16:19:56.153 -0700 WARN TcpOutputProc - Cooked connection to ip=ipaddress:9997 timed out

[bhavya_shah]

I tried and it does show me complete inputs.conf info but still dont know why its not pushing the log.

[bhavya_shah]

Even though I added stanza in inputs.conf file I still dont see logs coming into splunk. I am not sure what I am missing here.

[msettipane]

http://docs.splunk.com/Documentation/Splunk/5.0.3/Data/Configureyourinputs

[adrianathome]

You need to create an inputs.conf file on your forwarder that has a stanza for all the logs that you want.

[adrianathome]

http://docs.splunk.com/Documentation/Splunk/5.0.3/Deploy/Exampleaddaninputtoforwarders

[bhavya_shah]

If you dont mind can you share the example inputs.conf file?