How to write a monitoring stanza in inputs.conf to monitor a file in linux using splunk.

[Hemnaath]

Hi All, Can any help me on how to write a stanza in inputs.conf to monitor a file in linux.

Files to be monitored :
opt/solr-tomcat/logs/solr_access.logs

Inputs.conf :
[monitor:///opt/solr-tomcat/logs/solr_access.logs]
index = websphere
sourcetype = tom:web:solr

I have written the above stanza to read the file from this location to splunk, but unable to get the data in splunk. I have restarted the splunk services.

Please guide me on this.

[jethompson_splu]

Hemnaath,

there were a couple of Good questions asked in regards to what you are attempting to complete. Now I will follow up with those questions and provide further information on getting that Monitor Setup.

The first thing I noticed is that your Monitoring Stanza has an extra / in it and that can cause issues. The Stanza Entry should look like:

[monitor://opt/solr-tomcat/logs/solr_access.logs]

This is also stating that the Log you are wanting to monitor is named solr_access.logs. If the Log has a different name or extension then what is listed in the Monitor Stanza then it would not get picked up by Splunk. The next thing to double check is that the Splunk Service Account (the user account you are running Splunk as) has permissions on the Directory & Files otherwise Splunk again will not be able to ingest the file.

The following Links will provide further insight on setting up your Splunk Environment and "Getting Data Into" your Splunk Environment:
http://docs.splunk.com/Documentation/Splunk/latest/Data/Usingforwardingagents

http://docs.splunk.com/Documentation/Splunk/6.6.3/Data/Monitorfilesanddirectories

You will also want to review your Splunk Logs for any specific errors being produced as that will shed some light on the issue being experienced. The logs are located: $SPLUNK_HOME/var/log/splunk/ (the default install location for Splunk is: /opt/splunk -> $SPLUNK_HOME). If you are still having issues and have a Splunk License you can submit a Support Case to Splunk for further assistance.

[jkat54]

The monitor stanza contains the proper amount of slashes.

 [monitor://(path to file)] 

So in Linux you end up with the triple slash.

 [monitor:///path/to/file]

[Hemnaath]

Hi jethompson, thanks for your effort on this, hey I had checked the inputs.conf stanza and there are other log parameters configured to be monitored from the same node under different path and we could see those data in splunk.

Path to monitor : /opt/alfrxxx4.2/tomcat/logs/xxxx_access.log
inputs.conf stanza
[monitor:///opt/alf*/tomcat/logs/_access.log]
sourcetype = tomcat:web:access
index = websphere

this stanza is working fine.
Similarly under the same node we to monitor another log data from this location "opt/solr-tomcat/logs/solr_access.logs" so I had written a stanza like this to get this data into splunk but we are not getting the data in splunk.

Inputs.conf :
[monitor:///opt/solr-tomcat/logs/solr_access.logs]
index = websphere
sourcetype = tom:web:solr

kindly guide me how to fix this issue.

[ddrillic]

A good place to start is at I can't find my data!.

[alemarzu]

Just to be sure, file extension is log or logs ?

[jkat54]

Does the user that splunkd is running as on the server have permission to read the log files?

Check this search for more details:

 index=_internal solr_access.logs host=NameOfServer

Where NameOfServer is the name of the websphere/solr/tomcat server.

[blbr123]

How to check the permissions?

If required permissions are not there then how to give the permission?

[isoutamo]

Hi
the easiest way is just log to this server and then do sudo -u <UF User> tail -10 <path to log file>
Of course this expect that there are events on this file. If not you could use also sudo -u <UF User> ls -laF <path to file>. If you can ls the file you probably have also the read access (not 100% sure) to this file.
r. Ismo

[Hemnaath]

Hi jkat, thanks for your effort on this, Yes splunkd is running fine in this node, as we could see the data for other two logs source which is configured from this node.

The below stanza is also configured in inputs.conf along with another stanza which is not fetching the data.

Path to monitor : /opt/alfrxxx4.2/tomcat/logs/xxxx_access.log
inputs.conf stanza
[monitor:///opt/alf*/tomcat/logs/_access.log]
sourcetype = tomcat:web:access
index = websphere

[monitor:///opt/solr-tomcat/logs/solr_access.logs] -- > Not fetching the data in splunk
index = websphere
sourcetype = tom:web:solr
Path where log source resides : opt/solr-tomcat/logs/solr_access.logs

I had executed the query

index="_internal" "solr_access.logs*" host ="xxxx" log_level=INFO but getting no result found.

Kindly guide me how to fix this issue.

[jkat54]

I didn't say log_level=info

Check this search for more details:

index=_internal solr_access.logs host=NameOfServer

[Hemnaath]

Hi Jkat, thanks for your effort on this, I had checked the parameter given by the user and found a typo error instead of "solr_access.log" we trying to capture "solr_access.logs" . So corrected it and splunk started getting the data into it.

In-correct (typo error)
Inputs.conf :
[monitor:///opt/solr-tomcat/logs/solr_access.logs]
index = websphere
sourcetype = tom:web:solr

corrected stanza

Inputs.conf :
[monitor:///opt/solr-tomcat/logs/solr_access.log]
index = websphere
sourcetype = tom:web:solr

thanks

[richgalloway]

Are you monitoring the file using a Universal Forwarder? If so, is the inputs.conf file on the forwarder?

[Hemnaath]

Hi Richgalloway, thanks for your effort on this, Yes we have installed the UF in this node and I had checked the inputs.conf stanza, there are other log parameter configured to be monitored from the same node under different path and we could see those data in splunk.

Path to monitor : /opt/alfrxxx4.2/tomcat/logs/xxxx_access.log
inputs.conf stanza
[monitor:///opt/alf*/tomcat/logs/_access.log]
sourcetype = tomcat:web:access
index = websphere

this stanza is working fine.
Similarly under the same node we to monitor another log data from this location "opt/solr-tomcat/logs/solr_access.logs" so I had written a stanza like this to get this data into splunk but we are not getting the data in splunk.

Inputs.conf :
[monitor:///opt/solr-tomcat/logs/solr_access.logs]
index = websphere
sourcetype = tom:web:solr

kindly guide me to fix this issue.