Recursive monitoring of directories

[sushma7]

Hi Team,

I have a folder by name Mumbai under C drive with subfolders in it.

If i edit the inputs.conf file as monitor://C:\Splunk\NPCI\Mumbai\* , will that monitor all recursively all the sub folders?

Correct me if my command is wrong.

Thanks & Regards,
Sushma.

[linu1988]

Hello Sushma,
It will look for the current directory. You need to set the recursive option to True in the stanza.

recursive = [true|false]
* If false, Splunk will not monitor subdirectories found within a
monitored directory.
* Defaults to true.

Your Monitor stanza

[monitor://C:\Splunk\NPCI\Mumbai]
recursive = true

Thanks

[sushma7]

Added the mentioned aboev lines into inputs.conf file, but new files are not getting indexed automatically, below are the errors gathered from forwarder box:

-0400 WARN Filesystemchangewatcher - error reading directory "C:\Splunk\NPCI\Mumabi\New Folder" : The operation completed successfully

-0400 WARN Filesystemchangewatcher - error reading directory "C:\Splunk\NPCI\Mumabi\2014_05_06" : The operation completed successfully

-0400 WARN FileClassifierManager - Unable to open "C:\Splunk\NPCI\Mumabi\2014_05_06\App01\Sl11.log" : The operation completed successfully

[splunker12er]

Considering all the files are text files,

[monitor://C:\Splunk\NPCI\Mumbai]
recursive = true
index = mdc
sourcetype = Mumbai
whitelist = \.txt$

1.Delete the early index data in search head (index=mdc|delete)
2.Edit your inputs.conf as above
3.Restart the forwarder

[sushma6]

I tried this:

[monitor://C:\Splunk\NPCI\Mumbai]
recursive = true
index = mdc
sourcetype = Mumbai

But not working.

It is not permission issue, because when I read individual files it is getting indexed i.e

[monitor://C:\Splunk\NPCI\Mumbai\2014_05_02\App01]
index = mdc

[monitor://C:\Splunk\NPCI\Mumbai\2014_05_03\App01]
index = mdc

but when I give the complete folder i.e.

[monitor://C:\Splunk\NPCI\Mumbai]
index = mdc

to monitor its not getting indexed.

[linu1988]

It must be permission issue. Try giving proper access or move the files to another drive like E: or F: , it will start reading everything...

[monitor://E:\Splunk\NPCI\Mumbai]
recursive = true
index=mdc

[splunker12er]

that should be fine.. just give a try .. if folders doesn't exit create folders under the path i mentioned

[sushma6]

Under search folder I could not find any local folder, there's only 3 folders under it by names default, lookups and metadata. I think if we edit inputs.conf under /etc/system/local its enough, is it not so?

[splunker12er]

whatever may be the file format , splunk will monitor the folder that you mentioned in the monitor stanza.
Create a inputs.conf under the path:
/etc/apps/search/local/

[monitor://C:\Splunk\NPCI\Mumbai]
recursive = true
index = mdc
sourcetype = Mumbai

test:
Copy a text file inside the folder that you want to monitor and paste the file inside the same folder and rename the file. or edit the same and copy paste some contents for recent logs. Also check your outputs.conf is correct

[sushma6]

Yes I did the same as you specified i.e. [monitor://C:\Splunk\NPCI\Mumbai]
recursive = true
index = mdc
But it's not detecting, I even want all of them to be indexed into the mdc that I have created rather than they getting indexed into default main. Hence I included index = mdc

[linu1988]

did you change the monitor stanza which i have specified? If the files are not monitored until now, it will be monitored.

[sushma7]

The files within the folder are text document. What is meant by active logs? What should I change?

[splunker12er]

Is the monitored folder have active logs ?
Try some editing in the files under the monitored path.. you should get data at search head

[linu1988]

splunk needs to recognize the files. What are the file format?

[sushma7]

As directed by you, I have used the above syntax i.e. monitor://C:\Splunk\NPCI\Mumbai*
recursive = true

But it is not getting indexed automatically, Is there anything else that I need to include?

Note: The folder Mumabi has 3 subfolders by names Axis, Hyderabad and NPCI. I need to monitor all these folders and contents within them.

Kindly correct the command if I am wrong somewhere.

[somesoni2]

Go to link below and search for "Note concerning wildcards and monitor" in the page

http://docs.splunk.com/Documentation/Splunk/latest/admin/Inputsconf

[sushma7]

Should there be any slash after * in the above command?

[linu1988]

yes that should be fine if you are looking for reading out all the files recursively. May be you want to put sourcetype as well.

[sushma7]

So my command needs to be monitor://C:\Splunk\NPC\Munbai*
recursive = true

Is that right?

[sushma7]

There are forward splashes between Splunk, NPCI and Mumbai, I dont knew as why it is not showing up

[sushma7]

the command is not showing up properly, it is monitor://C:\Splunk\NPCI\Mumbai*