Is fschange supposed to generate an even each time it checks or only if a file was changed since the last check?
According to the following article, http://www.splunk.com/base/Documentation/4.0.9/Admin/Monitorchangestoyourfilesystem:
generates an event (in Splunk) when that directory undergoes any change.
However that's not what I'm getting when trying to monitor folder. I'm getting a message like this one each 60 seconds:
6/14/10 9:20:52.000 AM Mon Jun 14 09:20:52 2010 action=add, path="C:\TEMP\configs.txt", isdir=0, size=388, gid=-1, uid=-1, modtime="Mon Jun 14 09:17:56 2010", mode="rwxrwxrwx", hash=
What am I doing wrong?
inputs.conf:
[fschange:C:\TEMP\]
index = main
recurse = false
followLinks = false
signedaudit = false
fullEvent = false
sendEventMaxSize = 1048576
delayInMills = 1000
pollPeriod = 60
You should get an event like this only when a change is detected. Are you sure that configs.txt
is not being changed (or added and removed) periodically by some process on your system? You should see this repeat in such a situation.
You could look to see if you have duplicate events by running a search like this:
source=fschangemonitor | stats count by host,action,path,size,modtime,mode
If you see "count" higher than 1 than you could be encountering a bug. (It's also possible that the file is being added and removed, in which case you should see a similar number of "add" and "remove" events.)
yo whatup!
If somebody cares, we've finally found the reason for such weird behavior. Account that we use to run Splunk services didn't have appropriate permissions over C:\Program Files\Splunk\var\lib\splunk\persistentstorage folder.
It looks like Splunk service was failing to update information in it's internal records when monitored file is changed.
The difficulty was that Windows file audit wasn't showing any failed access attempts for Splunk directory. We had to use Process Monitor from Sysinternals to see which files Splunk is trying to access and double-check the permissions.
Conclusion is that if you change account to run Splunk services, make sure it has full control over Splunk directory and all its sub-folders. Also, Windows 2008 sometimes doesn't properly propagate permissions to sub-folders when you change them for parent folder.
I had the same problem Windows 2003 x64, latest version of splunk 4.3.3, build 128297. Adjusting permissions on C:\Program Files\Splunk\var\lib\splunk\persistentstorage fixed it. Thanks for posting!!
Thank you for the reply.
I've enabled file access audit to make sure no other process is accessing the file. There are only Splunk read attempts every 60 seconds:
An attempt was made to access an object.
Subject:
Security ID: TEIG-LOG\Splunk
Account Name: Splunk
Account Domain: TEIG-LOG
Logon ID: 0x2a57e87a7
Object:
Object Server: Security
Object Type: File
Object Name: C:\TEMP\configs.txt
Handle ID: 0x6a0
Process Information:
Process ID: 0x660
Process Name: C:\Program Files\Splunk\bin\splunkd.exe
Access Request Information:
Accesses: READ_CONTROL
Access Mask: 0x20000
Here is what search for duplicate events returns:
host action path size modtime mode count
2 TEIG-LOG add C:\TEMP\configs.txt 388 Mon Jun 14 09:17:56 2010 rwxrwxrwx 14
So you are saying it may be a bug? I'm running version 4.0.9, build 74233
I've opened a support case but haven't heard anything back yet. Also, I've upgraded to the latest version (4.1.3-80534) but it's still the same.
I don't know for sure, but it certainly sounds like a bug. I know there are issues with the fschange
input on Windows, but I have no idea if this is one of the known problems or not. In any case, it's probably best to open a support case with splunk support.
You should get an event like this only when a change is detected. Are you sure that configs.txt
is not being changed (or added and removed) periodically by some process on your system? You should see this repeat in such a situation.
You could look to see if you have duplicate events by running a search like this:
source=fschangemonitor | stats count by host,action,path,size,modtime,mode
If you see "count" higher than 1 than you could be encountering a bug. (It's also possible that the file is being added and removed, in which case you should see a similar number of "add" and "remove" events.)