Hello Splunkers.
I have an issue that I've been dealing with for the past 2 days but no success in solving it.
I'm working on a Splunk cluster environment, 3 SH and 2 IDX.
I have an UF installed in a SunOS machine.
This UF monitors a file called runlog.098880020 (the number is actually just an ID, it doesn't really matters).
This log can be found at the path /export/tsi/tsi/tsiout.1509/runlog.098880020
The thing is: the application creates a new folder every month (tsiout.1505, tsiout.1506, tsiout.1507, tsiout.1508, tsiout.1509....)
this is how I've setted my inputs.conf:
[monitor:///export/home/tsi/tsi/.../runlog*]
index = tsi
sourcetype = tsi_logs
However when Splunk starts to indexing the files, it indexes only a few folders (e.g., tsiout.1406 and tsiout.1409).
If I set my inputs.conf as following, I can see the current log beeing indexed:
[monitor:///export/home/tsi/tsi/tsiout.1509/runlog*]
index = tsi
sourcetype = tsi_logs
Do you guys know why this is happening?
Shouldn't the ...
tell Splunk to search in every folder for the runlog* file?
Thank you guys!
Regards!
Let's start with some basics:
Do all the directories and files underneath /export/home/tsi/tsi
have the same permissions? Does the user the UF is running as have appropriate permissions for those files and directories? Directories will need at least x to traverse them; files will need both r and x.
You may also need to look at the privileges assigned using usermod
on Solaris if the UF is not running as root (it should not be, that is a security risk). Look at /etc/user_attr
and review the bottom portion of this docs page to see if permissions match:
Run Splunk as Non-Root User
Are any errors being generated on the UF? You can search index = _internal
for the UF's hostname or IP; if nothing is showing up there, check on the UF itself. Logs will be in /opt/splunkforwarder/var/log/splunk/
Let's start with some basics:
Do all the directories and files underneath /export/home/tsi/tsi
have the same permissions? Does the user the UF is running as have appropriate permissions for those files and directories? Directories will need at least x to traverse them; files will need both r and x.
You may also need to look at the privileges assigned using usermod
on Solaris if the UF is not running as root (it should not be, that is a security risk). Look at /etc/user_attr
and review the bottom portion of this docs page to see if permissions match:
Run Splunk as Non-Root User
Are any errors being generated on the UF? You can search index = _internal
for the UF's hostname or IP; if nothing is showing up there, check on the UF itself. Logs will be in /opt/splunkforwarder/var/log/splunk/
What makes you think that it is not indexing? Tell the WHOLE story (did it ever get indexed before and now you are trying to reindex it)?
OK..
I don't need old logs..
I need to index this month logs and the ones that are coming.
I can see at the server that runlog.0901000208
exists under the folder tsiout.1509
, but Splunk is not indexing it (never indexed).
For your purposes, [monitor:///export/home/tsi/tsi/*/runlog*]
should work.
No luck in that either..
The files indexed where:
/export/home/tsi/tsi/tsiout.1409/runlog.0901000145
/export/home/tsi/tsi/tsiout.1209/runlog.0905001532
/export/home/tsi/tsi/tsiout.1209/runlog.0901001129
/export/home/tsi/tsi/tsiout.1209/runlog.0904213433
Maybe is there somthing to do with the month..
Only the folders tsiout.1209
and tsiout.1409
were indexed...
Have you checked the permissions on the folders that are not indexed?
Have you ever used ignoreolderthan
on these inputs?
I've never used ignoreolderthan