Whitelist not picking up files - heavy forwarder monitoring folder

[shantu]

I've set up a heavy forwarder on "stage2" (linux machine) and a central Splunk instance on "stage1" (another linux machine). I'm trying to monitor the following folder on stage2:

/x/web/stage2/logs/

and I want to pick up the following files:

ALERT.log.15-08-01.log
ALERT.log.15-08-06.log
ALERT.log.15-08-07.log
ALERT.log.15-08-08.log
ALERT.log.15-08-09.log
ALERT.log.15-08-10.log
ALERT.log.15-08-11.log
ALERT.log.15-08-12.log
ALERT.log.15-08-13.log
ALERT.log.15-08-14.log
ALERT.log.15-08-15.log
ALERT.log.15-08-16.log
ALERT.log.15-08-17.log
ALERT.log.15-08-18.log
ALERT.log.15-08-19.log
ALERT.log.15-08-20.log
ALERT.log.15-08-21.log
ALERT.log.15-08-22.log
ALERT.log.15-08-23.log
ALERT.log.15-08-24.log
ALERT.log

It is important that the rollover/historical logs get picked up. (FYI I had no say in the naming convention used for them)

Steps I took:

• Monitoring /x/web/stage2/logs/ with the whitelist /ALERT.*/g . This brings only the ALERT.log to stage1
• Maybe my regex was bad, so I purged all stage2 logs from stage1, and delete the monitoring rule on stage2. Starting over.
• Monitoring /x/web/stage2/logs/ with no whitelist defined. Restart stage2. Restart stage1 for good measure. Again, this brings only the ALERT.log to stage1.
• Purged all stage2 logs from stage1, and delete the monitoring rule on stage2. Starting again.
• Monitoring /x/web/stage2/logs/ with the most generous regex defined as the whitelist. Restart stage2, then stage1. Same result.

On stage2, under Settings -> Data Inputs -> Files & Directories, the monitoring rule shows a high enough number to where I believe stage2 is monitoring all the files I want it to, but only the one ALERT.log file makes it to stage1.

Is there something that prevents Splunk from picking up logs named a certain way? Anyone with any ideas about what might be going on here?

[agitelzon]

First check that the splunk user has permission to read all of these log files.

Here is what I would put in the inputs.conf file because all of your logs end in .log.

[monitor:///x/web/stage2/logs/]

disabled = false

blacklist = .(gz|bz2|swp|z|zip)$

whitelist = .log$