Can you help me with an inputs.conf wildcard issue?

[patouellet]

Hi,

I have a forwarder setup with this inputs.conf:

[monitor:///home/mqm/mqstatistics/splunk/*_QM_Q_*]
disabled = false
index = mq
sourcetype = qstats
crcSalt = <SOURCE>

[monitor:///home/mqm/mqstatistics/splunk/*_QM_CHL_*]
disabled = false
index = mq
sourcetype = chlstats
crcSalt = <SOURCE>

The location /home/mqm/mqstatistics/splunk/ has many files, here is a sample directory listing:

-rw-r--r---    1 mqm      mqm           30335 Sep 19 12:24 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-06.splunk
-rw-r--r---    1 mqm      mqm           29468 Sep 19 12:25 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-07.splunk
-rw-r--r---    1 mqm      mqm            5325 Sep 19 12:26 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-08.splunk
-rw-r--r---    1 mqm      mqm           10626 Sep 19 12:26 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-09.splunk
-rw-r--r---    1 mqm      mqm               0 Sep 19 13:18 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-10.splunk
-rw-r--r---    1 mqm      mqm           32233 Sep 19 13:19 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-11.splunk
-rw-r--r---    1 mqm      mqm           39100 Sep 19 13:20 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-12.splunk
-rw-r--r---    1 mqm      mqm           32861 Sep 19 13:20 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-13.splunk
-rw-r--r---    1 mqm      mqm           32758 Sep 19 13:21 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-14.splunk
-rw-r--r---    1 mqm      mqm            9269 Sep 19 13:21 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-15.splunk
-rw-r--r---    1 mqm      mqm           11222 Sep 19 13:22 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-16.splunk
-rw-r--r---    1 mqm      mqm           31818 Sep 19 13:23 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-17.splunk
-rw-r--r---    1 mqm      mqm           32847 Sep 19 13:23 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_CHL_statistics_2018-09-18.splunk
-rw-r--r---    1 mqm      mqm          178561 Sep 19 12:24 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-06.splunk
-rw-r--r---    1 mqm      mqm          177300 Sep 19 12:25 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-07.splunk
-rw-r--r---    1 mqm      mqm          128417 Sep 19 12:26 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-08.splunk
-rw-r--r---    1 mqm      mqm          140852 Sep 19 12:26 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-09.splunk
-rw-r--r---    1 mqm      mqm               0 Sep 19 13:18 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-10.splunk
-rw-r--r---    1 mqm      mqm          181606 Sep 19 13:19 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-11.splunk
-rw-r--r---    1 mqm      mqm          195047 Sep 19 13:20 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-12.splunk
-rw-r--r---    1 mqm      mqm          183082 Sep 19 13:20 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-13.splunk
-rw-r--r---    1 mqm      mqm          181658 Sep 19 13:21 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-14.splunk
-rw-r--r---    1 mqm      mqm          136505 Sep 19 13:21 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-15.splunk
-rw-r--r---    1 mqm      mqm          140286 Sep 19 13:22 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-16.splunk
-rw-r--r---    1 mqm      mqm          181603 Sep 19 13:23 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-17.splunk
-rw-r--r---    1 mqm      mqm          181470 Sep 19 13:23 /home/mqm/mqstatistics/splunk/BRT5TS01_QM_Q_statistics_2018-09-18.splunk

I confirm that I can read those files as the splunk ID. I also manually loaded a couple of those files in Splunk Enterprise and they look good.

Issue is: I'm not receiving any data. Everywhere I'm looking tells me I should be receiving data. The MQ index exists. There are no warning or errors in the logs. The forwarder reports this:

09-20-2018 12:46:49.014 -0400 INFO  TailingProcessor - Adding watch on path: /home/mqm/mqstatistics/splunk.
09-20-2018 12:46:49.014 -0400 INFO  TailingProcessor - Adding watch on path: /home/mqm/mqstatistics/splunk.
09-20-2018 12:46:49.013 -0400 INFO  TailingProcessor - Parsing configuration stanza: monitor:///home/mqm/mqstatistics/splunk/*_QM_Q_*.
09-20-2018 12:46:49.013 -0400 INFO  TailingProcessor - Parsing configuration stanza: monitor:///home/mqm/mqstatistics/splunk/*_QM_CHL_*.

I am receiving data from other sources for this Forwarder, just not this one. Why doesn't this inputs.conf work?

Thanks.

[patouellet · Accepted solution]

Have to thank Splunk Support for this one: the files were being ignored because they had a .splunk extension. Splunk ignores those as it thinks they are metadata files.

[nick405060]

I'm pretty sure inputs.conf treats * weird. Try using [\s\S] instead, or %

I had a problem with * in inputs.conf with a different issue:

https://answers.splunk.com/answers/671735/why-is-blacklisting-windows-event-logs-on-a-deploy-1.html

[stcrispan]

Do all your files end in .splunk?

My advice is to add that to your wildcard pattern. so instead of

[monitor:///home/mqm/mqstatistics/splunk/*_QM_Q_*]

Make it

[monitor:///home/mqm/mqstatistics/splunk/*_QM_Q_*.splunk]

[patouellet]

They all end with .splunk. Changed the input stanzas to QM_Q.splunk and QM_CHL.splunk. No dice.

[stcrispan]

Have you tried it without CRC? or, in your CRSalt line, try putting quotes around the entire thing.

https://answers.splunk.com/answers/35210/crcsalt-issue.html

[patouellet]

Tried both. Still no dice. Thanks for the suggestion though.