Splunk Search

Splunk Data Input Whitelist: Regex not matching

beat_grob
Engager

I am trying to add a directory input monitor to Splunk. In this directory I have many different CSV files. Since there are different types of CSV files in there, I'd like to create a data input for each CSV type.

The file I am trying to filter is named like this:
11046819.2017-07-07_23-26-33.messages.csv
11046868.2017-07-08_00-58-38.messages.csv

The files I am trying to avoid are named like this:
11046868.1.2017-07-08_15_23_01.transform_properties.cfg.csv
11046868.1.2017-07-08_15_23_01.print_properties.cfg.csv
11046868.1.2017-07-08_15_23_01.positional_attributes.csv
11046868.1.2017-07-08_15_23_01.policy_properties.cfg.csv

Currently there are 8753 files in the directory, of which only 1094 are relevant. My Splunk Data Input configuration looks like this:
data input configuration

After creating the input it shows me that it found 8753 files (amounting to the total number of files in that directory). However, it did not index any records, when I search for entries from that input type, no record is being displayed.

If I remove the whitelist regex all files are being indexed and I can see them in the log viewer. But that doesn't allow me to create different extraction strategies for the different CSV file types.

I have also tried using "*.messages.csv" and "*messages.csv", all of which produce the same outcome. What am I doing wrong?

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

These two regex strings work on regex101.com with your sample file names.

.*\.messages\.csv

\d+\.\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}\.messages\.csv
---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

These two regex strings work on regex101.com with your sample file names.

.*\.messages\.csv

\d+\.\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}\.messages\.csv
---
If this reply helps you, Karma would be appreciated.

beat_grob
Engager

Thanks for the response. I had tried that, but it didn't work. After creating and assigning a new index, it started to work.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...