Splunk Search

Splunk Data Input Whitelist: Regex not matching

beat_grob
Engager

I am trying to add a directory input monitor to Splunk. In this directory I have many different CSV files. Since there are different types of CSV files in there, I'd like to create a data input for each CSV type.

The file I am trying to filter is named like this:
11046819.2017-07-07_23-26-33.messages.csv
11046868.2017-07-08_00-58-38.messages.csv

The files I am trying to avoid are named like this:
11046868.1.2017-07-08_15_23_01.transform_properties.cfg.csv
11046868.1.2017-07-08_15_23_01.print_properties.cfg.csv
11046868.1.2017-07-08_15_23_01.positional_attributes.csv
11046868.1.2017-07-08_15_23_01.policy_properties.cfg.csv

Currently there are 8753 files in the directory, of which only 1094 are relevant. My Splunk Data Input configuration looks like this:
data input configuration

After creating the input it shows me that it found 8753 files (amounting to the total number of files in that directory). However, it did not index any records, when I search for entries from that input type, no record is being displayed.

If I remove the whitelist regex all files are being indexed and I can see them in the log viewer. But that doesn't allow me to create different extraction strategies for the different CSV file types.

I have also tried using "*.messages.csv" and "*messages.csv", all of which produce the same outcome. What am I doing wrong?

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

These two regex strings work on regex101.com with your sample file names.

.*\.messages\.csv

\d+\.\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}\.messages\.csv
---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

These two regex strings work on regex101.com with your sample file names.

.*\.messages\.csv

\d+\.\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}\.messages\.csv
---
If this reply helps you, Karma would be appreciated.

beat_grob
Engager

Thanks for the response. I had tried that, but it didn't work. After creating and assigning a new index, it started to work.

0 Karma
Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...