Splunk Search

Regular expression in source

gimbil
Explorer

Hi All,

I have multiple sources like
a1.gz
a2.gz
a3.gz
a4.gz
a5.gz

and so one. How can I have a subset these as source in search? I want to have something like
source=a[1-3]* | stats …

to use a1.gz, a2.gz and a3.gz as source but it does not accept [1-3] and gives error.

Thanks

Tags (3)

somesoni2
SplunkTrust
SplunkTrust

Best option is to use the wildcard to get you filter done, as specified by @Ayn and @richgalloway. The regex options may be inefficient based on your data distribution among the source and filter, however, another option that you can try is to specify the required source name in the base search, using subsearch, something like this

index=blah [| metadata type=sources index=blah | table source | regex source="a[1-3].gz" ] | rest of the search

The subsearch will grab all the required source (a1.gz/a2.gz/a3.gz in your example) and generate an OR condition into the base search, so effectively your search will become:-

 index=blah ((source=a1.gz) OR (source=a2.gz) OR (source=a3.gz)) | rest of the search
0 Karma

richgalloway
SplunkTrust
SplunkTrust

You can use wildcards in the source specifier, but (AFAIK) not regex. So 'source="a*.gz" | ...' should work.

There's also '| where like(source,"a%.gz") | ...', but that's still not regex and is probably less efficient.

Finally, you can try '| regex source="a[0-9]+.gz" | stats ...', but that's probably still less efficient that the first option.

---
If this reply helps you, Karma would be appreciated.

Ayn
Legend

The search command does not support filtering using regexes. You'll either have to filter using wildcards and/or explicit individual terms, or use the separate regex operator as your second command, like this:

source=a* | regex source="a[1-3]*"

The drawback to this approach is that Splunk will read all events matching source=a* first before sending them to the regex command that then performs the filtering, which means more events will be read from disk than what is strictly necessary.

0 Karma

Ayn
Legend

No that looks about right, depending on how your filter looks more exactly and what sources you expect to be filtered out.

0 Karma

gimbil
Explorer

Sorry but for some reason source=a* | regex source="b" is still returning everything and not filtering at all. Am I missing something?

0 Karma

landen99
Motivator

Ayn did not say "b". I would refine her answer to:

source=a* | regex source="a[1-3].gz"
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...