Splunk Search

Search Issues with regex field1= "..." | regex field1="..."

I-Man
Communicator

I think i may have stumbled upon an error in Splunk.

The following search will filter out any 10.x.x.x and 172 private IP ranges and show the top 10 external sources. Works like a charm:

sourcetype=syslog | regex srchost!="172.(1[6-9].|2[0-9].|3[0-1].)[0-9]{1,3}.[0-9]{1,3}" | regex srchost!="(10.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3})" | top srchost

If i wanted to display the 10.x.x.x and 172 private IP addresses as the source, I should just be able to take out the != and replace with = right? This gives me no results. However, if I change the search to only use one regex statement it does work.

sourcetype=syslog | regex srchost="172.(1[6-9].|2[0-9].|3[0-1].)[0-9]{1,3}.[0-9]{1,3}" | top srchost

I have also done this with the 10.x.x.x range and i get results as well. If i try to combine the two into one search, i get nothing. It doesn't make sense that you can combine multiple regex field!= and you cannot combine regex field= in a single search. I have tried to do this many different way but the main issues is that regex field="..." | regex field="..." does not yield any results when i know results exist however i can use regex field!="..." | regex field!="..." which will provide results.

Thanks a million in advance for any help.

I-Man

Tags (1)
0 Karma
1 Solution

Ayn
Legend

It's not an error, rather it follows from the laws of logic.

When you combine two regexes using pipes you're putting a logic AND between them in your search, rather than OR as you seem to be expecting. So the first search is roughly equivalent to "[...] AND NOT (172.x addresses) AND NOT (10.x addresses)". If any event has an address of any of these classes the filter will match and so they will not be shawn.

However when you reverse this it becomes equivalent to "[...] AND (172.x addresses) AND (10.x) addresses" which won't match any events unless they have BOTH 10.x addresses and 172.x addresses in them.

You should note that you're not escaping the dots in the IP addresses in your regex, so for instance an address starting with 172.169 will be accepted. A better option in your case would probably be the CIDR matching functionality that is already built into Splunk:

sourcetype=syslog AND NOT (srchost="172.16.0.0/12" OR srchost="10.0.0.0/8") | top srchost

Disclaimer - it's been a while since I used that functionality and I'm unable to access any Splunk instance right now so there is a small risk that I forgot the syntax, but it should be like that.

View solution in original post

Ayn
Legend

It's not an error, rather it follows from the laws of logic.

When you combine two regexes using pipes you're putting a logic AND between them in your search, rather than OR as you seem to be expecting. So the first search is roughly equivalent to "[...] AND NOT (172.x addresses) AND NOT (10.x addresses)". If any event has an address of any of these classes the filter will match and so they will not be shawn.

However when you reverse this it becomes equivalent to "[...] AND (172.x addresses) AND (10.x) addresses" which won't match any events unless they have BOTH 10.x addresses and 172.x addresses in them.

You should note that you're not escaping the dots in the IP addresses in your regex, so for instance an address starting with 172.169 will be accepted. A better option in your case would probably be the CIDR matching functionality that is already built into Splunk:

sourcetype=syslog AND NOT (srchost="172.16.0.0/12" OR srchost="10.0.0.0/8") | top srchost

Disclaimer - it's been a while since I used that functionality and I'm unable to access any Splunk instance right now so there is a small risk that I forgot the syntax, but it should be like that.

I-Man
Communicator

In my persistence of trying every combination possible i had already tried using OR in between the regex statements. Splunk still didn't like it. Also, i did have the escaped periods in the regex's, must have ID10T error when posting.

However, your CIDR matching suggestion worked perfectly. Thanks Ayn!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...