Splunk Search

How to search a lookup file of IP ranges without changing the format into CIDR?

vj1226
New Member

Hello,
I have several lookup files in txt and it's in form like "blacksite1:123.123.123.1-123.123.123.17blacksite2:456.456.456.7-456.456.456.12blacksite3...."
Is there any method to use this file in the Splunk Search and Reporting app without changing the format into CIDR?
I've tried with search src_ip=123.123.123.1-123.123.123.17, obviously it doesn't work. It works well if I try to convert the range format into CIDR or regular expression, however it is a long blacklist, and some ranges should be broke down into several CIDR expressions.

Thank you in advance,

0 Karma
1 Solution

DalJeanis
SplunkTrust
SplunkTrust

If you are worried about the size of the lookup table, you could do a strategy of breaking the IP address up into its four components, and putting the values for the blocked 4th node into a single multi value variable.

| makeresults count=25 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append 
   [| makeresults count=25 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4]
| append 
   [| makeresults count=25 | eval IP1=789, IP2=789, IP3=789 | streamstats count as IP4]
| table IP1 IP2 IP3 IP4
| eval FullIP=IP1.".".IP2.".".IP3.".".IP4
| eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)

| join type=left  IP1 IP2 IP3 
   [| makeresults count=17 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 
    | append  
      [| makeresults count=5 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 | eval IP4 = IP4+19]
    | append  
      [| makeresults count=6 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4 | eval IP4 = IP4+6]
    | eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)
    | stats values(IP4_3) as IP4List_3 by IP1 IP2 IP3 
    ]
| eval IP4Check=if(mvfind(IP4List_3,IP4_3)==0,"Blocked","NotBlocked")

| sort 0 FullIP
| table FullIP IP1 IP2 IP3 IP4 IP4Check IP4_3 IP4List_3

The above sample data generator create block records for these IP ranges

blacksite1:123.123.123.1-123.123.123.17
blacksite2:123.123.123.20-123.123.123.24
blacksite3:456.456.456.7-456.456.456.12

using this code

   [| makeresults count=17 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 
    | append  
      [| makeresults count=5 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 | eval IP4 = IP4+19]
    | append  
      [| makeresults count=6 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4 | eval IP4 = IP4+6]
    | eval IP4_3 = case(len(IP4)=3,IP4,len(IP4)=2,0.IP4,00.IP4)
    | stats values(IP4_3) as IP4List_3 by IP1 IP2 IP3 
    ]

and then checks to see if the lookup (here coded as a left join) gets the right results for the first 25 IP4s in 123.123.123, 456.456.456, and 789.789.789, generated by this code

| makeresults count=25 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append 
   [| makeresults count=25 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4]
| append 
   [| makeresults count=25 | eval IP1=789, IP2=789, IP3=789 | streamstats count as IP4]
| table IP1 IP2 IP3 IP4
| eval FullIP=IP1.".".IP2.".".IP3.".".IP4
| eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)

| join type=left  IP1 IP2 IP3  [the lookup table produced above]

then applies this test to see what returned, and pretties up the result with this -

| eval IP4Check=if(mvfind(IP4List_3,IP4_3)==0,"Blocked","NotBlocked")

| sort 0 FullIP
| table FullIP IP1 IP2 IP3 IP4 IP4Check IP4_3 IP4List_3

NOTE - code edited to format and use a 3-digit IP4, since a blocked IP4 =1 resulted in blocking all IP addresses with a 1 in them, and so on. OOPS.

Also edited to use sort 0 instead of sort in case there were more than 100 values to be sorted.

View solution in original post

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

If you are worried about the size of the lookup table, you could do a strategy of breaking the IP address up into its four components, and putting the values for the blocked 4th node into a single multi value variable.

| makeresults count=25 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append 
   [| makeresults count=25 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4]
| append 
   [| makeresults count=25 | eval IP1=789, IP2=789, IP3=789 | streamstats count as IP4]
| table IP1 IP2 IP3 IP4
| eval FullIP=IP1.".".IP2.".".IP3.".".IP4
| eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)

| join type=left  IP1 IP2 IP3 
   [| makeresults count=17 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 
    | append  
      [| makeresults count=5 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 | eval IP4 = IP4+19]
    | append  
      [| makeresults count=6 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4 | eval IP4 = IP4+6]
    | eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)
    | stats values(IP4_3) as IP4List_3 by IP1 IP2 IP3 
    ]
| eval IP4Check=if(mvfind(IP4List_3,IP4_3)==0,"Blocked","NotBlocked")

| sort 0 FullIP
| table FullIP IP1 IP2 IP3 IP4 IP4Check IP4_3 IP4List_3

The above sample data generator create block records for these IP ranges

blacksite1:123.123.123.1-123.123.123.17
blacksite2:123.123.123.20-123.123.123.24
blacksite3:456.456.456.7-456.456.456.12

using this code

   [| makeresults count=17 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 
    | append  
      [| makeresults count=5 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 | eval IP4 = IP4+19]
    | append  
      [| makeresults count=6 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4 | eval IP4 = IP4+6]
    | eval IP4_3 = case(len(IP4)=3,IP4,len(IP4)=2,0.IP4,00.IP4)
    | stats values(IP4_3) as IP4List_3 by IP1 IP2 IP3 
    ]

and then checks to see if the lookup (here coded as a left join) gets the right results for the first 25 IP4s in 123.123.123, 456.456.456, and 789.789.789, generated by this code

| makeresults count=25 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append 
   [| makeresults count=25 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4]
| append 
   [| makeresults count=25 | eval IP1=789, IP2=789, IP3=789 | streamstats count as IP4]
| table IP1 IP2 IP3 IP4
| eval FullIP=IP1.".".IP2.".".IP3.".".IP4
| eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)

| join type=left  IP1 IP2 IP3  [the lookup table produced above]

then applies this test to see what returned, and pretties up the result with this -

| eval IP4Check=if(mvfind(IP4List_3,IP4_3)==0,"Blocked","NotBlocked")

| sort 0 FullIP
| table FullIP IP1 IP2 IP3 IP4 IP4Check IP4_3 IP4List_3

NOTE - code edited to format and use a 3-digit IP4, since a blocked IP4 =1 resulted in blocking all IP addresses with a 1 in them, and so on. OOPS.

Also edited to use sort 0 instead of sort in case there were more than 100 values to be sorted.

0 Karma

vj1226
New Member

It's complicated but works well. Thank you!

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Glad to help!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...