Hello,
I have several lookup files in txt and it's in form like "blacksite1:123.123.123.1-123.123.123.17blacksite2:456.456.456.7-456.456.456.12blacksite3...."
Is there any method to use this file in the Splunk Search and Reporting app without changing the format into CIDR?
I've tried with search src_ip=123.123.123.1-123.123.123.17
, obviously it doesn't work. It works well if I try to convert the range format into CIDR or regular expression, however it is a long blacklist, and some ranges should be broke down into several CIDR expressions.
Thank you in advance,
If you are worried about the size of the lookup table, you could do a strategy of breaking the IP address up into its four components, and putting the values for the blocked 4th node into a single multi value variable.
| makeresults count=25 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append
[| makeresults count=25 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4]
| append
[| makeresults count=25 | eval IP1=789, IP2=789, IP3=789 | streamstats count as IP4]
| table IP1 IP2 IP3 IP4
| eval FullIP=IP1.".".IP2.".".IP3.".".IP4
| eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)
| join type=left IP1 IP2 IP3
[| makeresults count=17 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append
[| makeresults count=5 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 | eval IP4 = IP4+19]
| append
[| makeresults count=6 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4 | eval IP4 = IP4+6]
| eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)
| stats values(IP4_3) as IP4List_3 by IP1 IP2 IP3
]
| eval IP4Check=if(mvfind(IP4List_3,IP4_3)==0,"Blocked","NotBlocked")
| sort 0 FullIP
| table FullIP IP1 IP2 IP3 IP4 IP4Check IP4_3 IP4List_3
The above sample data generator create block records for these IP ranges
blacksite1:123.123.123.1-123.123.123.17
blacksite2:123.123.123.20-123.123.123.24
blacksite3:456.456.456.7-456.456.456.12
using this code
[| makeresults count=17 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append
[| makeresults count=5 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 | eval IP4 = IP4+19]
| append
[| makeresults count=6 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4 | eval IP4 = IP4+6]
| eval IP4_3 = case(len(IP4)=3,IP4,len(IP4)=2,0.IP4,00.IP4)
| stats values(IP4_3) as IP4List_3 by IP1 IP2 IP3
]
and then checks to see if the lookup (here coded as a left join) gets the right results for the first 25 IP4s in 123.123.123, 456.456.456, and 789.789.789, generated by this code
| makeresults count=25 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append
[| makeresults count=25 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4]
| append
[| makeresults count=25 | eval IP1=789, IP2=789, IP3=789 | streamstats count as IP4]
| table IP1 IP2 IP3 IP4
| eval FullIP=IP1.".".IP2.".".IP3.".".IP4
| eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)
| join type=left IP1 IP2 IP3 [the lookup table produced above]
then applies this test to see what returned, and pretties up the result with this -
| eval IP4Check=if(mvfind(IP4List_3,IP4_3)==0,"Blocked","NotBlocked")
| sort 0 FullIP
| table FullIP IP1 IP2 IP3 IP4 IP4Check IP4_3 IP4List_3
NOTE - code edited to format and use a 3-digit IP4, since a blocked IP4 =1 resulted in blocking all IP addresses with a 1 in them, and so on. OOPS.
Also edited to use sort 0 instead of sort in case there were more than 100 values to be sorted.
If you are worried about the size of the lookup table, you could do a strategy of breaking the IP address up into its four components, and putting the values for the blocked 4th node into a single multi value variable.
| makeresults count=25 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append
[| makeresults count=25 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4]
| append
[| makeresults count=25 | eval IP1=789, IP2=789, IP3=789 | streamstats count as IP4]
| table IP1 IP2 IP3 IP4
| eval FullIP=IP1.".".IP2.".".IP3.".".IP4
| eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)
| join type=left IP1 IP2 IP3
[| makeresults count=17 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append
[| makeresults count=5 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 | eval IP4 = IP4+19]
| append
[| makeresults count=6 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4 | eval IP4 = IP4+6]
| eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)
| stats values(IP4_3) as IP4List_3 by IP1 IP2 IP3
]
| eval IP4Check=if(mvfind(IP4List_3,IP4_3)==0,"Blocked","NotBlocked")
| sort 0 FullIP
| table FullIP IP1 IP2 IP3 IP4 IP4Check IP4_3 IP4List_3
The above sample data generator create block records for these IP ranges
blacksite1:123.123.123.1-123.123.123.17
blacksite2:123.123.123.20-123.123.123.24
blacksite3:456.456.456.7-456.456.456.12
using this code
[| makeresults count=17 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append
[| makeresults count=5 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 | eval IP4 = IP4+19]
| append
[| makeresults count=6 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4 | eval IP4 = IP4+6]
| eval IP4_3 = case(len(IP4)=3,IP4,len(IP4)=2,0.IP4,00.IP4)
| stats values(IP4_3) as IP4List_3 by IP1 IP2 IP3
]
and then checks to see if the lookup (here coded as a left join) gets the right results for the first 25 IP4s in 123.123.123, 456.456.456, and 789.789.789, generated by this code
| makeresults count=25 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append
[| makeresults count=25 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4]
| append
[| makeresults count=25 | eval IP1=789, IP2=789, IP3=789 | streamstats count as IP4]
| table IP1 IP2 IP3 IP4
| eval FullIP=IP1.".".IP2.".".IP3.".".IP4
| eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)
| join type=left IP1 IP2 IP3 [the lookup table produced above]
then applies this test to see what returned, and pretties up the result with this -
| eval IP4Check=if(mvfind(IP4List_3,IP4_3)==0,"Blocked","NotBlocked")
| sort 0 FullIP
| table FullIP IP1 IP2 IP3 IP4 IP4Check IP4_3 IP4List_3
NOTE - code edited to format and use a 3-digit IP4, since a blocked IP4 =1 resulted in blocking all IP addresses with a 1 in them, and so on. OOPS.
Also edited to use sort 0 instead of sort in case there were more than 100 values to be sorted.
It's complicated but works well. Thank you!
Glad to help!