Splunk Search

Extract only global IP with rex

kuga_mbsd
New Member

Hi there,
I want to extract only global IP addresses of destination from the internet access logs.
Our server segments has both 10.0.0.0/8 and 192.168.0.0/16 exist and web-proxy records even from PC to 192.168.0.0/16 and 10.0.0.0/8 servers.
I could extract either 10.0.0.0/8 or 192.168.0.0/16 with rex, however cannot get not matching both these private networks.
How can I make it?

Tags (1)
0 Karma
1 Solution

laserval
Communicator

Given your example from the comments,

[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://www.google.com/ - DIRECT/173.194.120.67 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://192.168.200.10/- DIRECT/192.168.200.10 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://10.0.200.10/ - DIRECT/10.200.200.10 text/html

If you have the field dhost extracted, you can use the eval function cidrmatch("X",Y):

index=proxy_log | where NOT (cidrmatch("10.0.0.0/8", dhost) OR cidrmatch("192.168.0.0/16", dhost))

You might be able to remove the pipe and the where command keyword, I didn't really have data to test that on.

View solution in original post

0 Karma

kuga_mbsd
New Member

OMG, it's embarrassing...
Yeah, cidrmatch definitely works in this case!

Thanks a lot, both laserval and stephanefotso!!

0 Karma

stephanefotso
Motivator

OK. Means you just want to pick www.google.com . But since www.google.com is not one of your dhost field values you must extract that value from your raw event. Here you go

index=proxy_log | rex field=_raw "http\:\/\/(?<webaccess>\w+\.\w+\.\w+)"|table webaccess

Thanks

SGF
0 Karma

laserval
Communicator

Given your example from the comments,

[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://www.google.com/ - DIRECT/173.194.120.67 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://192.168.200.10/- DIRECT/192.168.200.10 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://10.0.200.10/ - DIRECT/10.200.200.10 text/html

If you have the field dhost extracted, you can use the eval function cidrmatch("X",Y):

index=proxy_log | where NOT (cidrmatch("10.0.0.0/8", dhost) OR cidrmatch("192.168.0.0/16", dhost))

You might be able to remove the pipe and the where command keyword, I didn't really have data to test that on.

0 Karma

stephanefotso
Motivator

Let's get your sample event, and please can you be more specific on the global Ip you want to extract?

SGF
0 Karma

kuga_mbsd
New Member

Hi stephanefotso,
our web-proxy log is like as folllows,

[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://www.google.com/ - DIRECT/173.194.120.67 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://192.168.200.10/- DIRECT/192.168.200.10 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://10.0.200.10/ - DIRECT/10.200.200.10 text/html

2 bottom logs are both for access to our intra servers, want to only pick up the web access to external like google.com.
I tried following, but doesnt work...
index=proxy_log | rex field=dhost "(?!^10.\d+.\d+.\d+|^192.168.\d+.\d+")

0 Karma

kuga_mbsd
New Member

oops, escape has gone...

0 Karma

stephanefotso
Motivator

If i have understood, you want to extract

www.google.com
192.168.200.10
10.0.200.10

and put them in the same field. Isn't it?
Also, is dhost a field in your events? if yes, can you let us see a sample value of that field?

SGF
0 Karma

kuga_mbsd
New Member

I want to only pick up www.google.com from the sample log since 192.168./10. are our internal web servers.
yes, dhost is our field which has destination IP address of www servers:173.194.120.67, 192.168.200.10, and 10.200.200.10.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...