Splunk Search

How do I edit my search to dedup the same web page visited from the same user?

arkonner
Path Finder

I am using the following search to analyze which web pages have been visited.


index="access_combined_apache" Source_IP !=10.3.36.65 AND Source_IP !=localhost AND Source_IP !=127.0.0.1 AND Source_IP !=146.247* AND Source_IP !=- AND uri_path !=/access-denied.html AND uri_path !=*.ico AND uri_path !=*.png AND uri_path !=*.gif AND uri_path !=*.jpg AND uri_path !=*.js AND uri_path !=*.css AND uri_path !=*.jsp AND uri_path !=*.pdf AND uri_path !=*.ico AND uri_path !=*.html AND uri_path !=/image AND uri_path !=/c/* AND uri_path !=/c AND uri_path !=/image/* AND uri_path !=/template* AND uri_path !=/documents* | rename Source_IP as Client_Address| rename uri_path as Web_page_Visited |  table Client_Address Web_page_Visited _time

Using the dedup command on url_path, the result is a unique page visited over all users - I am looking to dedup the same page visited from the same user


Presently:

10.3.10.10 /
10.3.10.10 /
10.3.10.10 /pageA
10.3.10.10 /pageA
10.3.10.20 /
10.3.10.20 /tel
10.3.10.20 /pageC


What I am looking for:

10.3.10.10 /
10.3.10.10 /pageA
10.3.10.20 /
10.3.10.20 /tel
10.3.10.20 /pageC


Thank you in advance

Tags (1)
0 Karma

somesoni2
Revered Legend

Only the proper dedup is missing for your search. Try this

 index="access_combined_apache" Source_IP !=10.3.36.65 AND Source_IP !=localhost AND Source_IP !=127.0.0.1 AND Source_IP !=146.247* AND Source_IP !=- AND uri_path !=/access-denied.html AND uri_path !=*.ico AND uri_path !=*.png AND uri_path !=*.gif AND uri_path !=*.jpg AND uri_path !=*.js AND uri_path !=*.css AND uri_path !=*.jsp AND uri_path !=*.pdf AND uri_path !=*.ico AND uri_path !=*.html AND uri_path !=/image AND uri_path !=/c/* AND uri_path !=/c AND uri_path !=/image/* AND uri_path !=/template* AND uri_path !=/documents* | rename Source_IP as Client_Address| rename uri_path as Web_page_Visited |  table Client_Address Web_page_Visited _time | dedup Client_Address Web_page_Visited

This will the latest entries for each Client_Address Web_page_Visited, if they appear together (based on your sample data, you should get your expected output from this).

0 Karma

arkonner
Path Finder

Thank you and sorry I am new on this

0 Karma

somesoni2
Revered Legend

So.. did this search worked?

0 Karma

woodcock
Esteemed Legend

Just add another field; assuming this is your first command:

... | dedup url_path

You just do this:

... | dedup url_path user
0 Karma

arkonner
Path Finder

Thank you and sorry I am new on this

0 Karma

woodcock
Esteemed Legend

No problem; did it work for you?

0 Karma
Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...