Splunk Search

How do I edit my search to dedup the same web page visited from the same user?

arkonner
Path Finder

I am using the following search to analyze which web pages have been visited.


index="access_combined_apache" Source_IP !=10.3.36.65 AND Source_IP !=localhost AND Source_IP !=127.0.0.1 AND Source_IP !=146.247* AND Source_IP !=- AND uri_path !=/access-denied.html AND uri_path !=*.ico AND uri_path !=*.png AND uri_path !=*.gif AND uri_path !=*.jpg AND uri_path !=*.js AND uri_path !=*.css AND uri_path !=*.jsp AND uri_path !=*.pdf AND uri_path !=*.ico AND uri_path !=*.html AND uri_path !=/image AND uri_path !=/c/* AND uri_path !=/c AND uri_path !=/image/* AND uri_path !=/template* AND uri_path !=/documents* | rename Source_IP as Client_Address| rename uri_path as Web_page_Visited |  table Client_Address Web_page_Visited _time

Using the dedup command on url_path, the result is a unique page visited over all users - I am looking to dedup the same page visited from the same user


Presently:

10.3.10.10 /
10.3.10.10 /
10.3.10.10 /pageA
10.3.10.10 /pageA
10.3.10.20 /
10.3.10.20 /tel
10.3.10.20 /pageC


What I am looking for:

10.3.10.10 /
10.3.10.10 /pageA
10.3.10.20 /
10.3.10.20 /tel
10.3.10.20 /pageC


Thank you in advance

Tags (1)
0 Karma

somesoni2
Revered Legend

Only the proper dedup is missing for your search. Try this

 index="access_combined_apache" Source_IP !=10.3.36.65 AND Source_IP !=localhost AND Source_IP !=127.0.0.1 AND Source_IP !=146.247* AND Source_IP !=- AND uri_path !=/access-denied.html AND uri_path !=*.ico AND uri_path !=*.png AND uri_path !=*.gif AND uri_path !=*.jpg AND uri_path !=*.js AND uri_path !=*.css AND uri_path !=*.jsp AND uri_path !=*.pdf AND uri_path !=*.ico AND uri_path !=*.html AND uri_path !=/image AND uri_path !=/c/* AND uri_path !=/c AND uri_path !=/image/* AND uri_path !=/template* AND uri_path !=/documents* | rename Source_IP as Client_Address| rename uri_path as Web_page_Visited |  table Client_Address Web_page_Visited _time | dedup Client_Address Web_page_Visited

This will the latest entries for each Client_Address Web_page_Visited, if they appear together (based on your sample data, you should get your expected output from this).

0 Karma

arkonner
Path Finder

Thank you and sorry I am new on this

0 Karma

somesoni2
Revered Legend

So.. did this search worked?

0 Karma

woodcock
Esteemed Legend

Just add another field; assuming this is your first command:

... | dedup url_path

You just do this:

... | dedup url_path user
0 Karma

arkonner
Path Finder

Thank you and sorry I am new on this

0 Karma

woodcock
Esteemed Legend

No problem; did it work for you?

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...