How can I get the URL path list, visited by the user navigating through a particular application?

[rakeshyv0807]

Hi,

I have a requirement where I need to get the list of URL's the user has navigated through while accessing a particular application. I am able to get the list of URL's but I need to exclude some of these URL's which inlcude .css, .png, .gif, .favicon etc. For example I have the following URL's which were associated for a particular user's session.

https://example.com/abc/index.aspx
https://example.com/abc/region.aspx
https://example.com/abc/static/6bc3661/release/asdf.css
https://example.com/abc/static/6bc3661/release/asdf.gif
https://example.com/abc/static/6bc3661/release/asdfgh.jsp

What I want to achieve is that I need to display only the following URL's in the results and exclude .css, .gif etc.

https://example.com/abc/index.aspx
https://example.com/abc/region.aspx
https://example.com/abc/static/6bc3661/release/asdfgh.jsp

Can some please guide me how I can extract those.

Thanks in advance.

[niketn · Accepted solution]

@rakeshyv0807, in order to filter unwanted url suffix or retain specific url suffix, you can either create a filter black list or whitelist respectively. It is better to keep this as lookup file in Splunk so that it can be altered/maintained better.

Following is a run anywhere example which lists css,pnj,gif,favicon as balck list and filters out from the search result of URL field using NOT keyword

|  makeresults
|  fields - _time
|  eval URL="https://example.com/abc/index.aspx;https://example.com/abc/region.aspx;https://example.com/abc/static/6bc3661/release/asdf.css;https://example.com/abc/static/6bc3661/release/asdf.gif;https://example.com/abc/static/6bc3661/release/asdfgh.jsp"
|  makemv URL delim=";"
|  mvexpand URL
|  search 
   NOT [| makeresults
    | fields - _time
    | eval filterList="css,pnj,gif,favicon"
    | makemv filterList delim="," 
    | mvexpand filterList
    | eval URL="*.".filterList
    | table URL]

Please try out and confirm. The commands from first | makeresults till | mvexpand URL generates mock data as per the question.

[rakeshyv0807]

Please find below my current query which is extracting URL using eval. Currently I am displaying the list of the URL's with all the suffixes and I want to exclude few suffixes like .css, .gif, .png etc.

sourcetype="audit" subject=abcd  | eval SUB1=(split(resource," ")),baseurl=mvindex(SUB1,0) | eval URL="https://".baseurl.requestUri | transaction subject| sort _time | streamstats list(timestamp) as time_stamp,list(proxyRoundTripMS) as Application_Responsetime ,list(applicationName) as application,list(URL) as URL_Path by subject, applicationName | table subject, time_stamp, application, URL_Path, Application_Responsetime

Can you suggest me how to modify my query to exclude those?

Thanks in advance.

[PowerPacked]

can you try this search, for some reason this page is trying to take out the wildcards mentioned in the below search

   sourcetype="audit" subject=abcd  | eval SUB1=(split(resource," ")),baseurl=mvindex(SUB1,0) | eval URL="https://".baseurl.requestUri | transaction subject| sort _time | streamstats list(timestamp) as time_stamp,list(proxyRoundTripMS) as Application_Responsetime ,list(applicationName) as application,list(URL) as URL_Path by subject, applicationName | table subject, time_stamp, application, URL_Path, Application_Responsetime | search URL_Path!=*.css AND URL_Path!=*.gif

Thanks

[rakeshyv0807]

@PowerPacked I have tried adding it and it's still not excluding those.

[PowerPacked]

Hi @rakeshyv0807

If the results are extracted into a field called URL --- use the below search

your search | search URL_Path! = *.css AND URL_Path! = *.gif 

Thanks