Hi,
I have a requirement where I need to get the list of URL's the user has navigated through while accessing a particular application. I am able to get the list of URL's but I need to exclude some of these URL's which inlcude .css, .png, .gif, .favicon etc. For example I have the following URL's which were associated for a particular user's session.
https://example.com/abc/index.aspx
https://example.com/abc/region.aspx
https://example.com/abc/static/6bc3661/release/asdf.css
https://example.com/abc/static/6bc3661/release/asdf.gif
https://example.com/abc/static/6bc3661/release/asdfgh.jsp
What I want to achieve is that I need to display only the following URL's in the results and exclude .css, .gif etc.
https://example.com/abc/index.aspx
https://example.com/abc/region.aspx
https://example.com/abc/static/6bc3661/release/asdfgh.jsp
Can some please guide me how I can extract those.
Thanks in advance.
@rakeshyv0807, in order to filter unwanted url suffix or retain specific url suffix, you can either create a filter black list or whitelist respectively. It is better to keep this as lookup file in Splunk so that it can be altered/maintained better.
Following is a run anywhere example which lists css,pnj,gif,favicon
as balck list and filters out from the search result of URL field using NOT
keyword
| makeresults
| fields - _time
| eval URL="https://example.com/abc/index.aspx;https://example.com/abc/region.aspx;https://example.com/abc/static/6bc3661/release/asdf.css;https://example.com/abc/static/6bc3661/release/asdf.gif;https://example.com/abc/static/6bc3661/release/asdfgh.jsp"
| makemv URL delim=";"
| mvexpand URL
| search
NOT [| makeresults
| fields - _time
| eval filterList="css,pnj,gif,favicon"
| makemv filterList delim=","
| mvexpand filterList
| eval URL="*.".filterList
| table URL]
Please try out and confirm. The commands from first | makeresults
till | mvexpand URL
generates mock data as per the question.
@rakeshyv0807, in order to filter unwanted url suffix or retain specific url suffix, you can either create a filter black list or whitelist respectively. It is better to keep this as lookup file in Splunk so that it can be altered/maintained better.
Following is a run anywhere example which lists css,pnj,gif,favicon
as balck list and filters out from the search result of URL field using NOT
keyword
| makeresults
| fields - _time
| eval URL="https://example.com/abc/index.aspx;https://example.com/abc/region.aspx;https://example.com/abc/static/6bc3661/release/asdf.css;https://example.com/abc/static/6bc3661/release/asdf.gif;https://example.com/abc/static/6bc3661/release/asdfgh.jsp"
| makemv URL delim=";"
| mvexpand URL
| search
NOT [| makeresults
| fields - _time
| eval filterList="css,pnj,gif,favicon"
| makemv filterList delim=","
| mvexpand filterList
| eval URL="*.".filterList
| table URL]
Please try out and confirm. The commands from first | makeresults
till | mvexpand URL
generates mock data as per the question.
Please find below my current query which is extracting URL using eval. Currently I am displaying the list of the URL's with all the suffixes and I want to exclude few suffixes like .css, .gif, .png etc.
sourcetype="audit" subject=abcd | eval SUB1=(split(resource," ")),baseurl=mvindex(SUB1,0) | eval URL="https://".baseurl.requestUri | transaction subject| sort _time | streamstats list(timestamp) as time_stamp,list(proxyRoundTripMS) as Application_Responsetime ,list(applicationName) as application,list(URL) as URL_Path by subject, applicationName | table subject, time_stamp, application, URL_Path, Application_Responsetime
Can you suggest me how to modify my query to exclude those?
Thanks in advance.
can you try this search, for some reason this page is trying to take out the wildcards mentioned in the below search
sourcetype="audit" subject=abcd | eval SUB1=(split(resource," ")),baseurl=mvindex(SUB1,0) | eval URL="https://".baseurl.requestUri | transaction subject| sort _time | streamstats list(timestamp) as time_stamp,list(proxyRoundTripMS) as Application_Responsetime ,list(applicationName) as application,list(URL) as URL_Path by subject, applicationName | table subject, time_stamp, application, URL_Path, Application_Responsetime | search URL_Path!=*.css AND URL_Path!=*.gif
Thanks
@PowerPacked I have tried adding it and it's still not excluding those.
Hi @rakeshyv0807
If the results are extracted into a field called URL --- use the below search
your search | search URL_Path! = *.css AND URL_Path! = *.gif
Thanks