How to build a Browsing Report per User?

[abarnett]

Hi All,

I'm trying to build a weekly report showing all the URLs every user has been to over that past week. I'm getting syslog from a Palo Alto Firewall, so every event contains the info I need.

Here's a sample event:

Jun 23 12:15:19 10.10.10.180 Jun 23 14:15:20 1,2010/06/23 14:15:20,0002C100679,THREAT,url,16,2010/06/23 14:15:20,10.40.10.65,72.14.204.99,0.0.0.0,0.0.0.0,Network and Systems Admin Dept,DOMAIN\USER,,web-browsing,vsys1,inside-trust,inside-untrust,ethernet1/20,ethernet1/19,Andrew - Splunk,2010/06/23 14:15:19,283986,1,50677,80,0,0,0x0,tcp,alert,"www.google.com/ig/feedjson",(9999),search-engines,informational,0

If I use the following search, I'll get returned the URLs and the number of times each were hit:

sourcetype="pan_threat" srcuser="DOMAIN\USER" | top category misc

The category field extracts: web-browsing
The misc field extracts: www.google.com/ig/feedjson

There are two steps that I can't figure out:

1. I'd like to filter down to just the domain name, I've seen a couple of posts on custom extracts for this, but haven't figured out how to apply that to a search.

2. There are 100s of users and we'd like a separate report per user (reports will be sent to managers) Is there a way to put an expression/wildcard in for the username so that it will create a report for every username found?

Any suggestions would be very much appreciated!

Thank you, Andrew

[kbains]

Just created an app for PAN data:

http://splunkbase.splunk.com/apps/All/4.x/App/app:Splunk+for+Palo+Alto+Networks

[gkanapathy]

1. You can either use the rex search command to specific a regular expression to extract the domain, or configure it as an automatic/permanent extraction that occurs on that sourcetype in props.conf, in the Splunk Manager UI, or using the interactive field extractor.
2. You could use ... | top category misc by user, but that will give you a single report broken up by user. I'm not sure if that's what you want, or if you actually need to run it as separate reports. If the latter, you could consider the map and sendemail commands together.

[cblanton]

I'm trying to achieve the "latter" with separate reports or at least separate email alerts. Can you provide more explanation of how to combine the map and sendemail commands?

[abarnett]

Hi

I just realized that instead of having to parse the URL for the domain, I could just resolve the destination IP address. Do you know if there is a way to have Splunk do an NSLookup?

thanks again!

[abarnett]

Thanks for the quick reply, gkanapathy.

the 'by user' definitely helps. I'm still pretty new to Splunk, can I tack on the rex command at the end of my search string to parse the domain data from the 'misc' field?

so ...| top category misc by user | rex field=misc ... ?

Any recommendations on how to do that?

thanks again!