Splunk Search

Ironport WSA "Top Google Search Terms" query

imarks001
Explorer

I am trying to come up with a search that would parse Google search queries made though my Ironport web proxy. I would like to add this as a real time report on a dashboard eventually. Does anyone have any tips for a regex that could parse the a Google search request URL and display on the words in the search? Bing results would be great too.

Thanks

Tags (2)

pschulz
Engager

If someone's stumbling upon this, I've made a simple field extration regex which does the job by filtering out all text between ampersands (&) where "q=" is showing up.

Just go to "Manager" - "Field extractions" - Click "New" and enter the line below in the field "Extraction/Transform" after choosing your app & source/sourcetype/host:
(?i)&q=(?P<_google_query>\w+[^&]*)&

Regards,
Phil

PS: Maybe someone will cure the bug of not beeing able to post "<thisisnotag!!!>" as it's regarded as html-tag 😉

0 Karma

tidavids
Engager

Assuming you have the WSA app installed that extrapolates the cs_url field, try the following:

"www.google.com/search" "q=" | rex field=cs_url "q=(?P<search>[^&]*)\&" | rex field=search mode=sed "s/\+/ /g" | top search

nkpiquette
Path Finder

Digging up an old threat, but this just came in handy. The sed I had to use in 6.2 was as follows:
rex field=search mode=sed "s/+/ /g"

0 Karma

imarks001
Explorer

Thanks, I tried this but I receive the following error. 1. Error in 'rex' command: Encountered the following error while compiling the regex 'q=(?P[^&]*)&': Regex: unrecognized character after (?P

0 Karma

imarks001
Explorer

The URL is getting truncated when I post it.

Here is everything after google.com

search?hl=en&source=hp&q=world+cup+winners&aq=3s&aqi=g-s10&aql=&oq=word+cup+&gs_rfai=Cqt3eklUiTPiZO4XyzASIzfSXBgAAAKoEBU_Q0UPQ&fp=d2eda4910a7445ea

0 Karma

imarks001
Explorer

Sorry for the confusion. I would be looking to parse the actual search query out of our WSA logs. For instance, I would want to pull "world cup winners" out of the following log entry and be able to report on top searches.

1277318549.935 295 192.168.150.6 TCP_MISS/200 9957 GET http://www.google.com/search?hl=en&source=hp&q=world+cup+winners&aq=3s&aqi=g-s10&aql=&oq=word+cup+&g... - DIRECT/www.google.com text/html DEFAULT_CASE-NoAuthPolicy-NoAuthGlobal-NONE-NONE-DefaultRouting

0 Karma

Simeon
Splunk Employee
Splunk Employee

Your question is confusing as there is no sample data to reference. Also, your use of parse is confusing. My guess is that you want to produce an extracted field for query terms within google and bing URLs. Please update your question with more details.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...