All Apps and Add-ons

Website input add-on: How do you change the user agent string?

xsploit
New Member

Hi,

How do you change the user agent string of website input?

Can this be included in the data inputs interface?

Thanks.

0 Karma

LukeMurphey
Champion

It doesn't yet, but I can certainly add the ability to set this.

Could you provide some context regarding why you need this? I just want to know more details because that could influence how I design this feature.

Let me know and I'll get stated on this; it shouldn't be too hard.

Update:
This has been added in version 1.1. You can specify the user-agent string on the input manager page or select from a dialog which includes a list of user-agents.

xsploit
New Member

Hi Luke, Thanks for developing this app and supporting it.

I am attempting to scrape a website and receive a HTTP 403 forbidden. I encountered a similar issue when I was using liburl2 in a python script. The website in question is blocking the user agent string so I was looking for a work around. I thought you might be using something similar so a quick edit of one of your scripts may be an option if possible. If you decided to add this into the tool, you may have a drop down for common user agent strings, that can also allow people to modify them.

0 Karma

LukeMurphey
Champion

I opened a ticket to cover this work, see: http://lukemurphey.net/issues/1034

Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...