Getting Data In

How can I automate the downloading of universal forwarder?

petersonjared
Explorer

Everything I am reading is that to download via wget, cURL, etc, that you have to specify the full path that contains the specific version number in the name/path. How can I get the latest/current version through automation versus hard-coding the path?

whrg
Motivator

Hello @petersonjared,

You can see the latest download links in the HTML source code of https://www.splunk.com/en_us/download/universal-forwarder.html. Same for Splunk Enterprise with https://www.splunk.com/en_us/download/splunk-enterprise.html. Search for "data-link" and you will find the relevant lines which look like this:

<a class="splunk-btn sp-btn-solid sp-btn-pink" data-arch="x86_64" data-filename="splunkforwarder-9.1.2-b6b9c8185839-x64-release.msi" data-link="https://download.splunk.com/products/universalforwarder/releases/9.1.2/windows/splunkforwarder-9.1.2-b6b9c8185839-x64-release.msi" ...

I use grep to extract all such data-links:

$ curl https://www.splunk.com/en_us/download/universal-forwarder.html | grep -o -P 'data-link="[^"]+' | cut -c 12-
https://download.splunk.com/products/universalforwarder/releases/9.1.2/windows/splunkforwarder-9.1.2-b6b9c8185839-x64-release.msi
https://download.splunk.com/products/universalforwarder/releases/9.1.2/windows/splunkforwarder-9.1.2-b6b9c8185839-x86-release.msi
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839.x86_64.rpm
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-linux-2.6-amd64.deb
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-x86_64.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-armv8.deb
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839.aarch64.rpm
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-armv8.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-ppc64le.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839.ppc64le.rpm
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-s390x.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839.s390x.rpm
https://download.splunk.com/products/universalforwarder/releases/9.1.2/osx/splunkforwarder-9.1.2-b6b9c8185839-darwin-universal2.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/osx/splunkforwarder-9.1.2-b6b9c8185839-darwin-universal2.dmg
https://download.splunk.com/products/universalforwarder/releases/9.1.2/osx/splunkforwarder-9.1.2-b6b9c8185839-darwin-64.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/osx/splunkforwarder-9.1.2-b6b9c8185839-macosx-10.11-intel.dmg
https://download.splunk.com/products/universalforwarder/releases/9.1.2/freebsd/splunkforwarder-9.1.2-b6b9c8185839-FreeBSD11-amd64.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/freebsd/splunkforwarder-9.1.2-b6b9c8185839-freebsd-11.3-amd64.txz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/solaris/splunkforwarder-9.1.2-b6b9c8185839-SunOS-x86_64.tar.Z
https://download.splunk.com/products/universalforwarder/releases/9.1.2/solaris/splunkforwarder-9.1.2-b6b9c8185839-solaris-intel.p5p
https://download.splunk.com/products/universalforwarder/releases/9.1.2/solaris/splunkforwarder-9.1.2-b6b9c8185839-solaris-sparc.p5p
https://download.splunk.com/products/universalforwarder/releases/9.1.2/solaris/splunkforwarder-9.1.2-b6b9c8185839-SunOS-sparc.tar.Z
https://download.splunk.com/products/universalforwarder/releases/9.1.2/aix/splunkforwarder-9.1.2-b6b9c8185839-AIX-powerpc.tgz

Same for Splunk Enterprise:

$ curl https://www.splunk.com/en_us/download/splunk-enterprise.html | grep -o -P 'data-link="[^"]+' | cut -c 12-
https://download.splunk.com/products/splunk/releases/9.1.2/windows/splunk-9.1.2-b6b9c8185839-x64-release.msi
https://download.splunk.com/products/splunk/releases/9.1.2/linux/splunk-9.1.2-b6b9c8185839-Linux-x86_64.tgz
https://download.splunk.com/products/splunk/releases/9.1.2/linux/splunk-9.1.2-b6b9c8185839.x86_64.rpm
https://download.splunk.com/products/splunk/releases/9.1.2/linux/splunk-9.1.2-b6b9c8185839-linux-2.6-amd64.deb
https://download.splunk.com/products/splunk/releases/9.1.2/osx/splunk-9.1.2-b6b9c8185839-darwin-64.tgz
https://download.splunk.com/products/splunk/releases/9.1.2/osx/splunk-9.1.2-b6b9c8185839-macosx-10.11-intel.dmg

evolart
Engager

Awesome solution. Now I just need to figure out how I could achieve the same in Powershell since we automate installation through AWS SSM.

0 Karma

ReverendRhyme
Engager

Did you ever figure out how to automate the download in PowerShell? Looking for something similar.

0 Karma

kenzo1234
Observer

as I hit download from https://www.splunk.com/en_us/download/universal-forwarder.html#tabs/freebsd

it will only send me to the splunk software license agreement page and not going anywhere else. please tell me how can I download the file.

0 Karma

isoutamo
SplunkTrust
SplunkTrust
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...