Deployment Architecture

how to setup basic deployment for universalforwarder?

kyless
Explorer

What am I missing? (new to splunk, but have been reading all about deployment servers)

test environment with 2 servers --

Splunk 5 installed as deployment server @ server named x.y.z

Splunk UniversalForwarder 5 installed on client @ server name austest

On Deployment Server:

$SPLUNKHOME/etc/system/local/serverclass.conf

[serverClass:testing]

filterType = whitelist

repositoryLocation = /opt/splunk/etc/deployment-apps/testing/

whitelist.0 = aus*

Placed inputs.conf and outputs.conf at:

$SPLUNKHOME/etc/deployment-apps/testing/default

outputs.conf

[tcpout]

disabled = false

defaultGroup=splunkPOC

[tcpout:splunkPOC]

server=x.y.z:9997

[tcpout-server://x.y.z:9997]

inputs.conf

[monitor:///var/log/messages]

disabled=false

sourcetype=syslog

On client:

$SPLUNKHOME/etc/system/local/deploymentclient.conf

[target-broker:deploymentServer]

targetUri = x.y.z:8089


Enabled the receiver tcp port 9997 on the indexer. (Previously done when testing a 'non-deployment server' setup, which was full functional on forwarding from the client.)

Reloaded deploy-server

/opt/splunk/bin/splunk reload deploy-server

Check that client is configured:

/opt/splunkforwarder/bin/splunk list deploy-poll

Deployment Server URI is set to "x.y.z:8089".

I can see the client reaching the deployment server (at Splunk Web) and via:

(at deployment server)

/opt/splunk/bin/splunk list deploy-clients | grep 'hostname:'

hostname: austest

But the client doesn't appear to retrieve the inputs.conf or outputs.conf

(at client)

/opt/splunkforwarder/bin/splunk list forward-server

Active forwards:

None

Configured but inactive forwards:

None

No data is forwarded from the client.
What is wrong?

Where should the files be located after retrieval from the deployment server?

(Manual configuration works for the client to send data, but obviously isn't scalable.)

Tags (1)
1 Solution

sowings
Splunk Employee
Splunk Employee

If the app is called 'testing', the repositoryLocation you've specified are wrong. It's expected to be the directory containing the apps (e.g. $SPLUNK_HOME/etc/deployment-apps), not the name of the app itself.

Note that when the app is deployed to the client, it will be deployed to the $SPLUNK_HOME/etc/apps, so you can check the the filesystem for that. You may need to include a metadata/local.meta (to indicate sharing permissions) for the app in question. You may also need an app.conf in the app's local/ subdir.

Finally, changes to inputs.conf typically require a restart, so you won't see that system as a forwarder until the forwarder system has had its Splunk daemon restarted.

View solution in original post

sowings
Splunk Employee
Splunk Employee

If the app is called 'testing', the repositoryLocation you've specified are wrong. It's expected to be the directory containing the apps (e.g. $SPLUNK_HOME/etc/deployment-apps), not the name of the app itself.

Note that when the app is deployed to the client, it will be deployed to the $SPLUNK_HOME/etc/apps, so you can check the the filesystem for that. You may need to include a metadata/local.meta (to indicate sharing permissions) for the app in question. You may also need an app.conf in the app's local/ subdir.

Finally, changes to inputs.conf typically require a restart, so you won't see that system as a forwarder until the forwarder system has had its Splunk daemon restarted.

kyless
Explorer

Sorry for the delay. Other items took priority.

Adding in the [serverClass:testing:app:testing] to the serverclass.conf worked.

0 Karma

sowings
Splunk Employee
Splunk Employee

Were you ultimately able to get this working?

0 Karma

sowings
Splunk Employee
Splunk Employee

Ok, so the client is phoning home (good), but not realizing that it has to download the app, since you're not seeing it in PackageDownloadRestHandler.

If the snippet of serverclass.conf you're provided above is the whole thing, you may be missing an app declaration (i.e., send this app to servers in this class).

Consider adding [serverClass:testing:app:testing] to your serverclass.conf and reload the deployment server.

kyless
Explorer

Yes. Using grep from command line at both the DS and client against:
$SPLUNK_HOME/var/log/splunk/*

Yes. Client name appears on the DS when I run 'splunk list deploy-clients'.

0 Karma

sowings
Splunk Employee
Splunk Employee

Are you literally doing 'grep' from the command line, or using Splunk search? Splunk search won't find PackageDownload by itself, because that term doesn't exist "in isolation"; you'd have to search for the full PackageDownloadRestHandler.

But I'm going to guess that this client just doesn't know it's supposed to get the app. Does the host show up in splunk list deploy-clients?

0 Karma

kyless
Explorer

Ok. I'm new, thanks for helping get my mind around terminology.

I'm basically following --
http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Extendedexampledeployseveralstandardforwar...

I had used Splunk Web to create the serverclass configuration and it required a 'repository location'. I've now changed the serverclass.conf to reflect the base repository --

repositoryLocation = $SPLUNK_HOME/etc/deployment-apps

restarted Splunk on the DS
restarted Splunk on the client

Grepping for 'PackageDownload' doesn't show up on the DS logs, historical or new.

0 Karma

sowings
Splunk Employee
Splunk Employee

When you said that you created an inputs.conf and outputs.conf in the testing/default directory, you were in fact creating an "app" called testing, that should be sent to the client. The client will request it from the DS; this logs via a facility called PackageDownloadRestHandler. Grep for that in your splunkd.log on the DS.

It sounds like the client doesn't realize that it needs the app, OR the app isn't installing correctly. Did you update repositoryLocation in serverclass.conf, and reload deploy-server?

0 Karma

kyless
Explorer

Thanks.

I am not trying to install an app, just basic universalforwarder configuration. Installation of the *Nix app will come later if I can get the basics down.

I have tried restarting the client, with no joy.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...