As part of the work to upgrade all of our forwarders in our enterprise, we are also pointing forwarders to a NEW deployment server.
This is done by running a bash script on each forwarder that downloads a new deployment application that contains a different deploymentclient.conf -- which points to the new deployment server URL:PORT.
The version of the forwarder software is also upgraded and the forwarder restarted.
All is well.
However, on some forwarders, the new deployment application that replaced a deployment application of the same name is removed (I can only assume by the synchronization of the deployment server) -- though not necessarily right away; sometimes there is a delay of unknown time.
We have experienced issues before with synchronization in the deployment server and the forwarders. Does this problem sound familiar?
Regards.
I'm a little unclear on some of the details here but some thoughts:
With my luck, none of that will help BUT give it a read and respond back as I'm sure you'll catch any incorrect assumptions or misunderstands I had.
If you utilise a new deployment server the app checksum's that is provided to the client will differ. As such the client will remove the old app(s) and install the new one(s).
This is why you need to use "sticky sessions" if you load balance deployment servers. Each deployment servers checksums are unique even if they contan that EXACT same (ie. md5sum shows exactly the same) files.
If you replace one deployment server for another this same behaviour will occur also.
The delay in replacing the app will be the phonehome time. This is also based on the initial start time of the instance.
This sentence really trips me up "the new deployment application that replaced a deployment application of the same name is removed". I'll make up a name to try to digest: the deployment app containing the deploymentclient.conf will be 'jim_rocks', the old DS will be called 'old_ds' and the new one 'new_ds'.
So, are you saying: the forwarders already had a 'jim_rocks' app with a deploymentclient.conf file pointing to 'old_ds:port'. You then have a script run that updates that file to 'new_ds:port'. Then that client connects to 'new_ds' and 'jim_rocks' is totally blown away. Is that the scenario?
In this sequence, when did you upgrade? Before the script run or after?
Does 'jim_rocks' live on the 'new_ds'? If so, did it exist on the 'old_ds'? How were the two apps on the two different DS different?
I'm a little unclear on some of the details here but some thoughts:
With my luck, none of that will help BUT give it a read and respond back as I'm sure you'll catch any incorrect assumptions or misunderstands I had.