Slow Downloads with Updater

I have been having and issue with a server running multiple instances of Nextcloud, each one configured for separate divisions so we can define different rules and allow different apps. They work just fine but every time I do updates the download process takes forever. I can manually fetch the file form cli on the server (running FreeBSD 14) in about 20sec or less but then the download step on the update takes 20-40 minutes to do the download.
Is there a way for me to put the file on the local file system ahead of time? I can’t just leave it to continue as it eventually puts the system into maintenance mode and if I miss it then I get calls from users complaining its down.

You could look into setting up a squid proxy in front of your Nextcloud instances and configure a Squid-in-the-middle setup. This way you can cache current updates and future updates for bother apps and system updates transparently.

It’s strange the download is that different between cli and updater, are you using the OCC command, the CLI updater or the web updater? Accoring to the docs the update archive is downloaded to /updater-INSTANCEID/downloads/ you could try dumping the update there to see if this changes anything but I doubt it.

I am running php updater/updater.phar to install the update.

I tried copying it to /updater-INSTANCEID/downloads/ it deletes it and starts download again.

Oddly though while trying that this morning, I did my test instance, which is only available inside the network. Thought it fixed it because download completed in a less than 5 seconds (2gbps link). Then went to next instance and was slow again. There isn’t something about having trusted proxies with my reverse proxy defined, or overwrite.cli.url defined in config.php that could be causing this issue is there? Those are the biggest differences between the test instance and live ones that is consistent among all the live instances.

I wouldn’t expect much, if any, performance difference. The download step in Updater essentially just uses curl to download. Are you using a proxy directive in your Nextcloud config by chance?

Also, when fetching yourself from the CLI, are testing within the same environment as each instance? i.e. if you’re using VMs on your host, are you testing from the VMs or from the host?

There’s a log created called updater.log in your datadirectory which will have details from the download step (and all others). That might be helpful to review for clues as to why this is occurring.

EDIT: The updatedirectory parameter could also be relevant, depending on your storage topology.

Yes same environment, all instances are running from same VM on ESX, using name based virtual hosting within Apache. Using haproxy on a second machine in DMZ to route traffic to server. Only proxy setup in config.php is the trust reverse proxy. Tried temporarily setting proxy in config.php and pointing it at a squid forward proxy I already have setup the first one ran fine. Second problem was back. maybe I am hitting a intermittent internet issue and just having bad luck that it is happening more than not.
All local storage is on same SAN with drives presented by ESX to guest.

Can you provide / compare the updater.log from the fast versus slow download situations?