Nextcloud Upload Speeds Slow Depending on Geographical Location

Nextcloud version (eg, 20.0.5): 24.0.1
Operating system and version (eg, Ubuntu 20.04): Debian 11
Apache or nginx version (eg, Apache 2.4.25): Apache/2.4.53 (Debian)
PHP version (eg, 7.4): 8.0.19

The issue you are facing: Slow upload and download speeds in Ohio, fine in many other lokations.

Is this the first time you’ve seen this error? (Y): (Not an error specifically.)

Steps to replicate it:

  1. Go to Ohio.
  2. Attempt to upload.
  3. Speeds are lower than 1 MiB a second.

Speeds have been tested in both Washington D.C., North Carolina, und many other places between. iperf tests show that it is not a bandwidth issue affecting this, and multiple devices across multiple networks and months have shown this issue repeatedly. ISP’s even differ. It only seems to affect Ohio in particular, as no other testing location has shown this. I would like to resolve this quickly, as I am leaving Ohio soon and do not have a chance to return for possibly years, however, the other users of this instance are in Ohio (family and friends.)

I tuned PHP-FPM, used Redis and APCu, used Redis for file locking, used http 2, postgresql, and enabled the opcache. PHP JIT has been enabled and disabled for testing, with the same done to http2. The file chunking limit has been disabled to test, and while this improved performance on unaffected areas (30 MiB to 60 MiB a second) speeds in Ohio remain at 0.7 MiB a second. Speeds have also been tested using a Samba share through a VPN tunnel transferring the same files. They max out the drive at 60 MiB a second.

Nextcloud.log has been truncated to reduce file size, as it was 90 MiB initially.
nextcloud.log
config.php
apache2.log

Any help would be much appreciated! This is a real head-scratcher.

That is an insteresting one :smile:

Have you tested other Nextcloud setups, you could e.g. use try.nextcloud.com if you see the same difference.

In your configuration, you have set up proxies. Are these used to access Nextcloud, and is it always the same?

Is it possible to try to access Nextcloud through the same VPN tunnel?

Ah, apologies for not responding, I was driving for ~9 hours on the day you responded :sweat_smile:.

A bit more info on my setup: Both hairpin nat’ting and split DNS are setup, so if a device isn’t using my DNS servers, it goes through the hairpin, but if it is using those servers, it accesses Nextcloud’s IP directly.

Nextcloud was seeing the hairpin NAT as a proxy, as the request IP’s were rewritten to my router’s IP, so I added the router as a proxy. The router as a proxy is only used when a device is going through the hairpin, which sadly is not the case here. External connections such as the ones going to Nextcloud don’t use the proxy. These also include the internal connections I have used, such as my wired desktop, along with my phone & laptop on Wifi.

While in DC and NC, I used the VPN to access Nextcloud and saw the same (good) speeds, but I forgot to when in Ohio. As I have left, I can ask one of my family members there to do so.

I had no clue that try.nextcloud.com existed, and will try that now.

Thank you for taking the time to try and help, this problem has been bugging me for months! I was considering switching to OCIS as I heard it was faster, but I couldn’t figure out how to set it up.

Edit: I’ve noticed that try.nextcloud.com is much faster in the web interface alone. It is extremely snappy, and the photos menu actually works (on my instance it lags to the point where Firefox closes the tab) which suggests other potential issues with my instance. I have also reached out to family in Ohio to test through the same VPN connection, and to try uploading/downloading from try.nextcloud.com

On try.nextcloud.com, I get ~11 MB upload, ~16 MB download over wifi. The family member in Ohio gets 500 KB upload, 16 MB download over ethernet. We both have gigabit up and down plans from our ISP’s, with gigabit speeds reflected in our iPerf tests to one another’s networks, both normally and with the reverse flag. I am going to try the VPN soon.

Edit: It appears that the VPN also does not help their speeds. I am at a loss as to what is happening, as transfers through means other than Nextcloud to this same server are perfectly fine, so it seems to be Nextcloud itself.

It looks a bit as if this type of traffic upstream is throttled. For that reason I suggested the VPN, so the ISP doesn’t see the http-traffic. If there is a throttle somewhere, we don’t know if it is based on port, package inspection etc., if not you could try iperf with different options. On the computer, there is no antivirus/firewall that tries to inspect all packages? Problem is that it should affect the downstream traffic as well.

Nextcloud itself has no option to throttle traffic. This could only be done by the webserver (or lower level).

You already configured caching which is the first step. You can as well optimize your database cache (tuning-primer.sh and mysqltuner.pl can help you with that).

Sadly, it does not appear to be a throttle. Mediafire uploads reach speeds of over 16 MiB, even though the VPN did not help. iperf upload speeds also max out the internet connection, showing that they can actually be reached. There is no antivirus or firewall either. Thanks for the SQL tuning suggestions, I’ll look into their Postgres equivalents!

Hah! So, it turned out to be very bad luck. All of the people who tested it in Ohio had various issues impacting upload. It has been fixed now.