Sync speed limited to 30MB/s, WebDAV/php bottleneck?


#1

Closed bugreport here: https://github.com/nextcloud/server/issues/9736
Anyone here reached higher than 30MB/s with Nextcloud?
Nextcloud is awesome in everything but speed for copying large files, am looking for a way to improve it.
Have caching setup per documentation, tried fiddling with nginx/php settings without joy.

Edit: when monitoring a file transfer I see high CPU usage pegged at one core for processes of nginx and php-fpm.


#2

Bumping the post.


#3

I have a setup with similar computing power (running Apache without FPM) and tried to transfer a file with around 20GiB. Sometimes the upload would go up to more than 40MB/s but rarely stay that high. I also noticed that one Apache process constantly maxed out one CPU core.


#4

another bump


#5

I have fiddled with the exact issue lately and I have tuned any possible setting - at least to my knowlegde. However, I do think that these limitations come from SabreDAV, which is the WebDAV server NC deploys and I haven’t found anything particular useful for tuning Sabre/DAV for higher throughput with large uploads. Since my throughput speeds also seem to stall somewhere around 35 MB/s, there seems to be some general issue with that and it’s nothing you can get around tweaking Apache or Nginx.


#6

I have the same issue. For me, seems like a router bottleneck. When i upload/download large amount of data from my nextcloud, router cpu (Asus rt-66u) is loaded to 100%. There is no speed problems using samba o.o (i get full 1Gbps speed, router cpu don’t hit 100% load).


#7

…definetively no router bottleneck at as I am always checking from the local network first.


#8

i just setup a nextcloud server (m5.xlarge) and windows client (c4.xlarge) at aws.
if you want to check the ec2 size: https://www.ec2instances.info

i used the settings from carsten rieger.
https://www.c-rieger.de/nextcloud-13-nginx-installation-guide-for-ubuntu-18-04-lts

i guess his settings are tuned.

using firefox as a client the download speed is ~90MB/s.
(I think thats the wire speed of the c4.xlarge.if have to test it with a 10GBit ec2.)

using cyberduck webdav the up-/download speed is ~20MB/s.

is there a way to speed up webdav?


#9

Seems like you did reach more then 30MB/s then … Firefox is using the WebDAV backend of NC of course, just a little bit of protocol extensions like junking … so in your case the limiting factor seems the cyberduck client to me …


#10

how to test this? (if there is a speed limit in cyberduck.)


#11

Can you try another Webdav client like for example Windows integrated Webdav implementation (probably not a good choice anyway but should be OK for testing) or Netdrive or WinSCP? There are surely a few more clients, but maybe you can find a difference in them already.


#12

Well… I just learned something weird… on my home network, my NC instance, hosted on a VMWare 6.5 guest running Debian/Nginx, is able to upload ond download via Firefox with 100 MB/s, where as my NC instance at work, hosted on a Xen-Cluster running CentOS/Apache doesn’t go beyond 25 - 30 MB/s.

The thing is, both of them are configured equally, as far as the NC/redis/MySQL settings are concerned. And even today, where there is only very little load on the NC instance at work, there’s no higher throuput than in the busy hours.

I will probably setup another NC instance using Debian/Nginx to cross-check that on my work environment…