I have a setup with similar computing power (running Apache without FPM) and tried to transfer a file with around 20GiB. Sometimes the upload would go up to more than 40MB/s but rarely stay that high. I also noticed that one Apache process constantly maxed out one CPU core.
another bump
I have fiddled with the exact issue lately and I have tuned any possible setting - at least to my knowlegde. However, I do think that these limitations come from SabreDAV, which is the WebDAV server NC deploys and I haven’t found anything particular useful for tuning Sabre/DAV for higher throughput with large uploads. Since my throughput speeds also seem to stall somewhere around 35 MB/s, there seems to be some general issue with that and it’s nothing you can get around tweaking Apache or Nginx.
I have the same issue. For me, seems like a router bottleneck. When i upload/download large amount of data from my nextcloud, router cpu (Asus rt-66u) is loaded to 100%. There is no speed problems using samba o.o (i get full 1Gbps speed, router cpu don’t hit 100% load).
…definetively no router bottleneck at as I am always checking from the local network first.
i just setup a nextcloud server (m5.xlarge) and windows client (c4.xlarge) at aws.
if you want to check the ec2 size: https://www.ec2instances.info
i used the settings from carsten rieger.
https://www.c-rieger.de/nextcloud-13-nginx-installation-guide-for-ubuntu-18-04-lts
i guess his settings are tuned.
using firefox as a client the download speed is ~90MB/s.
(I think thats the wire speed of the c4.xlarge.if have to test it with a 10GBit ec2.)
using cyberduck webdav the up-/download speed is ~20MB/s.
is there a way to speed up webdav?
1 Like
Seems like you did reach more then 30MB/s then … Firefox is using the WebDAV backend of NC of course, just a little bit of protocol extensions like junking … so in your case the limiting factor seems the cyberduck client to me …
how to test this? (if there is a speed limit in cyberduck.)
Can you try another Webdav client like for example Windows integrated Webdav implementation (probably not a good choice anyway but should be OK for testing) or Netdrive or WinSCP? There are surely a few more clients, but maybe you can find a difference in them already.
Well… I just learned something weird… on my home network, my NC instance, hosted on a VMWare 6.5 guest running Debian/Nginx, is able to upload ond download via Firefox with 100 MB/s, where as my NC instance at work, hosted on a Xen-Cluster running CentOS/Apache doesn’t go beyond 25 - 30 MB/s.
The thing is, both of them are configured equally, as far as the NC/redis/MySQL settings are concerned. And even today, where there is only very little load on the NC instance at work, there’s no higher throuput than in the busy hours.
I will probably setup another NC instance using Debian/Nginx to cross-check that on my work environment…
Did you tested it by any chance?
Yeah - in fact I already set up two different NC instances, one on Ubuntu with php-fpm and one on CentOS 7.5 with php-fpm. Both installations yield approx. 30 MB/s, when uploading/downloading via the NC web interface.
…and the CentOS one really is a powerful machine, with lots of RAM and a hell of drives (36 x 8TB running ZOL with 128 GB of RAM).
what kind of client did you use?
I tested right now on AWS EC2s (m5.xlarge 4core ,8GB, Up to 10 Gigabit, 8GB gp2 SSD).
Using winscp as a client on a m5.xlarge Windows 2018 basemachine connecting to another m5.xlarge running Ubuntu 18 nginx/Nextcloud server: 28MBit/s (speed between two t2.large: 25MBit/s)
Using wget on an Ubuntu 18 as a client on a m5.xlarge connecting to the above Nextcloud server:
500MBit/s.
(There was no difference when I was running wget on the Nextcloud server. Also 500MBit/s.)
Probably I saw the SSD speed limits. I transferred a single 1.5GB zip file.
What does that mean? Where is the bottle neck?
I don’t know what you mean… I used Firefox to conduct my up- and download speed tests. Seemed like the right thing to do to me, since this is the standard client, that most peaople will use.
I don’t care that much about the transfer speeds of the sync clients, since those syncs are happening mostly in the background anyway.
Bottle neck is most likely ssl and the fact that you hit single core performance. Here, xeon is slower than most desktops, since it is optimized for multicore use and also usually has lower CPU frequency.
I got better performance with php-fpm and all redis.
Then Firefox and WinSCP are the bootle necks.
Not the Nextcloud Server. Since on a m5.xlarge it can handle 500Mbit/s.
Or?
Just now I tested the transfer speed on a Nextcloud installation in a virtual on my laptop.
Nextcloud setup is always the same. If you want to repeat the tests on your hardware.
install ubuntu 18 (or 16)
curl -s https://raw.githubusercontent.com/ReinerNippes/nextcloud/master/prepare_system.sh | /bin/bash
ansible-pull --clean --force -d ~/nextcloud -i 'nextcloud' -U https://github.com/ReinerNippes/nextcloud.git nextcloud.yml \
-e "talk_install=false" \
-e "install_collabora=false" \
-e "install_onlyoffice=false" \
-e "dhparam_numbits=1024"
login admin/tOpSecrET2018
Form the Windows host:
Download with Firefox: 15-16 Mbit/s
Down- and Upload with Cyberduck: 8-9 Mbit/s (webdav)
Down- and Upload with WinSCP: 8-10 Mbit/s (webdav)
Down- and Upload with WinSCP: 15Mbit/s (sftp)
From the Nextcloud virtualbox:
wget --user admin --password tOpSecrET2018 https://192.168.56.101/remote.php/webdav/VMware-VMvisor-Installer-201701001-4887370.x86_64.iso --no-check-certificate
-> 50-60 Mbit/s
The file is a 300MB iso image. And there no other files in this Nextcloud.
Everything is running on the same laptop. No “real” network involved.
The laptop is an actual model with 8gb ram and ssd disc. But Nextcloud installation takes 34 min vs. 5 min on the EC2 machine.
Hmmm. Webdav on Windows seems to be a problem.
I actually set up my latest server using your Ansible playbook. 
(And I wanted to create an issue already, since its missing the php-ldap component, but I got drawn away from that before hitting the Sumbit button…).
I do have to back-off a bit, as one of my colleagues just told me, that he uploads and downloads to our new NC server at 100 MB/s, which is as fast as it goes on his 1GbE connection, whereas my MBP still maxes out at 25 to 30 MB/s. Hosts are on the same network and even on the same switch for that matter.
Again: same client setup? Maybe you the hdd speed of your client is the limit.
Since I got 500MB/s on the AWS machines I would say the php code of nextcloud isn’t a limiting factor.
The list of php packages is in the os specific task of prep_php.
I will add the ldap modul soon. Anything else missing?
No, it is actually my client, MBP Retina with SSD, that is behaving differently on my home network and at work. However, other clients on my work network also transfer at 70+ MB/s up/down, so I will probably leave it alone for the moment, since I do have more urgent things ahead.
Afair, the php-ldap module has been the only thing, that was missing for me. I didn’t install the letsencrypt and collabora parts, since I didn’t need them. Nevertheless, your Ansible playbook worked really well for me.