Nextcloud only using around 50% of my internet connection's available upstream bandwidth

Nextcloud version: NextCloudPi 16.0.3 Docker
Docker engine version: 19.03.1
Operating system and version: macOS 10.12.6
Hardware: Mac Pro early 2009

The issue you are facing:

Nextcloud is only using around 50% of my internet connections available upstream bandwidth. My internet connection has a maximum upload speed of 20 mbps, and whenever I download a file from my Nextcloud server, it generally only uses around 8-12 mbps of bandwidth. Or put another way, the file will download at around 1-1.5 MiB/s, sometimes even less.

In case it’s relevant: My Nextcloud server was installed using NextcloudPi docker. Because my server is a 2009 Mac Pro, it cannot run the latest version of Docker and so the container is running inside a VirtualBox VM.

Some diagnostics and problem solving I’ve done:

It’s not another app or service using the bandwidth. I’ve made sure that no other service on that network is uploading anything when I performed these tests.

It’s not the server itself. Other services on the server that upload data utilize the full 20 mbps. Furthermore, using a bandwidth test such as speedtest.net on this server consistently reports close to 20 mbps of bandwidth.

It’s not docker or VirtualBox. Other docker containers can also utilize the full upstream bandwidth without any issue, including a container I installed that runs a bandwidth test. [EDIT: Apparently it is! See my posts below.]

It’s not my Nextcloud client. Every client I use exhibits the same issue.

It’s not the client’s internet connection. I’ve tested downloading the same file from my server from multiple different locations and ISPs.

Can anyone help me figure out what’s wrong? I routinely use this Nextcloud server to send large files, and as things stand, it will take twice as long to transfer them as it should, and sometimes even longer than that.

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

  1. Connect to my Nextcloud server using its web interface
  2. Navigate to a large file (100 Mb or larger)
  3. Download it.
  4. The average download speed should be close to 2.5 MiB/s, but in actuality it’s only ever 1 - 1.5 MiB/s.

The output of your Nextcloud log in Admin > Logging:
[There is no log data from the day I ran these tests]

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'passwordsalt' => redacted,
  'secret' => redacted,
  'trusted_domains' =>
  array (
    0 => 'localhost',
    11 => redacted,
    1 => redacted,
    5 => 'nextcloudpi.local',
    7 => 'nextcloudpi',
    8 => 'nextcloudpi.lan',
    6 => redacted,
    3 => redacted,
    2 => redacted,
    4 => redacted,
  ),
  'datadirectory' => '/data/nextcloud/data',
  'dbtype' => 'mysql',
  'version' => '16.0.3.0',
  'overwrite.cli.url' => redacted,
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => redacted,
  'dbpassword' => redacted,
  'installed' => true,
  'instanceid' => redacted,
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' =>
  array (
    'host' => '/var/run/redis/redis.sock',
    'port' => 0,
    'timeout' => 0.0,
    'password' => redacted,
  ),
  'tempdirectory' => '/var/www/nextcloud/data/tmp',
  'mail_smtpmode' => 'sendmail',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_from_address' => 'admin',
  'mail_domain' => 'ownyourbits.com',
  'preview_max_x' => '2048',
  'preview_max_y' => '2048',
  'jpeg_quality' => '60',
  'overwriteprotocol' => 'https',
  'maintenance' => false,
  'data-fingerprint' => redacted,
);

The output of your Apache/nginx/system log in /var/log/____:

I’m not sure which log file this is referring to, and I’m reluctant to post any of my server’s apache logs publicly. If any data from these logs is needed to diagnose the issue, please let me know what specifically is needed and I’ll post it.

Could you clarify the expected speed? I saw you said both 20 Mbps and 2.5 Mbps.

What speed do you get on a wired client on your LAN?

@KarlF12 Sorry, I was getting mixed up between megabits per second and megabytes per second. I’ve edited my post to fix the errors.

In any case, my Nextcloud server’s upload speeds should be around 20 megabits per second.

What speed do you get on a client on the same network?

@KarlF12 As expected, the speed is much faster. Interestingly, though, it is still slower than other methods of transferring a file locally.

Downloading the file locally via Nextcloud’s web interface: 18.0 MiB/s
Transferring the same file locally via SFTP: 35 MiB/s
Transferring the same file locally via SMB: 85 MiB/s

Edit: I should add that the two systems I was testing with are both connected to the same LAN via gigabit ethernet

Not sure what to tell you there. I just downloaded a file from mine over my WiFi and got about the same speed from the web interface and SCP, about 72 Mbps.

Maybe something to do with the web server? Try putting your file in the web root and downloading it directly (not from Nextcloud interface).

@KarlF12 Good idea. I gave it a try and it went at the same slow speed. So it looks like the culprit may be apache. Not sure where to go next from there, though. Part of the reason I went with NextcloudPi was so that I wouldn’t have to maintain an apache webserver.

I haven’t used NextcloudPi. I actually have two layers of Apache in my setup; the one in the official Nextcloud Docker image, and then a reverse proxy I run on the host.

Okay, after some more experimentation I’ve found that it’s not apache and is in fact VirtualBox that’s causing the trouble. If I set up a simple python http server in the same docker container as Nextcloud, it has precisely the same bad upload speed.

So I need to do something to improve VirtualBox’s network performance. I know this is getting off-topic, but does anyone have any idea as to what I could do? I tried playing with the settings as suggested in this serverfault thread but it only managed to make the problem worse not better.

What are you using for the network mode? Bridge?

@KarlF12 I’m using NAT mode as that seems to be the only network mode that works with Docker. I’ve tried several of the different virtual network adapters that VirtualBox offers and they all seem to have similar speed issues.

I’ve tested other VirtualBox VMs using a different network mode and it doesn’t seem to have the same issues. I may try investigating ways around using NAT mode.

Quick follow up: for reasons that are not clear to me the problem has gone away on its own. If I have anything else to contribute I’ll post it here, but for the time being I think this matter as it relates specifically to Nextcloud is resolved.