Can't get download speeds above 40MB/s. Single CPU core at 100%

Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version (eg, 20.0.5): 21.0.1
Operating system and version (eg, Ubuntu 20.04): Ubuntu 20.04.2 LTS
Apache or nginx version (eg, Apache 2.4.25): 2.4.41
PHP version (eg, 7.4): 7.4

The issue you are facing:

When connected to my NC instance locally, I cannot seem to get above 40MB/s download speeds without a CPU core being completely pegged at 100%. 40MB/s is not terribly slow by any means, but I know my setup can achieve much higher I/O.

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

When using the web client, if i try to download a file of around 2GB my speed is capped around 40MB/s. If i download multiple files at the same time, I can achieve total throughput of around 60MB/s but each file being downloaded completely maxes out a CPU core.

The output of your Nextcloud log in Admin > Logging:

No Errors.

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => ‘*******’,
  'passwordsalt' => ‘*******’,
  'secret' => ‘*********’,
  'trusted_proxies' => ‘*******,
  'trusted_domains' => 
  array (
    0 => ‘*******’,
    1 => ‘*******’,
  ),
  'datadirectory' => '/nextcloud-data',
  'dbtype' => 'mysql',
  'version' => '21.0.1.1',
  'overwrite.cli.url' => ‘*******’,
  'dbname' => 'nextclouddb',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => ‘********’,
  'dbpassword' => ‘*******’,
  'installed' => true,
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'mail_smtpauth' => 1,
  'mail_smtpname' => ‘*******’,
  'mail_smtppassword' => ‘******’,
  'allow_local_remote_servers' => true,
  'mail_smtpsecure' => 'ssl',
  'mail_from_address' => ‘******’,
  'mail_domain' => ‘*******’,
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtphost' => ‘*******’,
  'mail_smtpport' => '465',
  'maintenance' => false,
  'enable_previews' => true,
  'enabledPreviewProviders' => 
  array (
    0 => 'OC\\Preview\\PNG',
    1 => 'OC\\Preview\\Image',
    2 => 'OC\\Preview\\TIFF',
    3 => 'OC\\Preview\\MP3',
    4 => 'OC\\Preview\\Movie',
    5 => 'OC\\Preview\\MKV',
    6 => 'OC\\Preview\\MP4',
    7 => 'OC\\Preview\\AVI',
    8 => 'OC\\Preview\\BMP',
    9 => 'OC\\Preview\\HEIC',
    10 => 'OC\\Preview\\GIF',
    11 => 'OC\\Preview\\JPEG',
    12 => 'OC\\Preview\\3GP',
  ),
  'htaccess.RewriteBase' => '/',
  'default_phone_region' => 'US',
  'theme' => '',
  'loglevel' => 0,
  'memcache.local' => '\OC\Memcache\APCu',
  'updater.release.channel' => 'stable',
  'twofactor_enforced' => 'false',
  'twofactor_enforced_groups' => 
  array (
  ),
  'twofactor_enforced_excluded_groups' => 
  array (
  ),
);

The output of your Apache/nginx/system log in /var/log/____:

No Errors.

You don’t have redis installed, that will improve the speed if you have many small files since all the locking will be handled by redis instead of the database.
If you have many clients, the high performance backend will be interesting as well: GitHub - nextcloud/notify_push: Update notifications for nextcloud clients

However, in your case, 40 MB/s is not so bad. The 100% CPU is maxing out some operation, you need to know what it is, so even the options I mentioned might not help at all and are just if you want more file and users. If it’s related to database operations (also check the io-wait…), it could help.

Thank so much for trying to help!

Been quite a while and I had mostly given up on trying to improve this but i’m currently looking into it again. I had tried playing with enabling redis in the past but had not noticed any change in performance in relation to this issue.

40MB/s is definitely decent throughput but its more of a pain just knowing that I have the potential to get more out of it.

I have attached a screenshot from htop which shows that its apache2 being completely maxed out while the file download is in progress. As I had mentioned previously, this generally occurs right around the 40MB/s point.Screen Shot 2021-09-18 at 6.48.28 PM

As you said, the 40MB/s are already a decent speed and further improvements are harder and harder to find. Probably if you go through more of the code (e.g. with profiling tools), you can find further bottlenecks.

If you use workflow, antivirus apps etc. that can also add some delay. But that is all just guessing like you do yourself.

That definitely seems to be the case. Perhaps I am being too greedy. Is this about what I can expect performance-wise running Nextcloud? I have little to compare it to as far as what other users are seeing.