Large uploads fail in both web UI and using desktop sync client

Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version (eg, 20.0.5): 28.0.3
Operating system and version (eg, Ubuntu 20.04): Arch Linux
Nextcloud + Nginx Proxy Manager running in Podman-Docker using these docker-compose files: Nextcloud Docker Setup ($3683489) · Snippets · GitLab

The issue you are facing:
Large uploads are failing with cryptic errors in both the web UI and desktop sync client. I have already increased the PHP upload limit and the Apache body limit by passing the following environment variables to my containers:

PHP_UPLOAD_LIMIT=16G
APACHE_BODY_LIMIT=0

I have also configured Nginx Proxy Manager to disable client_max_body_size and proxy_buffering:

Behavior in Web UI
When I upload a large file (> 3GB) via the Web UI, it will start and at a random point in time fail. In the browser logs, I see

[ERROR] uploader: Chunk 15 146800640 - 157286400 uploading failed

This message is followed by an object saying that the error was an AxiosError with code ERR_NETWORK.

An odd thing I’ve noticed is that sometimes the upload will go through, but then the next upload of large file will almost immediately fail.

Behavior with the desktop sync client
When using the desktop client, it will also randomly fail during the upload, but with the message:

Error while reading: error:0A0003FC:SSL routines::ssl/tls alert bad record mac

The desktop client will occasionally attempt to retry and it will sometimes automatically complete the upload. Other times, I have to manually tell it to sync now.

Is this the first time you’ve seen this error? (Y/N): No

Steps to replicate it:

  1. Setup a Nextcloud instance behind a Nginx Proxy Manager using the official Nextcloud docker images (non-AIO). Docker compose files are provided in this post for reference
  2. Increase PHP upload_max_limit, etc.
  3. Upload a large file (> 3GB) and notice that it fails in both the Web UI and desktop client

The output of your Nextcloud log in Admin > Logging:

BadRequest Expected filesize of 2721691545 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 1332592640 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.
BadRequest Expected filesize of 2721691545 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 1332592640 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.

And it just keeps repeating after every fail.

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' =>
  array (
    0 =>
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 =>
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'instanceid' => '<redacted>',
  'passwordsalt' => '<redacted>',
  'secret' => '<redacted>',
  'trusted_domains' =>
  array (
    0 => 'mydomain.me',
  ),
  'trusted_proxies' =>
  array (
  ),
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'mysql',
  'version' => '28.0.3.2',
  'overwrite.cli.url' => 'https://mydomain.me',
  'overwriteprotocol' => 'https',
  'dbname' => 'nextcloud',
  'dbhost' => 'db',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud',
  'dbpassword' => '<redacted>',
  'installed' => true,
  'mail_smtpmode' => 'smtp',
  'mail_smtphost' => 'smtp.gmail.com',
  'mail_sendmailmode' => 'smtp',
  'mail_from_address' => 'mydomain.me',
  'mail_domain' => 'gmail.com',
  'mail_smtpsecure' => 'tls',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtpauth' => 1,
  'mail_smtpport' => '587',
  'mail_smtpname' => 'mydomain.me',
  'mail_smtppassword' => '<redacted>',
  'maintenance' => false,
  'maintenance_window_start' => 3,
  'app_install_overwrite' =>
  array (
    0 => 'bruteforcesettings',
    1 => 'richdocumentscode',
  ),
  'loglevel' => 2,
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' =>
  array (
    'host' => 'redis',
    'password' => '',
    'port' => 6379,
  ),
  'default_phone_region' => 'US',
  'theme' => '',
);

The output of your Apache/nginx/system log in /var/log/____:

Because these are podman containers, the logs go into journalctl. Here’s the paste bin: Nextcloud upload fail logs ($3683516) · Snippets · GitLab

Thanks in advance.

Okay, I solved the issue with the Web UI upload. The problem was that I didn’t set proxy_request_buffering off in addition to proxy_buffering off in my NPM Proxy Host settings. I mistakenly thought the latter option disables buffering bidirectionally, but it only applies to responses from the proxied server, not requests.

After setting proxy_request_buffering off, web UI uploads proceed without a hitch.

However, I am still getting the issue where the desktop client will randomly stop uploading with ssl/tls alert bad record mac message. Dang it, so close!

EDIT: Web UI still occasionally fails to upload a chunk, but it is happening much less often than before. This is probably the same issue that’s happening to the desktop client.

Okay, I think the issue with syncing on desktop might be a bug in the Linux version of the desktop sync client itself or maybe some OS level bug I am not aware of. When I test syncing in Windows, it’s perfectly fine and able to sync large files repeatedly.