Nextcloud fails to upload large files

Nextcloud version: 23.0.0
Operating system and version: Synology DSM 7.x
Apache version: 2.4.52 (Debian)
PHP version : 8.0.14

The issue you are facing:

Uploads are very slow, even on local LAN. But far worse is after uploading NC attempts to assemble some parts together and fails with a 504. Sometimes the file appears on refresh, other times it doesn’t. I set up this Nextcloud server to allow my non-techie friends to be able to upload and share with me large files. As such it needs to be quick and reliable. But it’s slow and unreliable.

Is this the first time you’ve seen this error? N

Steps to replicate it:

  1. Log into Nextcloud
  2. Attempt to upload a large file (2 Gig or greater)
  3. Wait, wait, wait - fails with 504

The output of your Nextcloud log in Admin > Logging:


The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'ocr9s4iyfz5x',
  'passwordsalt' => 'redacted',
  'secret' => 'redacted+',
  'trusted_domains' => 
  array (
    0 => 'redacted',
    1 => 'redacted',
    2 => 'jupiter',
    3 => '192.168.0.251',
  ),
  'default_phone_region' => 'US',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'mysql',
  'version' => '23.0.0.10',
  'overwrite.cli.url' => 'redacted',
  'overwritehost' => 'redacted',
  'overwriteprotocol' => 'https',
  'dbname' => 'nextcloud',
  'dbhost' => '192.168.0.251',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud',
  'dbpassword' => 'nextcloud',
  'installed' => true,
  'mail_from_address' => 'Andrew',
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'mail_domain' => 'DeFaria.com',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'defaria.com',
  'mail_smtpport' => '',
  'mail_smtpname' => 'andrew',
  'mail_smtppassword' => 'redacted',
  'maintenance' => false,
  'enable_previews' => true,
  'enabledPreviewProviders' => 
  array (
    0 => 'OC\\Preview\\Movie',
    1 => 'OC\\Preview\\PNG',
    2 => 'OC\\Preview\\JPEG',
    3 => 'OC\\Preview\\GIF',
    4 => 'OC\\Preview\\BMP',
    5 => 'OC\\Preview\\XBitmap',
    6 => 'OC\\Preview\\MP3',
    7 => 'OC\\Preview\\MP4',
    8 => 'OC\\Preview\\TXT',
    9 => 'OC\\Preview\\MarkDown',
    10 => 'OC\\Preview\\PDF',
  ),
);

The output of your Apache/nginx/system log in /var/log/____:

PASTE HERE

I dont know much of DSM but,
I think it’s your php.ini value witch is wrong…

My /etc/php/8.0/apache2/php.ini

; Maximum allowed size for uploaded files.
; http://php.net/upload-max-filesize
upload_max_filesize = 999GB

; Maximum number of files that can be uploaded via a single request
max_file_uploads = 20

DSM is Synology’s DiskStation Manager and I believe it’s derived from Debian.

I running Nextcloud in a Docker container. As such I don’t have an /etc/php at all. Under Docker, they seem to put php under /usr/local. Looking at /usr/local I find only:

root@nextcloud:cd /usr/local
root@nextcloud:/usr/local# find . -type f -exec grep upload_max_filesize {} +
grep: ./bin/php: binary file matches
grep: ./bin/phpdbg: binary file matches
./etc/php/conf.d/nextcloud.ini:upload_max_filesize=${PHP_UPLOAD_LIMIT}
./etc/php/php.ini-development:upload_max_filesize = 2M
./etc/php/php.ini-production:upload_max_filesize = 2M
./include/php/main/php_globals.h:	zend_long upload_max_filesize;
grep: ./lib/php/extensions/no-debug-non-zts-20200930/imagick.so: binary file matches
root@nextcloud:/usr/local# 

Uploading big files > 512MB says

The default maximum file size for uploads is 512MB". It seems to allude to setting values in php.ini but doesn’t tell you where that php.ini is.

So I created a phpinfo.php in the Docker container and found that my php.ini should be:

Configuration File (php.ini) Path	/usr/local/etc/php

Yet there is no /usr/local/etc/php/php.ini. There are php.ini-development and php.ini-production but as we can see above the upload_max_filesize in both of them are a poultry 2M. Are you saying go into the Docker container and set the parms as you indicate by creating php.ini in the Docker container? If so then what happens when the Docker container updates? Reapply the tweaks?

And there’s also this on that same page:

The upload_max_filesize and post_max_size settings may not apply to file uploads through WebDAV single file PUT requests or Chunked file uploads For those, PHP and webserver timeouts are the limiting factor on the upload size.

Nextcloud should really provide a UI in the settings interface to configure this stuff! Uploading to store and as they like to say have a “Safe place for all your data”, one would think that seamless and fast uploads of large data files would be a core piece of functionality.

Nextcloud will probably not provide for this, since they are not Nextcloud configurations, but PHP configurations and partially web server configurations. But I agree that many of the configurations mentioned are not optimal for Nextcloud.

Care to take a guess as to where I’m supposed to make these modifications and how do I get them to stick in the context of a Docker container?

OK this still isn’t working for me. Here’s where I’m at. Bear in mind I’m running NC in a docker container on Synology (a Debian derived OS).

I have managed to map a config file into the Docker container that NC seems to be using. My goal is to set the following:

; Must set post_max_size to be larger than upload_max_filesize
post_max_size = 5GB

; Maximum allowed size for uploaded files.
; http://php.net/upload-max-filesize
upload_max_filesize = 4GB

; Test to see if I can set a variable for PHP through this php.ini file
max_execution_time = 60

This shows up in the docker container:

$ docker exec -it nextcloud cat /usr/local/etc/php/php.ini
; Must set post_max_size to be larger than upload_max_filesize
post_max_size = 5GB

; Maximum allowed size for uploaded files.
; http://php.net/upload-max-filesize
upload_max_filesize = 4GB

; Test to see if I can set a variable for PHP through this php.ini file
max_execution_time = 60
$

And I have a phpinfo.php file set up. Looking at the output I see that php.ini seems to be correct:

2022-04-01@16:51:36

And my max_execution_time has taken effect (default is 30, not 60) (Sorry, image uploading not working, you’re gonna have to trust me on this), However, neither post_max_size nor upload_max_filesize have been changed. They still say 512 Meg whereas I was trying to set them to 5 and 4 GB.

How do you set these values for NC in a docker container?

More info!

Turns out there is a /usr/local/etc/php/conf.d/nextcloud.ini that contains

memory_limit=${PHP_MEMORY_LIMIT}
upload_max_filesize=${PHP_UPLOAD_LIMIT}
post_max_size=${PHP_UPLOAD_LIMIT}

and the docker container is supposed to set these environment variables.

Even with these settings uploads still fail with “Error when assembling chunks, status code 504” which seems to indicate timeout yet refreshing the page shows the file was uploaded apparently in full.

Oh I added max_execution_time and set it to 1 hour (3600). Didn’t help.

I was hoping that updating to NextCloud 24 would address these problems. It says it’s a lot quicker. I’m not seeing it. NC tries to add all kinds of functionality but the underlying filesystem is just way quicker! For example, copying a 2.2 gig file from my desktop to my NAS (which is also where NC is running) mounted via NFS takes 10 minutes and 19 seconds of wall time. However, dragging and dropping the same file into NC going through WebDAV takes <just past the 11-minute mark and it’s bouncing between 15 - 30 minutes…> an additional 11 minutes - twice as long!

Plus the indicator is all over the map! It’ll say 1 hour and then a second or two later that drops to 20 minutes, climbs up to 36 minutes, and back down. It’s sampling way too quickly and reporting all kinds of incorrect times.

Finally, the copy of this file into NC results in the same old error “Error when assembling chunks, status code 504” and yet the file appears intact with a simple refresh.