All my Linux and Windows Desktop clients can upload 9 GB files but downloads fail in the end

Server info :
Nextcloud Server : 24.0.6
Operating system and version : Buster on Raspberry Pi 4
Apache : Server version: Apache/2.4.38 (Raspbian)
PHP version : 8.0.26

Clients info:
Windows 10 with client 3.5.4 or 3.7.3
Ubuntu Ubuntu 20.04.5 LTS with clients 3.5.4 or Nextcloud-3.7.3-x86_64.AppImage

Clients access the server by IP address 192.168.2.4 when they are on the LAN or by my public hostname when they are on the internet.
For the access by IP address a self signed certificate is in use and for the access by public hostname I use a Let’s Encrypt certificate.

The clients on my LAN are connected to a 100Mb switch and I have a 100 Mb internet connection.

The issue I am facing:

I am facing the issue below when I run more than 1 client but also with 1 client up only.

The clients can upload large files like 9 GB without any issue but can not download them.
Downloads start but fail after some GB.

Red Cross in Nexcloud Settings screen with message ‘: Connection closed>’

In the main dialog pop up screen I see messages like :

‘Connection closed ( skipped due to earlier error, trying again in 8 second(s))’
and sometimes also :
‘ Server returned wrong content-range’

Not after 8 seconds but after some time download may continue but in the end it will fail anyway. During download there is a file in the folder where the file should appear. The name start with a . and has a random extension.
When I hold my mouse over the progress bar I see the name of the file and a download speed around 8-9 MB/s which is good for my 100 Mb network.
The file with the . disappears when the download fails.
After some time the client will start again, and again and again until I delete the file either on the client that sent it or via the web interface.
When I delete it from the web interface the file also disappears from the client that sent it in the first place. When I restore it via the web interface I have the issue on all the clients that try to download the file.

I have no encryption on my Nextcloud server so I can access the files directly via the file system.

On Ubuntu clients I can use rsync to copy the files back to the clients and on Windows I can use FileZilla to copy the files back.
When I have them back on the client the files are oke which means that the initial uploads from the clients which ran without issues are fine.

Steps to replicate it:

I can create any big file on WIndows or Ubuntu some way by for example adding some video VOB files together and put that in a Nexcloud folder.
It gets uploaded but clients will not succeed in downloading it.

In the web interface on the server there is no logging about the failing downloads.
Logging works because I see messages in the logging when I use impersonate to go to any of the 4 users I have in my Nextcloud to delete the big files and clean up the trash cans.

The output of your Nextcloud log in Admin > Logging:

I had no message so I forced some by impersonating my test user

Warning	impersonate	Changing to user test	
2023-03-08T14:40:34+0100
Warning	impersonate	User Jack trying to impersonate user test	
2023-03-08T14:40:34+0100

Info on /etc/php/8.0/apache2/php.ini
( find all lines not starting with a “;” and skip all empty lines :
grep /etc/php/8.0/apache2/php.ini -v -e “^;” | sed -e /^$/d )

[PHP]
engine = On
short_open_tag = Off
precision = 14
output_buffering = 4096
zlib.output_compression = Off
implicit_flush = Off
unserialize_callback_func =
serialize_precision = -1
disable_functions = pcntl_alarm,pcntl_fork,pcntl_waitpid,pcntl_wait,pcntl_wifexited,pcntl_wifstopped,pcntl_wifsignaled,pcntl_wifcontinued,pcntl_wexitstatus,pcntl_wtermsig,pcntl_wstopsig,pcntl_signal,pcntl_signal_get_handler,pcntl_signal_dispatch,pcntl_get_last_error,pcntl_strerror,pcntl_sigprocmask,pcntl_sigwaitinfo,pcntl_sigtimedwait,pcntl_exec,pcntl_getpriority,pcntl_setpriority,pcntl_async_signals,pcntl_unshare,
disable_classes =
zend.enable_gc = On
zend.exception_ignore_args = On
zend.exception_string_param_max_len = 0
expose_php = Off
max_execution_time = 30
max_input_time = 60
memory_limit = 512M
error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT
display_errors = Off
display_startup_errors = Off
log_errors = On
log_errors_max_len = 1024
ignore_repeated_errors = Off
ignore_repeated_source = Off
report_memleaks = On
variables_order = "GPCS"
request_order = "GP"
register_argc_argv = Off
auto_globals_jit = On
post_max_size = 1024M
auto_prepend_file =
auto_append_file =
default_mimetype = "text/html"
default_charset = "UTF-8"
doc_root =
user_dir =
enable_dl = Off
file_uploads = On
upload_max_filesize = 20480M
max_file_uploads = 20
allow_url_fopen = On
allow_url_include = Off
default_socket_timeout = 60
[CLI Server]
cli_server.color = On
[Date]
[filter]
[iconv]
[imap]
[intl]
[sqlite3]
[Pcre]
[Pdo]
[Pdo_mysql]
pdo_mysql.default_socket=
[Phar]
[mail function]
SMTP = localhost
smtp_port = 25
mail.add_x_header = Off
[ODBC]
odbc.allow_persistent = On
odbc.check_persistent = On
odbc.max_persistent = -1
odbc.max_links = -1
odbc.defaultlrl = 4096
odbc.defaultbinmode = 1
[MySQLi]
mysqli.max_persistent = -1
mysqli.allow_persistent = On
mysqli.max_links = -1
mysqli.default_port = 3306
mysqli.default_socket =
mysqli.default_host =
mysqli.default_user =
mysqli.default_pw =
mysqli.reconnect = Off
[mysqlnd]
mysqlnd.collect_statistics = On
mysqlnd.collect_memory_statistics = Off
[OCI8]
[PostgreSQL]
pgsql.allow_persistent = On
pgsql.auto_reset_persistent = Off
pgsql.max_persistent = -1
pgsql.max_links = -1
pgsql.ignore_notice = 0
pgsql.log_notice = 0
[bcmath]
bcmath.scale = 0
[browscap]
[Session]
session.save_handler = files
session.use_strict_mode = 0
session.use_cookies = 1
session.use_only_cookies = 1
session.name = PHPSESSID
session.auto_start = 0
session.cookie_lifetime = 0
session.cookie_path = /
session.cookie_domain =
session.cookie_httponly =
session.cookie_samesite =
session.serialize_handler = php
session.gc_probability = 0
session.gc_divisor = 1000
session.gc_maxlifetime = 1440
session.referer_check =
session.cache_limiter = nocache
session.cache_expire = 180
session.use_trans_sid = 0
session.sid_length = 26
session.trans_sid_tags = "a=href,area=href,frame=src,form="
session.sid_bits_per_character = 5
[Assertion]
zend.assertions = -1
[COM]
[mbstring]
[gd]
[exif]
[Tidy]
tidy.clean_output = Off
[soap]
soap.wsdl_cache_enabled=1
soap.wsdl_cache_dir="/tmp"
soap.wsdl_cache_ttl=86400
soap.wsdl_cache_limit = 5
[sysvshm]
[ldap]
ldap.max_links = -1
[dba]
[opcache]
[curl]
[openssl]
[ffi]

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => '............',
  'passwordsalt' => '................',
  'secret' => '............',
  'trusted_domains' => 
  array (
    0 => 'localhost',
    1 => '192.168.2.4',
    2 => 'firstpublicname.thehomeserver.net',
    3 => 'secondpublic.onthewifi.com',
  ),
  'datadirectory' => '/mnt/Share0/NextCloud',
  'dbtype' => 'mysql',
  'version' => '24.0.6.1',
  'overwrite.cli.url' => 'http://localhost/nextcloud',
  'dbname' => 'nextclouddb',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => '..........',
  'dbpassword' => '...........',
  'installed' => true,
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'ssl',
  'mail_sendmailmode' => 'smtp',
  'mail_from_address' => '.........',
  'mail_domain' => 'gmail.com',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'smtp.gmail.com',
  'mail_smtpport' => '465',
  'mail_smtpname' => '...........',
  'mail_smtppassword' => '...........',
  'default_phone_region' => 'NL',
  'overwriteprotocol' => 'https',
  'check_for_working_wellknown_setup' => false,
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 2,
  'filelocking.enabled' => true,
  'updatechecker' => true,
  'updater.release.channel' => 'stable',
);

I hope someone can help me so I can add another solution to my list which I keep since version 22.0.0 on GitHub - JackV2020/Nextcloud-Raspberry-Apache-Mariadb: Installed Nextcloud 20.0.4 several upgrades to 24.0.6, kept status green so solved run-time issues.

Many thanks in advance

Jack

Hey Jack,

Here is a quick article on the large file downloads with Apache.

Let me know if that helps.

Sebastian

Hi Sebastian,

I didn’t find a solution that solved the issue and decided for a different approach…
Domoticz prevented me to move to Bullseye because it did not support Bullseye until the last Domoticz version which is available since the beginning of this year. And because that is supported now I used the last days to wipe my Raspberry Pi uSD card, install Bullseye, Domoticz and more and at last Nextcloud 25.0.4 and it is running fine, after some necessary tweeking in config.php.
Thing I really really needed were:
to solve https issues:
‘overwriteprotocol’ => ‘https’,
To get rid of lots, lots, I mean lots of ‘Network error 99’ messages in the clients:
‘bulkupload.enabled’ => false,
Did also some other things to clean up the messages Administration settings Overview screen.
The last thing to tackle is APCu memory caching but this is not critical.
I am able to upload, download, delete and restore files of at least 9GB so I am very happy now.
My php.ini has the next entries, like I put in for my previous Nexcloud version:
post_max_size = 20480M
upload_max_filesize = 20480M
So it should work for files up to 20GB

Kind regards,

Jack

Hi Jack,

I’m glad you solved your issue!

I have one comment…

I think your post_max_size and upload_max_filesize should be about 35% larger then the largest file you want to upload. File sizes on a drive and while being uploaded via HTTP are totally different. When you upload a 1GB (on disk) file you are actually uploading about 1.35GB, the reason is that, the binary file is encoded to Base64 (Basically, Readable Ascii Text), which makes the file grow in size.

Also according to the php documentation your post_max_size should be larger then upload_max_filesize to allow for HTTP/Post overhead.

Sebastian

1 Like

Hi Sebastian,

a late reaction. My excuse is that I am just back from a small vacation, 2 days in a spa which we needed so much :wink:. Today I made the changes for the cache and read your comment and increased post_max_size. Many thanks for that. I use base64 encoding and decoding myself in Domoticz python plugins so I know what it is but I was not aware that it was also used for the file transfers by NextCloud.

Kind regards,

Jack

I feel yah, I need a vacation myself!

Yes, if its HTTP based its Base64 encoded. The file uploads through the WebUI are straight HTTP posts, and though the desktop client are WebDAV which is an extension of HTTP protocol.

Have a great day.
Sebastian