High memory usage client side when uploading large files in browser

ℹ️ Support
, , , , ,
1
System info

Nextcloud version 28.0.1
Ubuntu 22.04.3 LTS
Apache/2.4.54
PHP 8.1.2

The issue you are facing:
Clients memory usage rises to many GB during a large upload.
I had upload chunking turned off (using occ command) but have been forced to turn it on since it seems now all the currently uploading chunks are stored in memory.

Tested on

Mac mini 2020 M1, 16GB, Ventura 13.6.3.
Firefox 120.0

Is this the first time you’ve seen this error?
Yes… that doesn’t mean it wasn’t here before but certainly it wasn’t preventing non-chunked uploads before.

Steps to replicate it:

  1. Upload a large file (e.g. 4GB) using the web interface
  2. Monitor the memory usage of the browser.

Steps to make the upload fail:

  1. Turn off upload chunking using:
    sudo -u www-data php occ config:app:set files max_chunk_size --value 0
  2. Upload a large file (4GB) using the web interface.
  3. The progress bar never starts and the memory usage goes up to 4GB (the limit of Javascript, the browser clears the memory and the upload fails silently. The HTTP PUT request is never sent/started.

The output of your Nextcloud log in Admin > Logging:
Since the failure happens client side, nothing appears in the server logs.

The output of your config.php file:

'dbtype' => 'mysql',
'version' => '28.0.1.1',
'installed' => true,
'memcache.local' => '\\OC\\Memcache\\APCu',
  'filelocking.enabled' => 'true',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => '/var/run/redis/redis-server.sock',
'maintenance' => false,

The output of your Apache/nginx/system log in /var/log/____:

No relevant errors logged
2

Having chunking turned on is a big problem: large files (e.g. 60GB) can not be uploaded since the reassembling of the chunks takes too long and can time out.

3

Something was reported today that may be related:

4

That does indeed look related, thank you!