Web uploads stall at final chunk for files >3GB

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can. :heart:

The Basics

  • Nextcloud Server version (e.g., 29.x.x):
    • 30.0.8
  • Web server and version (e.g, Apache 2.4.25):
    • fpm
  • Reverse proxy and version _(e.g. nginx 1.27.2)
    • nginx 1.27.4
  • PHP version (e.g, 8.3):
    • 8.2.28
  • Installation method (e.g. AlO, NCP, Bare Metal/Archive, etc.)
    • Helm chart on k3s
  • Are you using CloudfIare, mod_security, or similar? (Yes / No)
    • No

Summary of the issue you are facing:

I’ve opened a Github issue with the details, but in summary:

Attempting to upload a file that is greater than 3GB through the web interface stalls and never completes at the final chunk. I can exec into the container and see the chunks being written to tmp, writing to the datadir/user/uploads, but upload never seems to complete and will hang with a message of “a few seconds left” indefinitely.
[…]
The following situations do work with files over 3GB:

  • Windows mounted network drive through WebDAV (provided the file is also under 4GB due to Windows restrictions)
  • curl --insecure -v -u admin -T <filename> "https://<hostname>/remote.php/dav/files/admin/test_4g.io"

Steps to replicate it (hint: details matter!):

  1. Install Nextcloud using the values.yaml seen in the Github issue.
  2. Upload a file >3GB through the web interface.

Log entries

Nextcloud

[no app in context] Error: Expected filesize of 10485760 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 10477568 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.
	PUT /remote.php/dav/uploads/admin/web-file-upload-219e6ce2ebc3f3f3/77
	from 192.168.0.46 by admin at Apr 10, 2025, 4:06:30 PM

Web Browser

No errors seen in the browser console or network tab. As seen below in the nginx logs, the last chunk will PUT with a response of 201, and then nothing happens after that. The site will simply say “a few seconds left” on the upload indefinitely.

Web server / Reverse Proxy

Snippet of nginx logs just before and after the last chunk

10.42.3.50 - - [10/Apr/2025:00:33:17 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-1dba1492b43a0d2a/303 HTTP/1.1" 201 0 "-" "<USER AGENT>" "192.168.0.38"
10.42.3.50 - - [10/Apr/2025:00:33:19 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-1dba1492b43a0d2a/304 HTTP/1.1" 201 0 "-" "<USER AGENT>" "192.168.0.38"
10.42.3.50 - - [10/Apr/2025:00:33:19 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-1dba1492b43a0d2a/306 HTTP/1.1" 201 0 "-" "<USER AGENT>" "192.168.0.38"
10.42.3.50 - - [10/Apr/2025:00:33:19 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-1dba1492b43a0d2a/305 HTTP/1.1" 201 0 "-" "<USER AGENT>" "192.168.0.38"
10.42.3.50 - - [10/Apr/2025:00:33:19 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-1dba1492b43a0d2a/308 HTTP/1.1" 201 0 "-" "<USER AGENT>" "192.168.0.38"
10.42.3.50 - - [10/Apr/2025:00:33:19 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-1dba1492b43a0d2a/307 HTTP/1.1" 201 0 "-" "<USER AGENT>" "192.168.0.38"
10.42.3.50 - - [10/Apr/2025:00:33:21 +0000] "GET /apps/logreader/api/poll?lastReqId=V7UTyLhFLRdBCWwHDtZx HTTP/1.1" 200 22 "-" "<USER AGENT>" "192.168.0.38"
10.42.3.50 - - [10/Apr/2025:00:33:22 +0000] "GET /apps/logreader/api/poll?lastReqId=V7UTyLhFLRdBCWwHDtZx HTTP/1.1" 200 22 "-" "<USER AGENT>" "192.168.0.38"
10.42.3.50 - - [10/Apr/2025:00:33:22 +0000] "POST /apps/text/session/57/sync HTTP/1.1" 200 469 "-" "<USER AGENT>" "192.168.0.38"
10.42.3.50 - - [10/Apr/2025:00:33:24 +0000] "GET /apps/logreader/api/poll?lastReqId=V7UTyLhFLRdBCWwHDtZx HTTP/1.1" 200 22 "-" "<USER AGENT>" "192.168.0.38"
10.42.3.50 - - [10/Apr/2025:00:33:27 +0000] "POST /apps/text/session/57/sync HTTP/1.1" 200 469 "-" "<USER AGENT>" "192.168.0.38"
10.42.3.50 - - [10/Apr/2025:00:33:29 +0000] "POST /apps/text/session/57/push HTTP/1.1" 200 22 "-" "<USER AGENT>" "192.168.0.38"
10.42.3.50 - - [10/Apr/2025:00:33:31 +0000] "GET /apps/logreader/api/poll?lastReqId=V7UTyLhFLRdBCWwHDtZx HTTP/1.1" 200 22 "-" "<USER AGENT>" "192.168.0.38"

Configuration

Nextcloud

{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "forwarded_for_headers": [
            "HTTP_X_FORWARDED_FOR"
        ],
        "check_data_directory_permissions": false,
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "overwriteprotocol": "https",
        "upgrade.disable-web": true,
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "hostname"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "sqlite3",
        "version": "30.0.8.1",
        "overwrite.cli.url": "https:\/\/localhost",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***"
    }
}

Apps

Default apps installed with the Helm installation:

Enabled:
  - activity: 3.0.0
  - app_api: 4.0.6
  - bruteforcesettings: 3.0.0
  - circles: 30.0.0
  - cloud_federation_api: 1.13.0
  - comments: 1.20.1
  - contactsinteraction: 1.11.0
  - dashboard: 7.10.0
  - dav: 1.31.1
  - federatedfilesharing: 1.20.0
  - federation: 1.20.0
  - files: 2.2.0
  - files_downloadlimit: 3.0.0
  - files_pdfviewer: 3.0.0
  - files_reminders: 1.3.0
  - files_sharing: 1.22.0
  - files_trashbin: 1.20.1
  - files_versions: 1.23.0
  - firstrunwizard: 3.0.0
  - logreader: 3.0.0
  - lookup_server_connector: 1.18.0
  - nextcloud_announcements: 2.0.0
  - notifications: 3.0.0
  - oauth2: 1.18.1
  - password_policy: 2.0.0
  - photos: 3.0.2
  - privacy: 2.0.0
  - provisioning_api: 1.20.0
  - recommendations: 3.0.0
  - related_resources: 1.5.0
  - serverinfo: 2.0.0
  - settings: 1.13.0
  - sharebymail: 1.20.0
  - support: 2.0.0
  - survey_client: 2.0.0
  - systemtags: 1.20.0
  - text: 4.1.0
  - theming: 2.6.0
  - twofactor_backupcodes: 1.19.0
  - updatenotification: 1.20.0
  - user_status: 1.10.0
  - viewer: 3.0.0
  - weather_status: 1.10.0
  - webhook_listeners: 1.1.0-dev
  - workflowengine: 2.12.0
Disabled:
  - admin_audit: 1.20.0
  - encryption: 2.18.0
  - files_external: 1.22.0
  - suspicious_login: 8.0.0
  - twofactor_nextcloud_notification: 4.0.0
  - twofactor_totp: 12.0.0-dev
  - user_ldap: 1.21.0

After four days of spending my entire evening on this, I believe I’ve come to a solution. It looks like the default chunking of 10MB for Nextcloud 30 (which is what the Helm chart defaults to) was causing issues. I know Nextcloud 31 changes this default to 100MB, so after changing that setting using php occ config:app:set files max_chunk_size --value 104857600, it seemed to have fixed that problem.

It did however introduce a new problem where the Nginx container was receiving a kill signal and restarting in the middle of uploads. After looking into it, it looks like the liveness probe was failing, likely due to the server loads not being able to respond to the status.php requests in time, so I had to increase the timeout from 10 seconds (default). 30 seconds seems to work for me but you may not have to do this if you run into this issue depending on your hardware.

Ultimately, I’ve added the following to my values.yaml:

lifecycle:
  # Set max_chunk_size to 100MB
  postStartCommand: ['php', 'occ', 'config:app:set', 'files', 'max_chunk_size', '--value', '104857600']

livenessProbe:
  periodSeconds: 30
  timeoutSeconds: 30
readinessProbe:
  periodSeconds: 30
  timeoutSeconds: 30

You don’t have to do the lifecycle option if you just set it once, but I figured having it declared like this would mean I wouldn’t have to remember it and makes it more reproducible.

This topic was automatically closed 8 days after the last reply. New replies are no longer allowed.