Help in figuring out why large files are uploaded and then deleted immediately

Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version (eg, 20.0.5): 28.0.5
Operating system and version (eg, Ubuntu 20.04): Rocky Linux 9.2
Apache or nginx version (eg, Apache 2.4.25): 2.4.57
PHP version (eg, 7.4): 8.2.15

The issue you are facing:

Is this the first time you’ve seen this error? (Y/N): N

Steps to replicate it:

  1. Upload 10GB file
  2. File does not save, and is deleted immediatly

The output of your Nextcloud log in Admin > Logging:

Error	webdav	
ServiceUnavailable Could not open file
May 24, 2024, 1:45:59 AM 	

Error	no app in context	
ServiceUnavailable Could not open file
May 24, 2024, 1:45:58 AM 	

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'xxxx',
  'passwordsalt' => 'yyyy',
  'secret' => 'zzz',
  'trusted_domains' => 
  array (
    0 => 'nextcloud',
  ),
  'datadirectory' => '/mnt/cloud/diskspace',
  'dbtype' => 'mysql',
  'version' => '28.0.5.1',
  'overwrite.cli.url' => 'https://mysite.com',
  'dbname' => 'dbName',
  'dbhost' => '0.0.0.0',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextUser',
  'dbpassword' => '1234',
  'installed' => true,
  'mail_smtpmode' => 'sendmail',
  'mail_smtphost' => 'mail.server.com',
  'mail_sendmailmode' => 'smtp',
  'mail_smtpport' => '581',
  'mail_smtpauth' => 1,
  'mail_from_address' => 'administration',
  'mail_domain' => 'nextcloud.com',
  'mail_smtpname' => 'administration@mysite.com',
  'mail_smtppassword' => '4321',
  'updater.secret' => 'rgiorgaond',
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 2,
);

The output of your Apache/nginx/system log in /var/log/____:

127.0.0.1 - - [24/May/2024:01:35:21 -0400] "PUT /xxx/remote.php/dav/uploads/MiniFireDragon/web-file-upload-18a17fb3bb8dea99/1008 HTTP/2.0" 201 - "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"
127.0.0.1 - - [24/May/2024:01:35:22 -0400] "PUT /xxx/remote.php/dav/uploads/MiniFireDragon/web-file-upload-18a17fb3bb8dea99/1010 HTTP/2.0" 201 - "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"
127.0.0.1 - - [24/May/2024:01:35:22 -0400] "PUT /xxx/remote.php/dav/uploads/MiniFireDragon/web-file-upload-18a17fb3bb8dea99/1009 HTTP/2.0" 201 - "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"
127.0.0.1 - - [24/May/2024:01:35:23 -0400] "GET /xxx/index.php/apps/files/api/v1/stats HTTP/2.0" 200 166 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"
127.0.0.1 - - [24/May/2024:01:35:27 -0400] "GET /xxx/ocs/v2.php/apps/notifications/api/v2/notifications HTTP/2.0" 304 - "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"
127.0.0.1 - - [24/May/2024:01:35:57 -0400] "GET /xxx/ocs/v2.php/apps/notifications/api/v2/notifications HTTP/2.0" 304 - "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"
127.0.0.1 - - [24/May/2024:01:36:20 -0400] "PUT /xxx/ocs/v2.php/apps/user_status/api/v1/heartbeat?format=json HTTP/2.0" 200 157 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"
127.0.0.1 - - [24/May/2024:01:35:22 -0400] "MOVE /xxx/remote.php/dav/uploads/MiniFireDragon/web-file-upload-18a17fb3bb8dea99/.file HTTP/2.0" 504 247 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"
71.185.229.66 - - [24/May/2024:01:36:23 -0400] "GET /xxx/index.php/apps/files/api/v1/stats HTTP/2.0" 200 165 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"
127.0.0.1 - - [24/May/2024:01:36:23 -0400] "DELETE /xxx/remote.php/dav/uploads/MiniFireDragon/web-file-upload-18a17fb3bb8dea99 HTTP/2.0" 204 - "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0"

PASTE HERE


Output errors in nextcloud.log in /var/www/ or as admin user in top right menu, filtering for errors. Use a pastebin service if necessary.

PASTE HERE

Additional Information:
I can upload large files (above 5gb) but when I get to 10gb, it just stops working.

Have you adjusted the PHP timeouts for large file uploads? Uploading big files > 512MB — Nextcloud latest Administration Manual latest documentation

ServiceUnavailable Could not open file

Can you please provide the full raw log entry?

Based on what you did provide, there is a timeout, but I believe there are several possible code paths for getting there.

There also should / is likely a PHP timeout or something in your Apache error log. So please check that.

I also encourage you to set-up memory caching: Server tuning — Nextcloud latest Administration Manual latest documentation

Is your client really on the same host (127.0.0.1) or do you have a reverse proxy misconfiguration?

As mentioned, I can upload over 5gbs, I tested recently on a 8gb file and that failed.

No, it is a public IP address, I just edited the file to obscure the actual IP Address

There are 2 entries:


{"reqId":"ZlAtvu4zmTagJt4mIT2P9wAAgAk","level":3,"time":"2024-05-24T06:04:43+00:00","remoteAddr":"71.185.229.66","user":"MiniFireDragon","app":"webdav","method":"MOVE","url":"/cloud/remote.php/dav/uploads/MiniFireDragon/web-file-upload-da59fb99cc08a35b/.file","message":"Could not open file","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0","version":"28.0.5.1","exception":{"Exception":"Sabre\\DAV\\Exception\\ServiceUnavailable","Message":"Could not open file","Code":0,"Trace":[{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Upload/AssemblyStream.php","line":294,"function":"get","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/kcloud/apps/dav/lib/Upload/AssemblyStream.php","line":164,"function":"getStream","class":"OCA\\DAV\\Upload\\AssemblyStream","type":"->"},{"function":"stream_read","class":"OCA\\DAV\\Upload\\AssemblyStream","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/icewind/streams/src/Wrapper.php","line":55,"function":"fread"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/icewind/streams/src/CallbackWrapper.php","line":96,"function":"stream_read","class":"Icewind\\Streams\\Wrapper","type":"->"},{"function":"stream_read","class":"Icewind\\Streams\\CallbackWrapper","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/lib/private/Files/Storage/Local.php","line":339,"function":"file_put_contents"},{"file":"/home/cloud/domains/cloud/public_html/cloud/lib/private/Files/Storage/Local.php","line":650,"function":"file_put_contents","class":"OC\\Files\\Storage\\Local","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":653,"function":"writeStream","class":"OC\\Files\\Storage\\Local","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":653,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Connector/Sabre/File.php","line":250,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Connector/Sabre/Directory.php","line":148,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":325,"function":"createFile","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":151,"function":"copyNode","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":181,"function":"copy","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Upload/ChunkingPlugin.php","line":94,"function":"move","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Upload/ChunkingPlugin.php","line":76,"function":"performMove","class":"OCA\\DAV\\Upload\\ChunkingPlugin","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"beforeMove","class":"OCA\\DAV\\Upload\\ChunkingPlugin","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":603,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpMove","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Server.php","line":373,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/remote.php","line":172,"args":["/home/cloud/domains/cloud/public_html/cloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Connector/Sabre/File.php","Line":505,"message":"Could not open file","exception":[],"CustomMessage":"Could not open file"},"id":"66542e3e2367c"}

{"reqId":"ZlAtvu4zmTagJt4mIT2P9wAAgAk","level":3,"time":"2024-05-24T06:04:42+00:00","remoteAddr":"71.185.229.66","user":"MiniFireDragon","app":"no app in context","method":"MOVE","url":"/cloud/remote.php/dav/uploads/MiniFireDragon/web-file-upload-da59fb99cc08a35b/.file","message":"Could not open file","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0","version":"28.0.5.1","exception":{"Exception":"Sabre\\DAV\\Exception\\ServiceUnavailable","Message":"Could not open file","Code":0,"Trace":[{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Upload/AssemblyStream.php","line":294,"function":"get","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Upload/AssemblyStream.php","line":164,"function":"getStream","class":"OCA\\DAV\\Upload\\AssemblyStream","type":"->"},{"function":"stream_read","class":"OCA\\DAV\\Upload\\AssemblyStream","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/icewind/streams/src/Wrapper.php","line":55,"function":"fread"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/icewind/streams/src/CallbackWrapper.php","line":96,"function":"stream_read","class":"Icewind\\Streams\\Wrapper","type":"->"},{"function":"stream_read","class":"Icewind\\Streams\\CallbackWrapper","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/lib/private/Files/Storage/Local.php","line":339,"function":"file_put_contents"},{"file":"/home/cloud/domains/cloud/public_html/cloud/lib/private/Files/Storage/Local.php","line":650,"function":"file_put_contents","class":"OC\\Files\\Storage\\Local","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":653,"function":"writeStream","class":"OC\\Files\\Storage\\Local","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":653,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Connector/Sabre/File.php","line":250,"function":"writeStream","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Connector/Sabre/Directory.php","line":148,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":325,"function":"createFile","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":151,"function":"copyNode","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":181,"function":"copy","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Upload/ChunkingPlugin.php","line":94,"function":"move","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Upload/ChunkingPlugin.php","line":76,"function":"performMove","class":"OCA\\DAV\\Upload\\ChunkingPlugin","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"beforeMove","class":"OCA\\DAV\\Upload\\ChunkingPlugin","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":603,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/koicloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpMove","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Server.php","line":373,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/home/cloud/domains/cloud/public_html/cloud/remote.php","line":172,"args":["/home/cloud/domains/cloud/public_html/cloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/home/cloud/domains/cloud/public_html/cloud/apps/dav/lib/Connector/Sabre/File.php","Line":505,"message":"Could not open file","exception":[],"CustomMessage":"Could not open file"},"id":"66542e3e236e3"}

I believe you are correct on a timeout, but the only timeout I could think of that is occuring is when it is copying the file from the tmp folder to what ever folder nextcloud uses. Other then that as I watch the apache log, it rolls on and on during the upload, followed by a move and then a delete entry.

As for an error, there is one:

[Fri May 24 00:40:42.184013 2024] [proxy_fcgi:error] [pid 1651542:tid 1651677] (70007)The timeout specified has expired: [client xx.xx.xx.xx:62222] AH01075: Error dispatching request to : (polling)

I have the exact same issue. I can see all the chunked parts are written to disk so the upload itself works. I can even see the “.part” temp file being constructed in the final destination, however it stops around 13 GiB and Nextcloud deletes it. Specifically which timeout do I need to adjust to give Nextcloud more time to construct the final file from the chunks?

EDIT: Figured it out in my case. I use the linuxserver.io Nextcloud docker along with Swag. I had proxy_read_timeout set in multiple places, one of which had a shorter timeout. Depending on the order the docker containers were started, the effective configuration would change.

I have given up on troubleshooting the problem. It seems to be only related to nextcloud. I have adjusted timeouts and sometimes it will upload just fine, other times it will not.

Watching the file upload there are times nextcloud will issue a delete command then a move command. Then complain that it cannot find the file.

I will reiterate I did get it to upload a large file (5gb always worked, after that it is random), successfully, but randomly.

One thing I did notice, and it is strange, is that I had more success when the file was uploaded on a slow connection vs a fast connection.

(post deleted by author)