Any uploaded files are 0 bytes in size, creating a new text file and adding to it works fine

About 2 weeks ago I upgraded my storage solution from a Synology to DIY truenas scale NAS, both operated as an NFS share for external storage in Nextcloud.

Nextcloud worked great until this move. Nextcloud still mounts and displays all data on the Truenas NFS share, I can even download, view, and share content currently on the new Truenas storage, but any file uploaded to it from the Nextcloud phone app or web client (from any of my users) uploads as 0 bytes and is completely unusable.

I can, however, on every user account create a new text file on the NFS share and edit/save it just fine. Typically i would suspect a permissions issue, but the fact that any user can create/edit a file anywhere on the NFS share through the nextcloud web GUI makes me doubt that now.

All files/mount-points on the nextcloud server show as owned by apache:apache. Any ideas on what could be different with the new backend storage solution? I don’t see any specific logs on nextcloud that make me suspect anything in particular.

Nextcloud deets:

Nextcloud version: 27.0.1 - 27.0.1.2

Operating system: Linux 5.14.0-162.23.1.el9_1.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Apr 11 19:09:37 UTC 2023 x86_64

Webserver: Apache/2.4.53 (Rocky Linux) OpenSSL/3.0.1 (fpm-fcgi)

Database: mysql 10.5.16

PHP version: 8.0.27

Oh also, I was on NC version 25.x.x I believe when I first noticed the issue, but it was present in every version until I stopped at the latest.

What does your nextcloud.log show when these zero byte files are uploaded?

And just to confirm I understand: creating new files in the web UI works fine, but uploading through the web UI does not?

If i tail -f the nextcloud.log file, nothing happens when I upload the files. They just appear with the correct name/extension and 0 bytes.

Also, you are correct, any user can hit the plus button to create a new file in any of the folders and edit/exit from it. The new file will show as being larger than 0B (depending on how much they added of course) and they can open it again and edit it etc…

While I was writing this post, I decided to test some different files, 99% of the files uploaded/downloaded from my Nextcloud are photos and small docs, so they instantly populate as 0 bytes when uploaded.

However something new I just discovered is that if I try to upload something larger like a 10GB file or even just a 1GB file, both of which used to work fine, I do actually get an error in the nextcloud.log, odd that I don’t see anything from uploading smaller files…

Here’s one of the errors that seems to be repeated when uploading “larger” files:

{“reqId”:“ZLx97QoerO6gD82HMHCmfwAAAEY”,“level”:3,“time”:“2023-07-23T01:10:06+00:00”,“remoteAddr”:“192.168.x.x”,“user”:“USER”,“app”:“webdav”,“method”:“MOVE”,“url”:“/remote.php/dav/uploads/USER/web-file-upload-64a8fbbe28ba17da5924d777716b5456-1690074604406/.file”,“message”:“Error while copying file to target location (copied: -1 bytes, expected filesize: -1 bytes)”,“userAgent”:“Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0”,“version”:“27.0.1.2”,“exception”:{“Exception”:“Sabre\DAV\Exception”,“Message”:“Error while copying file to target location (copied: -1 bytes, expected filesize: -1 bytes)”,“Code”:0,“Trace”:[{“file”:“/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php”,“line”:149,“function”:“put”,“class”:“OCA\DAV\Connector\Sabre\File”,“type”:“->”},{“file”:“/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php”,“line”:307,“function”:“createFile”,“class”:“OCA\DAV\Connector\Sabre\Directory”,“type”:“->”,“args”:[“*** sensitive parameters replaced ***”]},{“file”:“/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php”,“line”:133,“function”:“copyNode”,“class”:“Sabre\DAV\Tree”,“type”:“->”},{“file”:“/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php”,“line”:163,“function”:“copy”,“class”:“Sabre\DAV\Tree”,“type”:“->”},{“file”:“/var/www/nextcloud/apps/dav/lib/Upload/ChunkingPlugin.php”,“line”:94,“function”:“move”,“class”:“Sabre\DAV\Tree”,“type”:“->”},{“file”:“/var/www/nextcloud/apps/dav/lib/Upload/ChunkingPlugin.php”,“line”:76,“function”:“performMove”,“class”:“OCA\DAV\Upload\ChunkingPlugin”,“type”:“->”},{“file”:“/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php”,“line”:89,“function”:“beforeMove”,“class”:“OCA\DAV\Upload\ChunkingPlugin”,“type”:“->”},{“file”:“/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php”,“line”:603,“function”:“emit”,“class”:“Sabre\DAV\Server”,“type”:“->”},{“file”:“/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php”,“line”:89,“function”:“httpMove”,“class”:“Sabre\DAV\CorePlugin”,“type”:“->”},{“file”:“/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php”,“line”:472,“function”:“emit”,“class”:“Sabre\DAV\Server”,“type”:“->”},{“file”:“/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php”,“line”:253,“function”:“invokeMethod”,“class”:“Sabre\DAV\Server”,“type”:“->”},{“file”:“/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php”,“line”:321,“function”:“start”,“class”:“Sabre\DAV\Server”,“type”:“->”},{“file”:“/var/www/nextcloud/apps/dav/lib/Server.php”,“line”:364,“function”:“exec”,“class”:“Sabre\DAV\Server”,“type”:“->”},{“file”:“/var/www/nextcloud/apps/dav/appinfo/v2/remote.php”,“line”:35,“function”:“exec”,“class”:“OCA\DAV\Server”,“type”:“->”},{“file”:“/var/www/nextcloud/remote.php”,“line”:172,“args”:[“/var/www/nextcloud/apps/dav/appinfo/v2/remote.php”],“function”:“require_once”}],“File”:“/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php”,“Line”:279,“message”:“Error while copying file to target location (copied: -1 bytes, expected filesize: -1 bytes)”,“exception”:{},“CustomMessage”:“Error while copying file to target location (copied: -1 bytes, expected filesize: -1 bytes)”}}

Oh and just to be clear, the larger files never upload like the smaller ones, Nextcloud just errors out in the WebUI with the error:

Error while copying file to target location (copied: -1 bytes, expected filesize: -1 bytes)

In a last ditch effort before completely re-building my Nextcloud I decided to try a bunch of random reboots/resets/restarts, nothing seemed to help until I made a new dataset on TrueNAS and made it exactly like my original NFS share, copied all the settings and permissions and lo and behold, everything works fine on it.

So I’m currently rsync’ing all my data from the old dataset to the new one and I’ll just point everything at that share then kill the original dataset.

Really strange issue that I couldn’t find anyone else having, but oh well… This is as solid of a workaround as I’ll probably be able to find.