I’m using a shared server for NC, with the desktop apps on MacOS, and external storage for all files in DreamHost Objects.
Using NC sync app, large files are chunked in user/uploads before being moved to the selected folder in external storage.
The issue is that during large uploads, the temporary uploads folder can fill up to 10s of GB, which triggers a Disc Usage Warning at DreamHost’s end.
What I’d like to do is set the user uploads folder to also be on external storage, but there doesn’t seem to be an obvious way to do this. config.php options ‘tempdirectory’ and ‘part_file_in_storage’ seem like they might be relevant, but I have no idea if they can be used with S3-compatible storage.
So, my question is, is it possible to set the uploads folder or NC sync app temp folder to a bucket on external storage?
Thanks!
Steps to replicate it:
upload large files via sync app
file chunks are stored in user/uploads on shared server before being moved on upload completion to external storage
This becomes an even bigger problem when your file is large and your external storage is slow. Uploading a large file first fills up the Nextcloud server’s temporary storage, then a copy operation occurs to external storage. This could take quite some time and, is apparently handled synchronously during, say, a WebDAV MOVE request. For me, this results in the connection timing out every time, large files, (even less than 1G) never complete as a result.
Uploads need to go to their storage destination directly and not be first uploaded to the server. I understand that this is difficult to achieve under PHP, but this is 2019 and uploading multi-GB files to cloud services is commonplace. Maximum upload size should not be tied to the location where Nextcloud is running, but only to the available space of the final storage medium.
Is there any progress on this issue? The way the uploads work right now make it almost impossible to upload large files to an external storage, if the nextcloud instance runs on a shared hosting with limited space available. Is there any setting or workaround for this situation?
I don’t see any real way with a Nextcloud and uploading large files to more or less bypass the Nextcloud. What amount of data are you talking about? There is already free Nextclouds with 10 GB or more. Perhaps it would make sense to invest money in a paid Nextcloud that either has a large enough primary storage or uses S3 directly as the primary storage for somewhat higher requirements. You can get paid Nextcloud with 1 TB for monthly 4 euro.