Best way to upload a lot of files in Nextcloud ? (no ssh)

I understand this topic comes recurrently and as most of us I would say my case is special this is way I’m making a new thread after reading few of them and not finding answer to my question :slight_smile:

Anyway, I most of the case, yes rsync the data than indexing it is what I’m doing, and it gets the job done very well

I can’t SSH to the server

The server is remote and I still have a good amount of data to sync/upload on the NextCloud instance (50GB) and for unknown reasons, the connection drop and/or do timeout and the rsync simply hang.

So; from the server with all the data (let’s call it source) I mounted webdav in a directory (/mnt/nextcloud/) than I do a rsync -azvP /srv/data /mnt/nextcloud/

My question is I still have a lot of error during the transfert and everything I restart the rsync it seams to resynchronise the file even if the option P (for partial and progress) is on.

What would you recommend ?

What would you check or do if you were in my situation ?

thank for your guidance.

PS: I do backup of few ubuntu machine (10) from the same network to the same nextcloud server via webdav everyday and I have no issue.

when I try to upload via a browser I recurrently have this error:
Error when assembling chunks, status code 500

but I wrote to the server and asked him to manage the php.ini as recommended here: 504 Gateway timeout for large file uploads

So far, the best result for now is using the plugin FlowUpdate
The speed is not awesome, but it is be able to resume and we could visually notice the speed dropping (which is a must to show to my customer)

Peek 2020-11-24 13-44

To be honest, I would just sync it as normal with the desktop client. I have about 350 GB of data in mine, and all of it was synced through normal means (Win10 desktop app, browser upload, or iOS photo sync).

It’s going to take a bit no matter how you do it. It’s not worth trying to cut corners. Just my opinion.

1 Like