I have a NC with an external hosting provider, and I would like to have an immutable backup on my home server (Open Media Vault, Debian). The reason is to protect myself against ransomware attack and in general to reduce my dependence on the hosting company. I want OMV to pull the data rather the NC server to push it (it is not my server), and I only care about the files (not the NC database).
What would you recommend?
I see two options: 1) A backup software on OMV that connects to the NC with WebDav and saves the updates locally. Probably Duplicati can do that but maybe also borg can be configured. 2) A native CLI nextcloud-desktop client installed on OMV, with the local folder configured on a btrfs or zfs volume with daily snapshots kron jobs.
UPDATE
I’ve configured my OMV (Debian) to wake up daily, sync with the Hetzner NC with a backup user via cmdnextcloud, make a btrfs snapshot with btrbk, and shut itself down till tomorrow with rtcwake. My active users are sharing their folders with the backup user giving read only permissions. It looks that the initial sync was limited by my internet bandwith, but now it is a matter of minutes If anything happens to Hetzner - my files are easily accessible on the local network. If anything happens to the home server - reinstalling from scratch takes an hour. Breaking to home server cannot corrupt the data on the cloud. Breaking to my workstation/the cloud leaves my btrfs snapshots of uncorrupted data. The DB is backed up by Hetzner snapshots, but it is not crucial for my use anyway.
Personally, I don’t think WebDAV and the Nextcloud clients are suitable for your action. It would be much easier if your Nextcloud hoster offered you a real backup. Either that it performs the backup or that you can pull it via sftp/rsync, for example.
What does your provider offer? Can you possibly buy the backup service from the provider?
If not, you could use a Nextcloud client for synchronisation. Remember that synchronisation does not replace the backup. But you may be able to copy it more easily from the client to a real backup storage using real software such as sftp/rsync.
Alternatively, you may be able to host Nextcloud yourself on the Internet. You then have access via sftp/rsync and can easily copy all files using suitable methods such as sftp/rsync and not inadequate methods such as WebDAV.
WebDAV and Nextcloud clients may be quite nice for personal synchronisation. For real backups they are rubbish or at least unprofessional.
The provider does a backup - but a) it is theirs, if for any reason they block me, I will not be able to access the backup, and b) It is not infinitely incremental, a ransomware attack vector can be quietly write a new geebrish into my data for a few months. I do not want to self host as provider’s hardening is superior to mine.
Why do you say that sync does not replace backup? The intention is to put it on a filesystem that allows daily snapshots.
Hetzner don’t allow sftp access to their “Storage Share”.
I could consider migrating.
But how rsync is better than the NC client?
And how a backup software snapshot is better than a ZFS/btrfs snapshot?
I can think of a permission issue, I don’t want the homeserver to become another exposure surface, so I’d like to configure the NC client in read only mode, create a backup user with limited permissions.
Companies want to sell and, above all, not lose customers. A backup including a Nextcloud database dump that is accessible to the customer may be more customer-friendly, but it may be bad for business.
Try it out and you’ll know why. sftp/rsync is a completely different protocol to the rubbishy WebDAV.
I think most users synchronise their data with Nextcloud Client to a client and then - because they know that synchronisation is not a backup - make a backup of it.
In terms of reliability, however, Nextcloud synchronisation cannot be compared with rsync synchronisation. You can ask how Hetzner creates the backups of your Nextcloud. They are unlikely to use Nextcloud clients or WebDAV. They probably simply use sftp/rsync.
I think Hetzer but also Microsoft want to force a vendor lock. This is much easier with Microsoft, of course, as their software is proprietary. Hetzner’s Nextcloud is easy to replace.
Does anyone know the exit strategy for Nextcloud at Hetzner? Is that difficult? If it is hardly an obstacle, Hetzner could also release the backup for direct access.
I don’t have Nextcloud with Hetzner. But a properly accessible backup including Nextcloud database dump would be a reason for me to buy.
Maybe someone from Hetzner is reading this and can tell you more.
→
And if you’re not happy with it, don’t try to build something around it. Instead, find another Nextcloud hoster or host it yourself.
Is NC client a WebDAV? I thought it would be something more clever, connecting on the DB level. And at least theoretically it should be the golden etalon of NC synchronization, not rsync.
I could easily do that once the files are synced on my home server (just running a backup from one storage to another), but isn’t it kind of unnecessarily redundant?
They state they do a ZFS snapshot of everything (DB included)
Yes. But it can only use webservices and not a sftp-subsystem.
However, if you assume that the data on your synchronised nextcloud client is correct, then feel free to use this as the basis for your backup, e.g. on your NAS.
The problem is that synchronisation, whether via Nextcloud or rsync, is not a backup. The synchronised status must be backed up independently of the synchronisation, as the next synchronisation may destroy the data. You need e.g. backup versions for every day, week … but at least one.
That’s pretty common, with managed Nextcloud providers.
Just a guess, but they probably don’t use a dedicated VM for each instance, but some sort of distributed container infrastructure (K8s?) with shared databases and shared storage to provision those instances, so they probably couldn’t give you root access to the database and storage easily, even if they wanted to. And that’s likely one of the reasons why they’re so cheap.
But at the end of the day, you have to ask those questions to them, and if they can’t offer what you want, you have to switch to a provider that can, or you could host your own Nextcloud, either locally, on a dedicated server in their datacenter, or on a VPS.
For example, on a dedicated server you could use ZFS and then replicate the pool to your home NAS via SSH. However, this would be much more expensive and you would have to set up and configure everything yourself.
This “something clever” as you call it would require the database to be world accessible, and wouldn’t help either because the files aren’t stored in the database, they’re just referenced in the database.
@devnull I wouldn’t call WebDAV rubbish, it’s just not as performant as rsync, which by the way is not all that performant over SSH either, especially from or to a remote location. The same goes for SFTP, which I would argue is probably even slower on avaregae than WebDAV.
But rync mostly uses ssh connections (but can/could use others as well), no?
Regarding webdav, the native windows/macOS clients were/are not the best and have some issues, that is not an issue of the protocol itself.
Just one more point, native webdav vs. Nextcloud client: Nextcloud client does much more than just file transfer, it creates a lot of overhead, especially if you want to run the backup on a small device without desktop interface. There, a backup solution using native webdav has less dependencies, uses less resources, …
Yes, but you can also run an rsync deamon and then connect to it directly, which I wouldn’t recommend doing over the internet.
Or you can of course also also use rsync locally to synchronize data on the same machine or over SMB or NFS connections etc, in which case it is very fast.
Sure, but for me it works mostly fine. A few weeks ago I reinstalled my laptop and it took about 30, maybe 40 minutes (not exactley sure to be honest) to synchronise all my files back (20’514 files / 178 GB over a wired 1Gbit connection).
Is it slower compared to Rsync over SSH? Maybe, probably.
Does it matter in daily use? At least for me it doesn’t, since I rarley sync that much data at once.
I’d recommend going with the native CLI Next cloud client on OMV and using ZFS or BTRFS with snapshots. It’s simple, secure, and gives you versioned backups that are easy to restore if anything goes wrong. Duplicati with WebDAV is another option, but the snapshot feature with ZFS/BTRFS makes it more reliable and easier to manage in the long run. Just set up automated backups and you’ll be good to go!
So the initial sync started at 09:48:42 then was interrupted at 10:08:45 for some rason after synchronizing roughly 13’000 files.
The sync then continued at 12:41:12 and was completed at 12:49:09. So the total sync time for about 20’000 files / 170 GB was about 30 minutes, which would be an average download rate of about 90MB/sec, which I’d say is not all that bad for WebDAV.
But yeah, the synchronisation process has stopped at some point, so there’s that.
Personally, I would dump the database and then save the Nextcloud installation, the Nextcloud configuration and the Nextcloud data. And of course also web server configurations, …
Also very important: Try out the backup to see if it really works and fulfils its purpose. For example, you can upload the Nextcloud backup to a test server. You can change the domain name for this. Issues such as SSL certificates can be ignored during the test restore.
Maybe I’m missing something, but generally using WebDav allows you to create a good backup of the files residing in your own Nextcloud user’s realm.*
The speed of WebDav is more than adequate for a backup, as it is for initial and incremental sync (see the NC client) – if your connection is reasonably fast. And every file request will go through Nextcloud, allowing your backup to skip temporary, locked and incomplete files. Additionally, it can easily back up external storages as well. (I’ve used an external SFTP storage for a while and always backed it up to my Synology through a WebDav connection to Nextcloud, it was slow but worked fine.)
*Which means, all of this is not included:
Other users’ files that your user doesn’t have access to (maybe you share your NC with family or friends or plan to do that in the future?)
Older versions of your files
The database
The data of many apps that only use the database, i.e. calendar, contacts, photo metadata, deck cards, activity etc.
All user configuration and share configuration (you will have their files though)
PS: Before researching the perfect solution, I’d really like to advertise for making sure you have a bare minimum backup. This could just be a rsync backup of your synced local folder. Because you simply cannot rely on the hosting provider.
I never had a bad experience with Hetzner, but I did have another established provider delete the entire data of a paid Nextcloud instance earlier this year with no way of recovery. My only backup was the folder on my Mac of what was synced and the older versions in my TimeMachine backup. Other users of this instance lost all their data (which wasn’t a lot, fortunately), because they hadn’t synced anything.
