Nextcloud VM backup via PBS — how to ensure data consistency?

ℹ️ Support
,
1

Hi everyone,

I’m looking for best practices regarding backups of a Nextcloud instance running inside a virtual machine on Proxmox.

Previously, backups were handled via BorgBackup from inside Nextcloud itself (over SSH to a remote server). In that setup, Nextcloud handled consistency internally, so I didn’t have to worry much about the application state during backups.

Now we are planning to switch to Proxmox Backup Server (PBS) and perform VM-level backups instead of application-level backups.

My main concern is data consistency during backups:

  • Is it safe to back up a running Nextcloud VM using PBS (snapshot-based backups)?

  • Do I need to put Nextcloud into maintenance mode before the backup?

  • Should I stop specific services (e.g., web server, database) inside the VM?

  • Or is it recommended to fully shut down the VM before backup to ensure consistency?

I understand that PBS uses snapshot mechanisms (like QEMU guest agent / filesystem freeze), but I’m not sure if that guarantees consistency for Nextcloud (especially database + files).

What would be the recommended approach in production?

Options I’m considering:

  1. Do nothing (rely on snapshot consistency)

  2. Use a script to enable maintenance mode before backup

  3. Stop database / services before backup

  4. Fully shut down the VM before backup

Any advice or real-world experience would be greatly appreciated.

Thanks!

2

Hi, generally see GitHub - nextcloud/all-in-one: 📦 The official Nextcloud installation method. Provides easy deployment and maintenance with most features included in this one Nextcloud instance. · GitHub. There is also mentioned when to do backups.

1 Like
3

Thanks for the link — I like this approach :+1: it makes sense to let AIO handle a clean shutdown before doing VM-level backups with PBS.

However, I’d like to clarify a few details to make sure I understand the workflow correctly and avoid misconfiguration when moving away from Borg.

As I see it, the setup would look like this:

  • 03:55 — run a cron script:
    docker exec --env STOP_CONTAINERS=1 nextcloud-aio-mastercontainer /daily-backup.sh

  • 04:00 — PBS starts VM backup

  • 04:05 — backup finishes

At this point, all Nextcloud containers are still stopped.
How are they supposed to be started again automatically?
Do I need to explicitly start them (e.g. via a script), or does AIO handle this somehow?

Also regarding Borg:

Since I plan to stop using BorgBackup completely and rely only on PBS:

  • What exactly should I disable in AIO?

  • Is it enough to just remove/disable the backup configuration in the AIO interface?

And one more thing about updates:

In the provided example, the update script still uses:
docker exec --env AUTOMATIC_UPDATES=1 nextcloud-aio-mastercontainer /daily-backup.sh

Why is the same script (/daily-backup.sh) used both for stopping containers and for updates (with different env variables)?

Right now (with Borg enabled), are updates tied to the backup process internally?
If Borg is disabled, will automatic updates stop working unless I explicitly schedule this second script?

I’d appreciate clarification on these points to better understand how AIO is intended to be used together with external backup solutions like PBS.

Thanks!

4

This is clarified in the docs. You could for example use START_CONTAINERS=1. See GitHub - nextcloud/all-in-one: 📦 The official Nextcloud installation method. Provides easy deployment and maintenance with most features included in this one Nextcloud instance. · GitHub

5

It is simply a wrapper script for the CLI starting, stopping, updating and co

6

Depends on if you trust the QEMU tools. I would say yes, but still use the stop mode instead. Better save than sorry. Also stop mode reboots the VMs, which can be quiet nice.

These are just assumptions on my part and NOT truths, so hopefully @szaimen can chime in.

The very best way would be this:

03:00 start maintenance mode with crontab. That way a user that is currently logged in an maybe editing a file with collabora will not loose the last keystroke letter ‘R’. Also it is transparent to the user that the system will soon go down.

03:01 Proxmox starts backup in stop mode. It issues a shutdown command to the VM. Since AIO has no problem at all handling a OS shutdown, every container will be shut down correctly and smoothly.

03:02 VM is down. Proxmox can do a data 100% guaranteed consistant snapshot of the VM.

03:02 Proxmox starts the VM again. VM is getting started, while Proxmox itself is still doing a Backup to PBS.

03:03 all AIO containers have started, Nextcloud is up and running again.

03:04 cronjob that disables maintenance mode

03:20 backup to PBS is finished. At what time does not matter much.

Maybe adjust the default shutdown timeout for the VM (5min?), so Proxmox does not kill the VM because it times out.

7

I’m using a slightly different approach in my setup.

I run daily backups using the built-in BorgBackup in Nextcloud AIO, and I also use PBS for regular VM snapshots.

I’ve adjusted the timing so that BorgBackup runs first, and PBS snapshot follows after that.

From my experience, PBS snapshots have been sufficient for full VM recovery. I’ve tested restores multiple times, and the VM (including Nextcloud AIO) always started without any issues after restore.

So in my case:

  • BorgBackup provides application-consistent backups (database + files)

  • PBS is used for full VM recovery

This setup works well for me without introducing downtime or additional complexity.

8

I decided to make two scripts for stopping and starting. After stopping, PBS makes a backup. Thanks for the help.

1 Like
9

Would you mind sharing them with us?
I am currently in the same boat.

Only thing that I do different is that I offload the files from the VM into a NFS share. IMHO storing files on the VM disk, which in return is 16k volblock is not great for many reasons. I rather make a small PBS backup of the 50GB VM and backup the data itself in seconds with ZFS snapshot send.

Borg is IMHO pretty cool for most users, since it does dedup and compression and offsite. Only downside is that it is happening on an application level, so I would assume it has to read all files, while ZFS does not.

10

sorry @szaimen for pinging you again, but could you affirm that shutting down a VM will do a clean shutdown of your AIO containers?

11

Just try it out yourselves

1 Like
12

When you shut down the system, roughly the following happens:

A SIGTERM is sent to systemd → systemd sends a SIGTERM to the Docker daemon → the Docker daemon tells the containerd service: “Hey, I received a SIGTERM, stop all containers” → containerd then stops the containers (see how that works here: docker container stop | Docker Docs).

So yes, everything should shut down cleanly. In practice, if something hangs, there will eventually be a timeout and containerd will send a SIGKILL, meaning the process gets forcefully terminated.

So if you really want to be 1000% sure that database shuts down gracefully (and the database is really the only component in the stack where, in an absolute worst-case scenario, something could go seriously wrong), you could write a script that first stops the PostgreSQL service inside the container, then stops all containers, and only then shuts down the system. In that case, you’d also need to change the restart policy from unless-stopped to always, or create a script that starts the containers again after reboot.

Is it likely that something goes completly south because of a shutdown/reboot? I’d say no!

I’ve been using Nextcloud for almost 10 years (not with Docker, though), and during that time I’ve upgraded my MariaDB instance in place multiple times. My Proxmox backups are done with freeze, I take hourly ZFS snapshots of my VM, and I’ve rolled those back multiple times, which is basically the same as cutting power and restarting the system, I had actual power losses. And guess what everything is still working, no database issues at all. Modern databases can handle that. Things would have to go really wrong for you to lose data due to a regular reboot, let alone end up with a completely corrupted database.

Long story short: yes, a normal “shutdown” is fine. Modern systems handle this gracefully, and modern databases are robust and stable enough. We’re not in the 90s anymore, where you needed dozens of workarounds and scripts to keep things stable, especially not in a homelab, where most of the time nothing is happening that these systems couldn’t handle gracefully anyway. :slightly_smiling_face:

1 Like
13

Cheers for that. Just saying “try it yourself” did not help me much, because I would not know where to look for :slightly_smiling_face:

So with that in mind, I would argue for PBS, the way like I described is the cleanest option.

03:00 start maintenance mode with crontab. That way a user that is currently logged in an maybe editing a file with collabora will not loose the last keystroke letter ‘R’. Also it is transparent to the user that the system will soon go down.

03:01 Proxmox starts backup in stop mode. It issues a shutdown command to the VM. Since AIO has no problem at all handling a OS shutdown, every container will be shut down correctly and smoothly.

03:02 VM is down. Proxmox can do a data 100% guaranteed consistent QEMU bitmap snapshot of the VM (not a ZFS snapshot!).

03:02 Proxmox starts the VM again. VM is getting started, while Proxmox itself is still doing a Backup to PBS.

*Note: PVE backup works with a bitmap. After the reboot, ever write the VM is doing, QEMU will first have to make sure it has already written that block to PBS, before changing it. That way it can guarantee consistency and otherwise it could wrongfully backup a newer state of the block. If your PBS destination is slow, this could severely delay the boot and start of the containers and even lead to timeouts for some applications (MSSQL, QEMU guest agent). If that is the case you should look into fleecing to mitigate this problem.

03:03 all AIO containers have started, Nextcloud is up and running again.

03:05 cronjob that disables maintenance mode

03:20 backup to PBS is finished. At what time does not matter much.

Side effect is that if the system is not up and running until 03:05, you instance will stay in maintenance mode. IMHO you could argue that is a positive or a negative.

1 Like
14

Sounds like a plan. :+1:

And yes, I’d say that’s the safe approach, with a short downtime, which usually shouldn’t be a problem at 3 a.m., unless, of course you have users actively using it at night. In that case, you could use snapshot mode during the week and maybe run stop mode once a week (e.g., on the weekend) as a compromise.

I personally only use snapshot mode, and I’ve restored backups multiple times (usually because I broke something, especially in the early days) and never had any issues. That said, my instance is basically idle at night. (so yeah I guess I could, and probably should use stop mode as well :wink: )

By the way, I’ve set up ZFS snapshots in addition to nightly Proxmox backups for convenience, so that I can instantly roll back if something goes wrong, and as I mentioned, I’ve never experienced any issues after a zfs rollback. However, I probably wouldn’t rely on ZFS snapshots as my only backup method, even if I would replicate them to another system, which I don’t.

1 Like
15

Small heads up, you don’t even need to disable maintenance mode after a reboot.

I have set this crontab with sudo crontab -e:

0 3 * * tue..sat docker exec --user www-data nextcloud-aio-nextcloud php occ maintenance:mode --on

do a backup with PBS with this time plan ‘1 3 * * tue..sat’

16

Please forgive me, I didn’t see any notifications that communication on this issue was ongoing.

What you described works perfectly for me, so you did everything correctly. If you have any questions or can help in any way, I’m always at your service.

1 Like