Thinking of switching to BTRFS, it this the correct process?

Hi,

I’ve been running NCP for years on a Pi 3 booting from a flash drive containing the system and with my data drive on a HDD formatted as ext4. I’ve been automaticity generating dataless backups (with occasional manual full backups). However I’m finally looking at using BTRFS so I can backup with snapshots.

Is the process ‘simply’ to format a new HDD as BTRFS, connect the new HDD to the NCP system, copy all the data from the original disk across then switch the mount point to use the new drive?

Assuming this is correct and works my next steps will be to reformat the existing HDD as BTRFS and create snapshots to this, and eventually to copy these snapshots to an additional BTRFS share on an Unraid server. But I’m initially just wanting to get the data drive set up as BTRFS.

Cheers for any help.

Hi DAveShillito.

tl;dr

Sounds good :slight_smile: (…but “Ts and Cs apply”). Why BTRFS, though?

more

Disclaimer: I haven’t used BTRFS for a few years; see below…

Using snapshots is a good idea, because it helps reduce the possibility that your backups will be internally inconsistent :+1: But remember that for a complete backup of your data, you’ll also need a (contemporaneous) dump of the database (i.e. in addition to the file system).

The way I do this is:

  1. Put my Nextcloud instance into Maintenance Mode
  2. Dump out the DB
  3. Take a snapshot of the filesystem where Nextcloud data live
  4. Take my instance out of Maintenance Mode again
  5. Create the backup from the DB dump and the 'shot

…which means Nextcloud is only offline - in Maintenance Mode - very briefly. (This is run by a systemd .service file and .timer file).

At the risk of stating the obvious: if you’re going to transfer your data from their current home to a new disk, you should do it when those data are offline (…so you’re not trying to copy “a moving target”).

EDIT Re “offline” -
To be clear, what I mean is: 1. Stop Nextcloud entirely (i.e. stop Apache/Nginx/whatever) 2. Re-mount the existing data disk read-only 3. Copy the data from there to the new disk.

Qs.

  1. AAMOI, why BTRFS? Once upon a time it was quite popular, I think…but nowadays it seems to have been eclipsed by ZFS(?) which is what I use. And if all you need is snapshots, you don’t need either of them: LVM can do that :slight_smile:
  2. How comparable are the two data disks, and could you use both at once? Is it worth combining them somehow (RAID1 via mdadm, or LVM, or ZFS, or whatever) for redundancy…?

hth

P.S. I highly recommend https://www.borgbackup.org/

1 Like

Cheers @t-cubed,

Yes, your approach is basically what I was envisioning, thanks for confirming I am on the right path :grinning:

As to why BTRFS… I don’t really know much anything about filesystems, but my belief was that ZFS is more complicated that BTRFS, and more designed for pools of multiple drives. This is just for a single drive home Nextcloud server with a couple of users.

Thanks for mentioning LVM, I shall investigate that, but again this is just for a small server running on a Pi, so I’m happy to keep everything as simple as possible. Every new thing I try is an opportunity to break something due to my lack of knowledge :smile:

Indeed just occasionally copying the exported full backup to the second drive would probably be enough, but since the data will not change much, if I am understanding it correctly, after the initial snapshot the later ones will be smaller, just containing the delta of the changes, rather than a full export of the data folder.

ZFS is more complicated that BTRFS

That’s probably true. Though as I say, I haven’t used BTRFS for a while; my memory may be rusty! I know ZFS fairly well, and it does lots of cool stuff. But it’s not the best choice for everything, I think.

If you don’t need all the fancy new-fangled features of BTRFS and ZFS, my suggestion would be to stick with good old ext4. But put it in a Logical Volume, so you can snapshot it.

Indeed just occasionally copying the exported full backup to the second drive would probably be enough, but since the data will not change much, if I am understanding it correctly, after the initial snapshot the later ones will be smaller, just containing the delta of the changes, rather than a full export of the data folder.

Taking a snapshot of the file system with the “live” data folder, then rsync’ing that to the other disk, you mean? Yep, that sounds good :slight_smile: I recommend automating this as much as possible (with systemd or cron); personally, I’m very good at forgetting to do this kind of thing (unfortunately)… :slightly_frowning_face:

Do check out borg, though; it’s really good. (Or restic, which is sort-of comparable).

1 Like