Nextcloudpi: backup fails, btrfs-transaction takes 100% CPU for hours

Nextcloudpi with two harddisks and a ssd. Worked for some years. Two harddisks connected over an USB-hub with its own energy supply. Setup:

NextcloudPi version	v1.52.0
NextcloudPi image	NextCloudPi_10-30-22
OS	Raspbian GNU/Linux 11. 6.1.21-v8+ (aarch64)
automount	yes
USB devices	sda sdb sdc
datadir	/media/CloudDrive-1/nextcloud/subncdata
data in SD	no
data filesystem	btrfs
data disk usage	1.6T/4.6T
rootfs usage	7.0G/118G

I’m logging in as ncp “https://192.168.127.101:4443”. Then I click on make backup (the German text is “Sicherung erstellen”), including the data. The process, which always worked starts, but after displaying “checking for free space” nothing more happens.

I can log in via ssh and top tells me there is a btrfs-transaction process that consumes 100% CPU. It can not be killed, “kill ” does nothing!

So I pulled the plug. After restarting the nextcloudpi everything is ok, nextcloud is running, but I still have no backup.

Any ideas?

I took a new hard disk, the backup started, but crashed:

check free space...
Maintenance mode already enabled
backup database...
backup files...
/usr/local/bin/ncp-backup: line 99: 8650 Segmentation fault tar $compress_arg -cf "$destfile" "$dbbackup" --exclude "$data/.opcache" --exclude "$data/{access,error,nextcloud}.log" --exclude "$data/access.log" --exclude "$data/ncp-update-backups" -C "$(dirname "$datadir"/)" $data --exclude "nextcloud/data/*/files/*" --exclude "nextcloud/data/.opcache" --exclude "nextcloud/data/{access,error,nextcloud}.log" --exclude "nextcloud/data/access.log" --exclude "nextcloud/data/appdata_*/preview/*" --exclude "nextcloud/data/*/uploads/*" --exclude "nextcloud/data/*/cache/*" --exclude "nextcloud/data/*/files_trashbin/*" --exclude "nextcloud/data/*/files_versions/*" --exclude "nextcloud/data/ncp-update-backups" --exclude "nextcloud/data/__groupfolders" --exclude "nextcloud/.data_*" -C $basedir nextcloud/
error generating backup
Maintenance mode already enabled

Segmentation fault? Why?

Did you check syslog if there are btrfs errors?

Thank you: 1643 btrfs errors on the data disk. I’ll have to replace it. Has been running for four years.

I’d had those errors twice and I was lucky to have synced snapshots on another device (in addition to regular backups). Syncing back worked without problems.

Syncing back? I’ve got snapshots and backups, but how do I sync back to a new disk? If you had just some hints of the general direction, I’d be grateful.

Did you set up sync of your snapshots? If so there should be a cron job

/etc/cron.d/ncp-snapsync-auto

that calls

/usr/local/bin/btrfs-sync <from source device snap-shot-dir> <to target device snap-shot-dir>

Just get this line and switch source and target directory and run this command. The very last step is to create a subvolume named data from the very last snapshot (these are usually read only). See also How to backup and restore using nc snapshot.