Ncp-update-nc BTRFS not supported

ncp
bananapi
#1

Hello.
I have NC installation running, but it seems that when I try to update with:
root@ncp:~# ncp-update-nc
BTRFS not supported
I cannot find a reason why is btrfs anyway “special” that it is not supported. Can someone give me or point me to an answer?
How those installations should be updated?

#2

Hello,

First, without any clue about your system or config, how do you expect to get helped !

As up today, BTRFS is mainly unstable on every distro with a kernel below 5.0.
Also, with kernel 5.0, if you have some kind of raid, you may experience “starbge” bug like write holes…

#3

this doesnt mean that it’s compatible. you claim to have nc installed. but if it’s not based upon ncp you won’t find any of the nc-commands working for you-

#4

I would like an answer, so this is more like “help me find an answer”. :wink: So I’m not sure how it would help as I’m asking about filesystem support, but here: I have Armbian running on BananaPi with nextcloud installed via nextcloudpi.

I’m btrfs user for 5+ years now and no major issues so far. That’s one of the reasons why I’m looking for an answer. Filesystem tend to have bugs - some of them have more bugs, some less, but why one is supported and the other is not? What if I’m using bcachefs for my setup? Or somebrandnewuntestedfs? That’s a users choise and if there is no valid arguments except “because it ate my data!!!” (like “because we’re using some feature that doesnt work in btrfs”) it should just work (maybe with --yes-i-know-i-m-using-btrfs option).

It is, ncp-update (and others) works fine.
There’s even a line ncp-update-nc:
[[ “$(stat -fc%T “$BASEDIR”)” == “btrfs” ]] && { echo “BTRFS not supported” ; exit 1; }

It seems I’ve found an answer (well, a bit weird, but it seems to be related to “BTRFS not supported” message):

#5

It seems NCP supports btrfs only for ncdata. if the whole drive and OS is on btrfs it does not (yet) support that.
You will have to update using NC’s native updater to upgrade NC.

#6

What do you mean by “NCP supports btrfs only for ncdata”?

I’ve commented out the btrfs check in ncp-update-nc and it “just worked” as on any other fs. So
one question solved - you can comment it out and “on your own risk” update even on btrfs.
[I can consider this as “second question solved”]

But still the first question is without an answer:

#7

Good to know.

Did you see https://docs.nextcloudpi.com/en/configuration-reference/#backups ?

NCP supports btrfs (and snapshots) when you let ncp format the drive and move the data to it with nc-datadir.
Maybe @nachoparker can enlighten us why this check is included.

#8

You can’t delete a BTRFS subvolume with a rm command. Since in BTRFS ncdata is a subvolume, the script needs to be adapted so I added that check until that happens.

This only affects when we have / in BTRFS. Most common setup has / in ext.

It should be an easy fix. PRs are welcome.

#9

My point was maybe not clear enough. I am not saying btrf is not fit for regular use. But:
Using btrf in a raid 60 right now is clearly Mark as unstable by the dev team.
All my server are raid 60 based, in clustering mode. I have already experience a “write hole” bug, witch cascaded through a whole cluster.
It was more than my tech capacities. Had to reach to a data recovery us based company who charged us a lot.
So, when someone ask about btrf, I use to give that answer.

Sorry if it was not really accordingly for your scope of used.

Over the 20+ server I had today, only one is still in raid60 btrf, kind of ‘lab’ server

#10

@nachoparker This I dont understand - I have btrfs on / and without ncdata subvol. I would even say that without any subvols except those I’ve created.

@stratege1401 One should always remember that raid is not a backup.

#11

YOU ARE ABSOLUTELY RIGHT !

but it could if done in raid60 clustering over iscsi, with de-localized units.

#12

Funny, I have the same setup and did the same mistake when choosing to have btrfs on the SSD. :smile:
After all, I was following some recommendations that said, btrfs is better for backup purposes…
Now, NC is running fine but sometimes I find the Banana locked up with some file system error message.
I checked some logs and came across a log message saying
swap file has holes
My search revealed that btrfs does not support swap files.
And that’s when I stopped due to time reasons.
Since, I have been thinking of repartitioning the SSD to add an ext FS where the swap could be placed on - but that doesn’t seem easy also.
If I understood right, the “not supported” issue is due to a wrong call how the backup subvolume is removed at the end of the upgrade process. If there is more about it, maybe the separate ext would also help in this case?

#13

disable swap and enable zram

#14

if you use nc-datadir to a BTRFS volume then the datadir will be placed in a subvolume so it can be snapshotted

#15

Is there a way to check and confirm this (on my device)?

#16

nc-info should provide that or
to list subvolumes you can also use

sudo btrfs subvolume list /path/to/ncdata

#17

But I didnt make any [checks his backups] mistakes. :wink: As I said I’m using btrfs and know it’s pros and cons. I’m using zram swap, so no swapfile issues.

@nachoparker So maybe check if ncdata is a subvol and if not just support normal update? It seems the check is trival: https://stackoverflow.com/a/25908150 (or just add a switch --i-know-what-i-m-doing)

#18

I invite you to study the problem and provide a solution for all the community. The idea is that each person conttributes with improving a particular itch and NCP gets better everyday.

I am using a different setup so I am not particularly interested in solving this but I would be happy to review and accept a PR. :slight_smile:

#19

Hi nachoparker,

I have installed Debian on a btrfs partition and then used NCP to install nextcloud. Accordingly, I ran into the same error message when trying to upgrade Nextcloud via NCP. I do have the same setup:

  • System and Nextcloud is installed on a btrfs partition, with / as btrfs subvolume.
  • This is the ONLY subvolume on the system partition, besides others I created. (But this should not affect the update process.)
  • I have moved the ncdata directory to an external SSD using nc-datadir.sh. The partition on the external SSD was formatted as btrfs by nc-datadir.sh. Therewith, the datadir does not reside on the same partition than the Nextcloud program.

I would like to try and find a small fix for this.

I do have a small request for you: I would like to make sure, that I properly understand your reason for blocking the NC Update/Upgrade on a system, where NC resides on a btrfs partition.

The reason I understand as follows:

  • On line 114 of ncp-update-nc the command rm -rf "$BASEDIR"/nextcloud will fail, if the datadir still is under /var/www/nextcloud/data, since a btrfs subvolume cannot be removed by rm.
  • So you abort the upgrade process if the root partition / tested in line 32 with the expression [[ "$(stat -fc%T "$BASEDIR")" == "btrfs" ]] reports as true.

My thoughts run as follows:

  1. As I understand, the btrfs subvolume underneath a directory, which is removed, is NOT removed itself. Instead, all files and directories in it are removed. Essentially, the btrfs subvolume is empty after the rm command.
  2. If now a new directory is created, where the still existing btrfs subvolume would be a subdirectory beneath, this particular subdirectory is not newly created. Instead, the existing btrfs subdirectory is used! So, when you use in line 115 the command mv "$BASEDIR"/nextcloud-old "$BASEDIR"/nextcloud, this would again populate the still existing btrfs subdirectory. There will be some error messages, but the command should continue.

Are these thoughts correct, or do I have made a mistake?

Best regards
LittleAlf

#20

Your diagnosis is basically correct, I might be forgetting some detail.

On rm it will fail with Operation not permitted, so we have to do btrfs subvolume delete in the BTRFS case, which will remove the subvolume.

If you start digging into this it will be pretty obvious :wink: