Nextcloudpi - errors with nc-datadir --> USB drive not mounting anymore

Hi there,

following situation Iā€™m running NextCloudPi v1.25.0 on a Raspberry Pi 4. Data is on an external HDD. The server worked perfectly until I decide to move datadir to a SDD.
So I shutdown the R-Pi, unplug the HDD, plug in SDD and reboot. Then format SDD via web interface with BTRFS. Great. Of course always applying proper labels as per wiki.
Then I shutdown, plug in both HDD and SDD. Both are mounted and after reading some forum thread execute nc-datadir on the SDD, expecting that datadir is copied to SDD.

However, I get error messages and it doesnā€™t work. So I decide to shutdown, unplug SDD and reboot again. BUT now NCP doesnā€™t mount my orginal HDD anymore.
How do I fix this? Tried mounting via command-line but there is no mount target.
NCP webinterface still shows datadir with the original HDD folder, which what it should be, but the drive is not mounted

Appreciate any input on this.

Regards
FD

1 Like

Ok next thing I tried:

Complete re-install of NCP from scratch on SD-card, then re-installed backup without data.
Now the original HDD cannot be mounted (see above), but I also cannot move datadir to either SD-card or an empty external drive (which is mounted just fine) to recover data backup.

I really donā€™t want to loose my configuration, since clients are already set-up and running.
Can anyone point me to how I can fix mounting OR nc-datadir?
Searching forums didnā€™t give much results yetā€¦

Thanks
FD

Ok next trial:

I did a complete clean install. Installed a new data-dir on USB SDD (empty no data in there yet).
Re-did all my configuration as needed, set up all users and groups again.

Now there are two options to get my data back:

  1. Mount my original external HDD, where my datadir used to be (with all user data on it) and copy the files to new SDD via command line.
    This doesnā€™t work because I still canā€™t mount the HDD. When I plug it in, it shows up under NCP web interface as /dev/sdb. However logging in via ssh and doing df shows itā€™s not mounted. Also canā€™t mount it via command line, even with sudo.

  2. Somehow recover from a full backup I did, but obviously donā€™t want to get back all the broken configuration. I need ONLY the data.
    However, after googling quite a bit, I couldnā€™t find any information on restoring data only.

So Iā€™m stuck again.

Any ideas?

FD

The only thing you should have done is copy the data to your new drive and change the datadir in the config file.
To help you we need a lot more information, what is the output of ā€œdf --ā€? and what did you do when you tried to mount your old disk? And what is the current datadirectory according to your config.php file?

Re: copy data and change datadir in config file
Iā€™m using NextCloudPi because Iā€™m not too familiar with all the details, thatā€™s why I didnā€™t want to just copy data through terminal commands and tampering with config files.
I read in one of the forums that nc-datadir from NCP webinterface does in fact copy the data to the new datadir location and set config to the new location, which is exactly what I wanted (when transferring from my USB HDD to a new USB SSD).
Unfortunately the window on the webinterface spit out an error.

Re: more info
Mounting old disk

  • I only plugged in using automount, which works perfectly fine with my SSD, as well as my backup-drive, but NOT the old HDD (It used to work that way, before it ā€œbrokeā€). All drives have labels. All show up on NCP webinterface system info as sda, sdb, etc.
    BUT only the SDD and the backup-drive are mounted as shown per df on the terminal

  • current data directory has always been /media/if-ncp-hdd1/ncdata (/media/if-ncp-hdd1 was the mounting point for my old HDD) until Iā€™ve just recently installed everything from scratch and set to /media/if-ncp-ssd1/ncdata (/media/if-ncp-ssd1 is the mounting point of my new SDD)
    Mount point for my backup disk is /media/79034ff0-5bab-40a4-aa39-c3c394b82d22

  • output of df

Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 7345912 2204404 4811380 32% /
devtmpfs 1867780 0 1867780 0% /dev
tmpfs 1999876 0 1999876 0% /dev/shm
tmpfs 1999876 8696 1991180 1% /run
tmpfs 5120 4 5116 1% /run/lock
tmpfs 1999876 0 1999876 0% /sys/fs/cgroup
/dev/mmcblk0p1 258095 54359 203737 22% /boot
/dev/sda1 976700872 92136 974515160 1% /media/if-ncp-ssd1
/dev/sdb1 1953524124 1092954040 858560808 57% /media/79034ff0-5bab-40a4-aa39-c3c394b82d22
tmpfs 399972 0 399972 0% /run/user/1000

Questions

  • Any way I can force the HDD to mount? Normal mounting with sudo mount doesnā€™t seem to work
  • Since I have a backup on my backup drive, I did tar -xvf the file last full backup into a folder:
    sudo tar -xvf /media/79034ff0-5bab-40a4-aa39-c3c394b82d22/nextcloud-bkp_20200418_1587219017.tar
    But I did a mistake and un-tarā€™ed the files into my /home/pi folder first, which is on root drive / SD-card (because that was my active directory). I stopped the process and changed to the correct backup folder on the same backup drive. No big deal I though, can delete the few files later, SD-card still has plenty of space left.
    HOWEVER, now nextcloud webinterface doesnā€™t load anymore, saying " * Your data directory is invalid Ensure there is a file called ā€œ.ocdataā€ in the root of the data directory."
    Iā€™m completely confused now. How is my data directory in any way related to un-tarā€™ing files on a separate drive?

FD

Are your drives self powered?
If not try connecting them via a powered usb hub.

so you have 2 harddisks mounted now, I am not sure if a Pi can power up to 3 disks, so there might be a problem there.

Whats the output of:
ls -l /dev/disk/by-uuid/

Above step might not be necessary if your backup is good enough.
Does the folder ncdata already exists on your SSD?
/media/if-ncp-ssd1/ncdata
you could extract your backup with command tar -xvf archive.tar -C /media/if-ncp-ssd1/ncdata

@OliverV
Good point. The backup drive is. The SSD and small HDD are not. Iā€™ve had the old HDD run for a year that way without a problem however. The new SSD should consume much less and seems to be fine as well.
Youā€™re right though that it may be an issue if I connect HDD and SSD together however. Iā€™ll try with an USB Hub to make sure itā€™s not an issue.

@forzajappie
no I have only two disks connected max. mainly because three are only 2 USB3.0 ports.
And it doesnā€™t explain, why the old HDD that Iā€™ve used for a long time, suddenly does not automout any more, even if I try with it being the only disk.

Currently only SSD connected:

ls -l /dev/disk/by-uuid/
total 0
lrwxrwxrwx 1 root root 10 May 10 11:02 0b21f39a-9702-42a0-bdfd-c7f3760e27c6 -> ā€¦/ā€¦/sda1
lrwxrwxrwx 1 root root 15 May 10 11:02 45e99191-771b-4e12-a526-0779148892cb -> ā€¦/ā€¦/mmcblk0p2
lrwxrwxrwx 1 root root 15 May 10 11:02 4BBD-D3E7 -> ā€¦/ā€¦/mmcblk0p1

I can connect the others and run again if it helps.

Yes, /media/if-ncp-ssd1/ncdata is there:
sudo ls -l /media/if-ncp-ssd1/ncdata/
total 6688
-rw-r----- 1 www-data www-data 6844876 May 11 03:25 nextcloud.log

I tried un-tar to a BACKUP folder at the backup drive, since I have the space there and donā€™t want to mess anything else up at this point. Still running at this pointā€¦

If you used the command: ā€œtar -xvf archive.tarā€ you could stop it, as the command will not remove the tar file, how long is the process running now?

ls -l /dev/disk/by-uuid/ will show connected harddrives, so your old HDD needs to be connected.

the output of ls -l /dev/disk/by-uuid/

total 0
lrwxrwxrwx 1 root root 10 May 10 11:02 0b21f39a-9702-42a0-bdfd-c7f3760e27c6 -> ā€¦/ā€¦/sda1
lrwxrwxrwx 1 root root 15 May 10 11:02 45e99191-771b-4e12-a526-0779148892cb -> ā€¦/ā€¦/mmcblk0p2
lrwxrwxrwx 1 root root 15 May 10 11:02 4BBD-D3E7 -> ā€¦/ā€¦/mmcblk0p1
lrwxrwxrwx 1 root root 10 May 11 10:06 58a4f579-a361-4a65-ab41-bddf564baf7c -> ā€¦/ā€¦/sdb1

thatā€™s with the SSD (sda1) and HDD (sdb1) connected.
Backup drive is not connected at the moment.

BTW df shows that only SSD is mounted:
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 7345912 4846716 2169068 70% /
devtmpfs 1867780 0 1867780 0% /dev
tmpfs 1999876 0 1999876 0% /dev/shm
tmpfs 1999876 183676 1816200 10% /run
tmpfs 5120 4 5116 1% /run/lock
tmpfs 1999876 0 1999876 0% /sys/fs/cgroup
/dev/mmcblk0p1 258095 54359 203737 22% /boot
/dev/sda1 976700872 107192 974500840 1% /media/if-ncp-ssd1
tmpfs 399972 0 399972 0% /run/user/1000

But my biggest frustration is that I donā€™t understand why Nextcloud is broken and comes back with that ā€œYour data directory is invalid Ensure there is a file called ā€œ.ocdataā€ in the root of the data directory.ā€ error. I didnā€™t touch the SSD. Even rebooting didnā€™t bring it back. NCP panel (webinterface) works, so NCP is runningā€¦

there should be a .ocdata file in your datadirectory
You can execute a search to get the location: ā€œfind / -name .ocdataā€ (the slash will means it will start with the search from root)

But first I would try to mount the old hdd again.

create a mountpoint, for example:
sudo mkdir /mnt/hdd
mount the harddrive: sudo mount /dev/sdb1 /mnt/hdd
cd /mnt/hdd
find ./ -name .ocdata (now you start searching from the current directory)
and then you can rsync data between your old and new datadirectory, make sure you start with a dryrun.

This is where it finds .ocdata:
sudo find / -name .ocdata
/home/pi/ncdata/.ocdata
/var/www/nextcloud/data/.ocdata
/media/if-ncp-ssd1/ncp-snapshots/autobackup_2020-05-09_115020/.ocdata

Very strange that itā€™s not in /media/if-ncp-ssd1/ncdata
Assuming it was there before while Nextcloud still worked, I have no idea how this disappeared.

Now mounting of /dev/sdb1 fails:

pi@nextcloudpi:~ $ ls -l /dev/disk/by-uuid/
total 0
lrwxrwxrwx 1 root root 10 May 10 11:02 0b21f39a-9702-42a0-bdfd-c7f3760e27c6 -> ā€¦/ā€¦/sda1
lrwxrwxrwx 1 root root 15 May 10 11:02 45e99191-771b-4e12-a526-0779148892cb -> ā€¦/ā€¦/mmcblk0p2
lrwxrwxrwx 1 root root 15 May 10 11:02 4BBD-D3E7 -> ā€¦/ā€¦/mmcblk0p1
lrwxrwxrwx 1 root root 10 May 11 10:06 58a4f579-a361-4a65-ab41-bddf564baf7c -> ā€¦/ā€¦/sdb1

pi@nextcloudpi:~ $ ls -l /media/
total 20
lrwxrwxrwx 1 root root 10 May 13 06:47 USBdrive -> /media/hdd
lrwxrwxrwx 1 root root 18 May 13 06:47 USBdrive1 -> /media/if-ncp-ssd1
drwxr-xr-x 2 root root 4096 May 13 06:47 hdd
drwxrwxrwx 1 root root 38 May 9 11:50 if-ncp-ssd1

pi@nextcloudpi:~ $ sudo mount /dev/sdb1 /media/hdd
mount: /media/hdd: wrong fs type, bad option, bad superblock on /dev/sdb1, missing codepage or helper program, or other error.

Check or post output of

cat /etc/fstab

and

sudo fdisk -l /dev/sdb

Depending output, Iā€™d try fsck or btrfs check. Maybe got dirty bit set due to removing drive without unmounting? Could be drive or powersource failure too.

fstab:

pi@nextcloudpi:~ $ cat /etc/fstab
PARTUUID=738a4d67-01 /boot vfat defaults 0 2
PARTUUID=738a4d67-02 / ext4 defaults,noatime 0 1
'# a swapfile is not a swap partition, no line here
'# use dphys-swapfile swap[on|off] for that

pi@nextcloudpi:~ $ sudo fstab -l /dev/sdb
sudo: fstab: command not found

sorry should be fdisk instead of fstab, typo, corrected now.

Gotcha. Already wondered whether there is a fstab commandā€¦

pi@nextcloudpi:~ $ sudo fdisk -l /dev/sdb
Disk /dev/sdb: 931.5 GiB, 1000170586112 bytes, 1953458176 sectors
Disk model: My Passport 0837
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5FFCF24A-E5A9-4098-BEBA-1B5DD67FFD9E

Device Start End Sectors Size Type
/dev/sdb1 2048 1953456127 1953454080 931.5G Linux filesystem

Do you know what the filesystem is?
Like Oliver said you could try to repair the disk, for example if it is ext4 you could use: sudo fsck /dev/sdb1.

To get nextcloud working again you could change the config.php file and make sure the datafolder is pointing at the right folder. Or change config.php to the new folder on your SSD and then untar your archive file with the command I mentioned above.

The filesystem is BTRFS on all disks

I tried btrfs check on the hdd:
pi@nextcloudpi:~ $ sudo btrfs check /dev/sdb1
Opening filesystem to checkā€¦
parent transid verify failed on 265273344 wanted 15679 found 15167
parent transid verify failed on 265273344 wanted 15679 found 15167
parent transid verify failed on 265273344 wanted 15679 found 15167
parent transid verify failed on 265273344 wanted 15679 found 15167
Ignoring transid failure
leaf parent key incorrect 265273344
ERROR: cannot open file system

What does that say. How can I fix this?

Regarding datadir: as I mentioned before Iā€™ve done a clean install as part of trying to fix this all and all worked at first. But at some point nc web interface comes back with that error, claiming data directory is invalid.
BUT my ncp webinterface shows /media/if-ncp-ssd1/ncdata
Also If I check look at /var/www/nextcloud/config/config.php, same thing:
ā€˜datadirectoryā€™ => ā€˜/media/if-ncp-ssd1/ncdataā€™,

Thatā€™s what I donā€™t get. Anyways, when I try to set datadir again in ncp webinterface, I get the following error:
[ nc-datadir ]
Nextcloud is in maintenance mode - no apps have been loaded
Error reading data directory. Is NextCloud running and configured?

My plan now:

  • do ANOTHER clean install of ncp on the SD-card
  • configure AGAIN the whole system
  • try to untar the backup from backup drive

I donā€™t know what else to do, unless you have some suggestions

Have you tried ?

btrfs check --repair /dev/sdb1

Same thing:

pi@nextcloudpi:~ $ sudo btrfs check --repair /dev/sdb1
enabling repair mode
Opening filesystem to checkā€¦
parent transid verify failed on 265273344 wanted 15679 found 15167
parent transid verify failed on 265273344 wanted 15679 found 15167
parent transid verify failed on 265273344 wanted 15679 found 15167
parent transid verify failed on 265273344 wanted 15679 found 15167
Ignoring transid failure
leaf parent key incorrect 265273344
ERROR: cannot open file system