NextCloudPi - recover SD crash and permanent defect

ℹ️ Support 📦 Appliances (Docker, Snappy, VM, NCP, AIO)
1

Hi,

I have been using the following configuration to run a NextCloudPi instance on a Raspberry Pi v3B+

  • Raspberry Pi v3B+
  • NCP v1.26.2 was running
  • SSD via USB for folder and file storage

After some months of flawless operation, we had a mains power defect in the house. Here is what happened and what I did so far after the SD card (hosting NCP) did not seem to have recovered the mains power drop:

  • an attempt to connect via browser just said something like “service not available”
  • so, I unplugged the RaspberryPi power, waited some seconds and re-plugged power
  • doing so, only the red power LED constantly shines
  • I unplugged power, got myself a working SD card from another RPi v3B+, plugged it in, re-plugged power -> RPi works.
  • my suggestion, SD is permanently broken

Now, all the data is still on a 240 GB SSD which was connected to the RPi.

My question: Is it possible to

  • re-install the SD card using the latest NCP image,
  • plug in the SSD (holding all the data) unto the USB,
  • do some configurations to mount the SSD and then
  • let NextCloud re-configure itself to be used again

?

What would be the standard process to get up and running again?

2

If you have made backups you can use nc-restore

Sorry but NCP or Nextcould does not re-configure itself, but you can use ncp to restore a previously backed-up instance.

If not, you can start with a new install, re-create users, mount your ncdata, using uuid in fstab , edit config to point to ncdata on ssd and scan files to rebuild database.

3

Hi @OliverV,

thanks for your reply. I started installing NCP from scratch on a new SD card, now.

You mentioned some activities to be performed in order to connect NCP to the existing file structure on my SSD. Just some questions for me to avoid messing things up:

new install
done.

re-create users
here, I would log into NextCloud with my ncp (admin) user and use the standard NC-GUI for re-creating the user base, right?

mount ncdata
this I will have to do using the console on the NC server, right? identification of correct UUID for my SSD and setting an appropriate entry within fstab. or is there a way to do it using the NCP-GUI?

edit config
you mean the NC config at [ <install> ] / [ config ] / config.php and the datadirectory setting to point to the [ ncdata ] folder on my SSD?

scan files
is this something I have to trigger manually? then, how can I do it using NCP-GUI? or via NC server console? is there a special command to do so? as I don’t want to accidentally mistake the desired scan with an init…

4

observations about another NCP installation which is working properly (just as an example to find a way to get my defective system to work again):

I’m trying to find and mount my SSD (containing the data I want to re-cover). So, I looked into another NextCloudPi installation and found that even though the SSD is working properly as data storage for the NCP system, I can not find the mount configuration within that system:

lsblk on the working system gives

NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0 223,6G  0 disk
└─sda1        8:1    0 223,6G  0 part
mmcblk0     179:0    0  14,9G  0 disk
├─mmcblk0p1 179:1    0  43,9M  0 part /boot
└─mmcblk0p2 179:2    0  14,9G  0 part /

this clearly shows dev/sda1 to be the device I’m looking for. And within my config.php I see the following row as being correct and operational:

sudo more /var/www/nextcloud/config/config.php gives

...
  'datadirectory' => '/media/SSD240GB/ncdata',
...

When I look into /media/SSD240GB/ncdata/, I see the known structures NC stores for user data.

I would now expect to see some kind of mount connection between /dev/sda1 and /media/SSD240GB but both, mount and cat /etc/fstab don’t show any hint about the already working connection:

mount gives

/dev/mmcblk0p2 on / type ext4 (rw,noatime)
devtmpfs on /dev type devtmpfs (rw,relatime,size=469544k,nr_inodes=117386,mode=755)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
mqueue on /dev/mqueue type mqueue (rw,relatime)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/mmcblk0p1 on /boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=94828k,mode=700,uid=1000,gid=1000)

which doesn’t contain /dev/sda1 and

cat /etc/fstab gives

PARTUUID=df45002e-01  /boot           vfat    defaults          0       2
PARTUUID=df45002e-02  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that

the latter only shows references to the SD card within the RPi.

How can I mount (connect my SSD with the media folder) in the way NextCloud does it? How does NextCloud mount? NextCloud doesn’t seem to use neither mount nor fstab.

5

Yes to all
To scan files you can use ncp-config>TOOLS>nc-scan or use ncp-web:4443 to run it from there.

Mounting an external drive

Preparations:

  • Create a mountpoint for the external drive and ncdata
sudo mkdir /media/ncp
  • Find UUID of external drive
ls -lh /dev/disk/by-uuid/

(Mine looks like this)
total 0
lrwxrwxrwx 1 root root 15 Aug 11 09:10 29075e46-f0d4-44e2-a9e7-55ac02d6e6cc → …/…/mmcblk0p2
lrwxrwxrwx 1 root root 10 Aug 11 09:10 35e98a80-8895-41f4-93e7-39940a1fda70 → …/…/sda1
lrwxrwxrwx 1 root root 15 Aug 11 09:10 9304-D9FD → …/…/mmcblk0p1

  • Add a ligne to /etc/fstab
sudo nano /etc/fstab

add line for your drive (copy/paste your drive’s UUID and change ext4 to btrfs if needed):

UUID=YourDrive’sUUID /media/ncp ext4 rw,users 0 0
  • mount the drive
sudo mount -a

Check if mounted

df -h

6

Thanks a lot, @OliverV,

this is what I did to now be able to operate my service again, after mounting the external drive as you described using the NC server console:

  • set up the users identical to the times before the crash. usinf a new default password and notifying each individual user to try and re-set his password again.
  • scan the files within the user-dependent folders below [ <SSD-drive> ] / [ ncdata ] / [ <user> ] / [ files ] on a one-by one basis as prompted by the scan tool
  • re-set the permissions for users / groups on the files / folders

…and, now I activated the backup features provided by NC. Next time, a recovery might take some miniutes, not hours :wink: . A pitty that some SD cards are not reliable enough to base an important service upon them.

you rescued me!

7

Some suggestions to prevent a lot of hustle next time:
Enable in ncp pannel:
Automatic Backups
Automatic Snapshots
Move your DB to the ssd as well, as that generates lots of write access to the sdcard
Enable zRam instead of swap file for the same reason

If you have this set up, you could just plug your ssd to a new device/installation, use nc restore to restore everything, delete the old database folder from the ssd and move the current db back there again. Then just enable all the things in nc pannel again and request a fresh letsencrypt certificate and you are golden…

8

Thanks @FadeFx .

I now enabled all the features you listed, now. As well as auto-updates for NCP itself. Hopefully, my system will now be up and running until I decide to manually shut it down.

One thing, though, I have not managed to enable: letsencrypt. It doesn’t seem to like my system configuration:

  • I have multiple NCPs in my LAN and
  • each of them needs a set of unique port numbers for HTTPS / HTTP as otherwise my router is not able to determine the target for the incoming packets within my LAN
  • letsencrypt requires ports 443 / 80 to work properly
  • so, only the NCP which has 443 / 80 port settings will have the opportunity to be equipped with the letsencrypt certificate
  • all other NCPs will get an “insecurity warning” when trying to connect from the outside.

So, I have to decide which target gets the certificate by configuring its ports accordingly.

Maybe there is a solution for providing certificates for LAN devices using multiple sets of port numbers apart from 443 / 80. But after days of searching and trying, I gave up on this.

9

Unfortunately i also have no idea on how to handle that automatically. Letsencrypt definitely needs those two ports, to renew a certificate you would need to configure your router to portforward to the machine needed, then do the certificare renewal and revert the portforwarding afterwards.

10

Regarding updates id suggest to also enable nc-autoupdate-nc
nc-update-nc-apps-auto
To stay on a recent and tested version of nextcloud. This way you have a worry free, always up to date system, especially if you have multiple instances to manage this saves you a lot of work