NCP: RasPi Login problems / no space left on device

ncp

#1

Hi everyone,

I use a RasPi 3 B+ with Berryboot installed on a 16GB microSD to mount the image NextCloudPi_RPi-Berryboot_07-03-18.img from an external 1.4TB HDD and it worked for a couple of months.
Twice I had to manually disable maintenance mode after an NCP update (that way), the last time was some 2 weeks ago. A little after that my Windows desktop and Android clients stopped working. Via SSH a did a ncp-info and discovered that

  1. the MariaDB service was down and
  2. 1.4TB/1.4TB of my external USB drive were in use.

I had no time to look into it back then and now I’m paying the price.

For a week now SSH access isn’t working either. I attached my Pi to a screen + keyboard and everytime during boot a couple of services fail (it’s different every time, sometimes the HTTP server, sometimes the time sync service, always MariaDB and “Advanced key-value store”). Then I get one of the following errors:

  1. Booting stops at some point. I get some [ OK ] … or [FAILED] … and nothing more happens
  2. I get stuck at the login prompt: I can enter anything including line breaks, everything I type is displayed at the screen. The password prompt never pops up but about 15s after hitting return (I can still type after that), the login prompt re-appears (see attached screenshot)
  3. I get the password prompt, but instead of logging in it says something like “error initializing module” and I get the login prompt again.
  4. I managed to properly login only once and found the following:
    a) any command in ncp-config would only give me the attached error message but appearantly wouldn’t be executed
    b) I tried to free up some space (as suggested in NCP Issue #201 ) but the Pi would tell me that “apt-get” was an unknown command and the remove command didn’t work either
    c) I tried “ls” in root and it would give me an error and the info that there are 0 entries.

I’m sorry that I can’t even tell you my NCP or NC version, since I can’t get that login working again (and I forgot to document it when it worked that one time, shame on me). I assume it’s pretty much up to date though, because i had the unattended auto updates activated.

I would be thankful for any suggestions! :slight_smile:
I’d rather not do a full reset (it took forever to backup all my photos), but if I had to I would at least like to understand what caused the problem to not run into it in another 3 months …

Merry Christmas and thanks a lot,
lotiolentus

error message after any command in ncp-config

raspi login problem


#2

Looks like quickest route might be a fresh install.

Do you know if you had nc-autobackup, nc-autoupdate and nc-autoupdate-nextcloud enabled ?

I’d try and use another pc to move some files/folders from your 1.4T data drive to another drive, to make some room, or at least check the drive and files on there. Best would be take a new sd-card with fresh berryboot and larger (2Tb or more) external and use the latest image to make a fresh start, import/copy files from old 1.4T and use occ to scan the files.

Are you on Linux Mac or Windows?


#3

OK, thanks! Then I guess I’ll do just that.

nc-autoupdate and nc-autoupdate-nextcloud were enabled, not sure about nc-autoupdate but I might have enabled it upon setup. Edit1: I mean nc-autobackup of course, not sure if that was enabled.
I use Windows 10. Best way to access ext4 would be Linux on a VirtualBox maybe?

Since I don’t have a larger drive at hand, I will probably have to delete some data. What would be the safest thing to delete? Some user data folder? File version history? Where could I find that?

Thanks for your help!

Edit2:
Any idea what caused this behaviour? Did I forget something during setup? To reserve some space for administrational tasks? Nextcloud choking itself to death on user data is something that should really not happen, am I right?


#4

Yes, but not through VirtualBox tho i suspect, just download a GNU/Linux Live CD/DvD iso, burn it to disk or USBstick and boot any pc/laptop you own, is what i would do, Knoppix should do fine. But there must be programs that once installed allow a windows system to read from ext file systems. I do not run MSwindows myself so I’d duckduckgo that one :wink:

Look for backup files, if any, you’ll want to keep them, they will look like:

/path/to/ncp-backups/nextcloud-bkp_20181003_1538561948.tar.gz (.gz only if compressed)

As to ‘the behaviour’: When a disk is full, the OS cant do basic tasks anymore, it is why you can set quota’s per user while their accounts are created!

Wouldnt be bad idea tho, to have an alarm triggered by; disk space getting dangerously low,


#5

I’d try one thing first. Use a liveboot image of linux on your desktop computer. On this system you have access to external linux file systems. This way, you can check the file system of your RPi-devices (sd card and external disk), and you can perhaps delete some files to have some free space.

On your data disk, it is up to you what to delete, if there is a problem of free storage. I think more likely and more critical is the sd card with the operating system. Check the log files (/var/log/*) and the size of them. Often, when you have problems, they grow quite large. Make a copy on a different USB stick (in case you need them later) and empty them (don’t delete them, if they are missing this can perhaps cause other problems).


#6

No so, in this case, berryboot has presumably been used to install the OS on the external drive. That is what berryboot is used for. The sd card will be used at boot time only, and its first partition mounted as /boot. The1.4T external drive will be mounted as /

So when using a live system, the log files will most likely be found at:

/media/livesessionuser/driveLabel/var/log/


#7

Thank, then everything is on the disk. Setting up different partitions for system and data is a bit more difficult and you have to think about the design and sizes, but in such a case, you won’t end up with a unusable system like this.


#8

Thanks guys! I’m rolling again.
Trying the partition approach this time :slight_smile: