Server error after power outage

Yes, I just checked - I can access it through PuTTY.

Hi, not sure if youā€™re still there, Irdi, but I would really appreciate some help on this if youā€™re able. Or, if anyone else has recommendations, that would be nice as well. Thanks!

1 Like

Can you try systemctl redis status and see whats the status of the service?

Thanks. I get: Unknown operation redis

Sharing my personal experience from running my nextcloud instance on a raspberry pi, I think it is not reliable enough to host complex services. I had multiple cases of power surges, but even regular shutdowns, corrupting my filesystem.
I switched to a Thin-Client, having much more fun with a potentially even cheaper device.

Unfortunately, this wonĀ“t help you for now.

No, unfortunately, that wonā€™t help me just now - and for now, Iā€™m dead in the water :slightly_smiling_face:

Does anyone have any insight and recommendations they could offer, please? My Nextcloud has been unreachable from WAN or LAN for almost two weeks after the power outage we had. I can, however, get into the NextcloudPi panel, or access it through PuTTY. If you need any additional information, please let me know. Thanks!

Try:

sudo systemctl status redis.service

1 Like

Hi @Skyhooker - Letā€™s start with the basics by verifying the integrity of the underlying system since NC has no chance of working if there is are issues with your filesystem or drives (likely after a power outage).

  • At the command-line, run dmesg. Are there any errors? Feel free to post some of the contents (none of it should be sensitive generally, but look it over).
  • What hardware are you using?

Thank you. That command returns:

  • redis-server.service - Advanced key-value store
    Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor pre
    Active: failed (Result: exit-code) since Tue 2023-08-15 12:58:35 UTC; 19min a
    Docs: https://redis.io/docs/,
    man:redis-server(1)
    Process: 2480 ExecStart=/usr/bin/redis-server /etc/redis/redis.conf (code=exit

Aug 15 12:58:35 nextcloudpi systemd[1]: redis-server.service: Service RestartSec
Aug 15 12:58:35 nextcloudpi systemd[1]: redis-server.service: Scheduled restart
Aug 15 12:58:35 nextcloudpi systemd[1]: Stopped Advanced key-value store.
Aug 15 12:58:35 nextcloudpi systemd[1]: redis-server.service: Start request repe
Aug 15 12:58:35 nextcloudpi systemd[1]: redis-server.service: Failed with result
Aug 15 12:58:35 nextcloudpi systemd[1]: Failed to start Advanced key-value store
lines 1-13/13 (END)

Thank you. ā€œdmesgā€ produced quite a long output. I couldnā€™t find anything insecure in it, so I put it on pastebin. There are a number of lines in red, but I donā€™t know whether any are significant.

dmesg output

The hardware is an Odroid HC2: Odroid-HC2

EDIT: It just occurred to me that pastebin is monochrome text only. The following lines from dmesg are in red:

0.148307] CPU4: Spectre v2: firmware did not set auxiliary control register IBE bit, system vulnerable

0.168290] CPU5: Spectre v2: firmware did not set auxiliary control register IBE bit, system vulnerable

0.176590] CPU6: Spectre v2: firmware did not set auxiliary control register IBE bit, system vulnerable

0.184591] CPU7: Spectre v2: firmware did not set auxiliary control register IBE bit, system vulnerable

2.036025] exynos-hdmi 14530000.hdmi: Failed to get supply ā€˜vddā€™: -517

3.284427] s5p_mfc_load_firmware:73: Firmware is not present in the /lib/fir mware directory nor compiled in kernel

3.435846] i2c i2c-2: sendbytes: NAK bailout.
[ 3.467766] i2c i2c-2: sendbytes: NAK bailout.
[ 3.714333] i2c i2c-2: sendbytes: NAK bailout.
[ 3.746153] i2c i2c-2: sendbytes: NAK bailout.
[ 3.992071] i2c i2c-2: sendbytes: NAK bailout.
[ 4.023834] i2c i2c-2: sendbytes: NAK bailout.
[ 4.269196] i2c i2c-2: sendbytes: NAK bailout.
[ 4.300868] i2c i2c-2: sendbytes: NAK bailout.
[ 4.545745] i2c i2c-2: sendbytes: NAK bailout.
[ 4.577350] i2c i2c-2: sendbytes: NAK bailout.
[ 4.821502] i2c i2c-2: sendbytes: NAK bailout.
[ 4.853022] i2c i2c-2: sendbytes: NAK bailout.
[ 5.096616] i2c i2c-2: sendbytes: NAK bailout.
[ 5.128045] i2c i2c-2: sendbytes: NAK bailout.
[ 5.371054] i2c i2c-2: sendbytes: NAK bailout.
[ 5.402412] i2c i2c-2: sendbytes: NAK bailout.
[ 5.644735] i2c i2c-2: sendbytes: NAK bailout.
[ 5.676034] i2c i2c-2: sendbytes: NAK bailout.
[ 5.917695] i2c i2c-2: sendbytes: NAK bailout.
[ 5.948866] i2c i2c-2: sendbytes: NAK bailout.
[ 6.190009] i2c i2c-2: sendbytes: NAK bailout.
[ 6.221143] i2c i2c-2: sendbytes: NAK bailout.
[ 6.461555] i2c i2c-2: sendbytes: NAK bailout.
[ 6.492601] i2c i2c-2: sendbytes: NAK bailout.
[ 6.732397] i2c i2c-2: sendbytes: NAK bailout.
[ 6.763365] i2c i2c-2: sendbytes: NAK bailout.
[ 7.002410] i2c i2c-2: sendbytes: NAK bailout.
[ 7.033274] i2c i2c-2: sendbytes: NAK bailout.

7.303187] OF: graph: no port node found in /soc/hdmi@14530000

END OF EDIT

Redis nor starting is interesting since itā€™s probably the simplest of the services.

Unfortunately sometimes the status doesnā€™t give a useful error output. You might try manually to run it to see if you get a more useful error message:

/usr/bin/redis-server /etc/redis/redis.conf

My gut tells me the outage corrupted your filesystem a bit so permissions or ownership or similar are maybe wrong. From the looks of your dmesg output it appears you have a 32GB SD card + a 12TB USB-SATA (presumably external) drive.

Maybe make sure things arenā€™t in read-only mode. That can happen if the OS considers it necessary for some manual filesystem checking. Easiest way to see whatā€™s mounted and the mode is probably to run mount without any parameters.

After running the command you posted, the status command returns:

  • redis-server.service - Advanced key-value store
    Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor pre
    Active: failed (Result: exit-code) since Tue 2023-08-15 12:58:35 UTC; 6h ago
    Docs: https://redis.io/docs/,
    man:redis-server(1)
    Process: 2480 ExecStart=/usr/bin/redis-server /etc/redis/redis.conf (code=exit

Warning: Journal has been rotated since unit was started. Log output is incomple

Would it help to restart and run those commands again?

The 12TB drive is attached to the SATA port of the unit. It isnā€™t USB, though NCP lists it as such.

Hereā€™s the result of mount - looks normal to me, but as I said, Iā€™m very much a noob at this:

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=953008k,nr_inodes=187225,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=204484k,mode=755)
/dev/mmcblk1p1 on / type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup2 on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,name=systemd)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,relatime)
configfs on /sys/kernel/config type configfs (rw,relatime)
tmpfs on /tmp type tmpfs (rw,nosuid,relatime)
/dev/mmcblk1p1 on /var/log.hdd type ext4 (rw,noatime,nodiratime,errors=remount-ro,commit=600)
/dev/zram0 on /var/log type ext4 (rw,relatime,discard)
/dev/sda1 on /media/myCloudDrive type btrfs (rw,nosuid,nodev,relatime,space_cache,subvolid=5,subvol=/,uhelper=udisks2)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=204480k,mode=700)

Well, itā€™s been three weeks now since my NCP went down. It seems clear Iā€™m not going to be able to get this thing repaired as it is, so on to Plan B, which is to backup and reinstall. Whatā€™s the best way to backup the user files in preparation for a complete wipe and reinstall? I have a NAS on my network with sufficient space.

Please keep in mind that Iā€™m far from expert at this, so would appreciate help formatting necessary commands and so forth. Thanks!

One more note: this is Nextcloudpi, if that matters - Iā€™ve seen that there are other ways people install Nextcloud. Thank you.

Iā€™ve just had the same issue. Noticed that redis.conf had been touched/updated on the same day & was owned by root.
I used

chown -R redis:redis /var/log/redis
chmod -R u+rwX,g+rwX,u+rx /var/log/redis

chmod +r /etc/redis/redis.conf

via ubuntu - Failed to start Advanced key-value store.redis-server.service: Control process exited, code=exited status=1 - Stack Overflow
(There are some other options, some more ā€œnuclearā€ here: ubuntu - Failed to start Advanced key-value store.redis-server.service: Control process exited, code=exited status=1 - Stack Overflow)

& redis was able to start.

I doubt that itā€™ll help you, but Iā€™m putting this here for the next person to hit this issue.

1 Like

EVERY time I shut off power to my NextcloudPi on my RockPro64 without first doing a shutdown command of some sort, it has /var/log/redis owned by root. If I do a normal shutdown via the web interface (and give it time to process the shutdown) before turning off the power, this does not happen. I can reliably expect this to happen if I shut off the power without doing the shutdown, and since I treat this as an appliance like the router, this happens very often.

The solution, like @Linecutterx mentions, is to ssh into the device and run:
sudo chown -R redis:redis /var/log/redis and reboot the device with sudo /sbin/shutdown -r now.

Now, if I can only have this happen at boot time (the chown command) then I wonā€™t have to go through this each time. This issue did not happen at first (about three years ago), so I donā€™t know its genesis. I first noticed it a few months after starting using the image. I am running this from a 128GB SD card with an external USB 2TB drive for the data and database.

Thank you, Linecutterx, that was indeed the solution. I finally found it in an old post here several weeks ago, apparently not long before you posted, but was called away and forgot to post the solution. I appreciate it!

1 Like

i dont know if any help, but twice now after 2 unexpected power outages the :4443 ip address which started at 192.168.1.186 , has now incremented up thru 187 and now 190, on its own for some reason. after cant connect to server messages i did a networks scan and found the pi server on different ip and changed my apps and shortcuts. all back up and running