Permission denied on ncp-report, ncp-backup, ncp-update etc

ℹ️ Support 📦 Appliances (Docker, Snappy, VM, NCP)
#1

Suddenly my NextcloudPI (1.24.0) stopped working, probably after an auto-update being applied. This is actually a docker image running on an OMV4, running atop a RasPi4. “Stopped working” here means: database down, apache down, redis down, php down, so no access via the web at all.

When I tried to trouble-shoot the issue, I ran into the following:

root@558aaa988200:~# whoami 
root
root@558aaa988200:~# cd /usr/local/bin/
root@558aaa988200:/usr/local/bin# ls
freedns.sh   ncp                ncp-check-version  ncp-dist-upgrade               ncp-provisioning.sh  ncp-scan          ncp-update           noip2
nc-previews  ncp-backup         ncp-config         ncp-notify-unattended-upgrade  ncp-report           ncp-suggestions   ncp-update-nc
ncc          ncp-check-updates  ncp-diag           ncp-notify-update              ncp-restore          ncp-test-updates  nextcloud-domain.sh
root@558aaa988200:/usr/local/bin# ls -l ncp-report
-rwxr-xr-x 1 root root 1917 Mar 26  2020 ncp-report
root@558aaa988200:/usr/local/bin# ./ncp-report
bash: ./ncp-report: Permission denied

Same happens for all the scripts there. I could run ncp-report via bash (i.e. bash ncp-report) and saw that all major services (db, apache etc.) were down. I was able to start all services manually via /etc/init.d/ scripts and could then also see (after logging in) in the admin console that nc-check is giving an okay.

However, I still can’t trigger a backup or an upgrade because of the shown permission issue. I also see the following in the container logs (after triggering a restart of the image in OMV), which looks like I still have an issue:

/run-parts.sh: line 6: /etc/services-enabled.d/020nextcloud: Permission denied
/run-parts.sh: line 6: /etc/services-enabled.d/010lamp: Permission denied
/run-parts.sh: line 6: /etc/services-enabled.d/000ncp: Permission denied
/run-parts.sh: line 42: /etc/services-enabled.d/000ncp: Permission denied
/run-parts.sh: line 47: /etc/services-enabled.d/010lamp: Permission denied
/run-parts.sh: line 47: /etc/services-enabled.d/020nextcloud: Permission denied

I was suspecting an issue with mount options, but I don’t see a “no exec” on the root overlay:

root@558aaa988200:~# mount | head -1
overlay on / type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay2/l/5RRL7ASEWQKYQFNHDHQGAH235J:/var/lib/docker/overlay2/l/B5ZDXA4W54I3PZMBXP76OLC4N3:/var/lib/docker/overlay2/l/G5SSIFEKMQJNKXC2XWR6WSACBA:/var/lib/docker/overlay2/l/LILDXKG6FBYAMXLW2Y2JZYJJBN:/var/lib/docker/overlay2/l/3GFMCRZGT3NMV2SM6PIDFEBLB4:/var/lib/docker/overlay2/l/T3ZZNN2DQ5W47DXRWJWE2W7PDC

Update

I just recognized that apparently the relevant directories are on /data, which is mounted no-exec:

root@558aaa988200:/usr/local/bin# df .
Filesystem     1K-blocks      Used Available Use% Mounted on
/dev/sda1      477753232 451312800   2102256 100% /data
root@558aaa988200:/usr/local/bin# mount | grep data
/dev/sda1 on /data type ext4 (rw,noexec,relatime,discard,stripe=8191,jqfmt=vfsv0,usrjquota=aquota.user,grpjquota=aquota.group)

Has maybe changed something here and what would be the appropriate corrective action to take?

Thanks in advance,

Holger

#2

The storage is full. Please delete first a lot of files.
Then boot the system. Perhaps than all is fine.

find big files in /data
find /data -size +100M
(set the size to +100M, +50M, +10M, …)

The problem is “noexec”.

If after boot it is still not working, you can test:

mount /data -o remount,exec

#3

Thanks for the reply, but the storage is not the culprit here. I freed up some space and restarted the docker image:

Filesystem     1K-blocks      Used Available Use% Mounted on
/dev/sda1      477753232 388945052  64470004  86% /data
root@558aaa988200:/# mount | grep data
/dev/sda1 on /data type ext4 (rw,noexec,relatime,discard,stripe=8191,jqfmt=vfsv0,usrjquota=aquota.user,grpjquota=aquota.group)
root@558aaa988200:/# mount /data -o remount,exec
mount: /data: permission denied.

So apparently I can’t remount. :thinking: What is also unclear to me is why is this mounted as noexec in the first place? I don’t even see any entries in /etc/fstab, so it’s not obvious to me where to look next.

#4

Okay, found the culprit: the issue was that the partition was suddenly now mounted as noexec in OMV (i.e. outside Nextcloud, which provides the volume bound to the docker image).

Hadn’t checked that before, so no NextcloudPi issue at all. Sorry about the noise.
Now I just need to figure out why the config suddenly changed in OMV …

Kind regards,

Holger

1 Like