output of ncp-report:
NextCloudPi diagnostics
NextCloudPi version v1.39.6
distribution Armbian 21.08.2 Buster \l
automount no
USB devices sdc
datadir /data/ncdata
data in SD yes
data filesystem btrfs
data disk usage 67G/2.8T
rootfs usage 7.8G/104G
swapfile /dev/sda2
dbdir /var/lib/mysql
Nextcloud check ok
Nextcloud version 21.0.4.1
HTTPD service up
PHP service up
MariaDB service up
Redis service up
HPB service down
Postfix service up
internet check ok
port check 80 open
port check 443 open
IP ***REMOVED SENSITIVE VALUE***
gateway ***REMOVED SENSITIVE VALUE***
interface eth0
certificates ***REMOVED SENSITIVE VALUE***
NAT loopback no
uptime 1day
Nextcloud configuration
{
"system": {
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": {
"0": "localhost",
"11": "62.251.101.202",
"1": "192.168.178.59",
"5": "nextcloudpi.local",
"7": "nextcloudpi",
"8": "nextcloudpi.lan",
"20": "adacloud",
"21": "cloud.ada.gallery",
"3": "localhost"
},
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "mysql",
"version": "21.0.4.1",
"overwrite.cli.url": "https:\/\/localhost\/",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"mysql.utf8mb4": true,
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"instanceid": "***REMOVED SENSITIVE VALUE***",
"memcache.local": "\\OC\\Memcache\\Redis",
"memcache.locking": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 0,
"timeout": 0,
"password": "***REMOVED SENSITIVE VALUE***"
},
"tempdirectory": "\/data\/ncdata\/tmp",
"mail_smtpmode": "sendmail",
"mail_smtpauthtype": "LOGIN",
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"preview_max_x": "2048",
"preview_max_y": "2048",
"jpeg_quality": "60",
"overwriteprotocol": "https",
"htaccess.RewriteBase": "\/",
"trusted_proxies": "***REMOVED SENSITIVE VALUE***",
"loglevel": "2",
"log_type": "file",
"maintenance": false,
"logfile": "\/data\/ncdata\/nextcloud.log",
"mail_sendmailmode": "smtp"
}
}
HTTPd logs
[Mon Sep 20 14:52:29.994345 2021] [ssl:error] [pid 2903:tid 281473281574256] AH02604: Unable to configure certificate localhost:4443:0 for stapling
[Mon Sep 20 14:52:29.995774 2021] [mpm_event:notice] [pid 2903:tid 281473281574256] AH00489: Apache/2.4.38 (Debian) OpenSSL/1.1.1d configured -- resuming normal operations
[Mon Sep 20 14:52:29.995821 2021] [core:notice] [pid 2903:tid 281473281574256] AH00094: Command line: '/usr/sbin/apache2'
[Mon Sep 20 14:59:27.137583 2021] [core:warn] [pid 2903:tid 281473281574256] AH00045: child process 24470 still did not exit, sending a SIGTERM
[Mon Sep 20 14:59:27.137768 2021] [core:warn] [pid 2903:tid 281473281574256] AH00045: child process 24473 still did not exit, sending a SIGTERM
[Mon Sep 20 14:59:29.139948 2021] [core:warn] [pid 2903:tid 281473281574256] AH00045: child process 24470 still did not exit, sending a SIGTERM
[Mon Sep 20 14:59:29.140087 2021] [core:warn] [pid 2903:tid 281473281574256] AH00045: child process 24473 still did not exit, sending a SIGTERM
[Mon Sep 20 14:59:31.142271 2021] [core:warn] [pid 2903:tid 281473281574256] AH00045: child process 24470 still did not exit, sending a SIGTERM
[Mon Sep 20 14:59:31.142413 2021] [core:warn] [pid 2903:tid 281473281574256] AH00045: child process 24473 still did not exit, sending a SIGTERM
[Mon Sep 20 14:59:33.144589 2021] [core:error] [pid 2903:tid 281473281574256] AH00046: child process 24470 still did not exit, sending a SIGKILL
[Mon Sep 20 14:59:33.145110 2021] [core:error] [pid 2903:tid 281473281574256] AH00046: child process 24473 still did not exit, sending a SIGKILL
[Mon Sep 20 14:59:34.147062 2021] [mpm_event:notice] [pid 2903:tid 281473281574256] AH00491: caught SIGTERM, shutting down
[Mon Sep 20 14:59:34.361059 2021] [ssl:warn] [pid 25564:tid 281473728611696] AH01909: localhost:4443:0 server certificate does NOT include an ID which matches the server name
[Mon Sep 20 14:59:34.361287 2021] [ssl:error] [pid 25564:tid 281473728611696] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: CN=adacloud / issuer: CN=adacloud / serial: 20A99445D43364A72DC2BFC94CB08157A550EA00 / notbefore: Jul 25 00:54:28 2021 GMT / notafter: Jul 23 00:54:28 2031 GMT]
[Mon Sep 20 14:59:34.361306 2021] [ssl:error] [pid 25564:tid 281473728611696] AH02604: Unable to configure certificate localhost:4443:0 for stapling
[Mon Sep 20 14:59:34.386588 2021] [ssl:warn] [pid 25565:tid 281473728611696] AH01909: localhost:4443:0 server certificate does NOT include an ID which matches the server name
[Mon Sep 20 14:59:34.386892 2021] [ssl:error] [pid 25565:tid 281473728611696] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: CN=adacloud / issuer: CN=adacloud / serial: 20A99445D43364A72DC2BFC94CB08157A550EA00 / notbefore: Jul 25 00:54:28 2021 GMT / notafter: Jul 23 00:54:28 2031 GMT]
[Mon Sep 20 14:59:34.386924 2021] [ssl:error] [pid 25565:tid 281473728611696] AH02604: Unable to configure certificate localhost:4443:0 for stapling
[Mon Sep 20 14:59:34.391112 2021] [mpm_event:notice] [pid 25565:tid 281473728611696] AH00489: Apache/2.4.38 (Debian) OpenSSL/1.1.1d configured -- resuming normal operations
[Mon Sep 20 14:59:34.391205 2021] [core:notice] [pid 25565:tid 281473728611696] AH00094: Command line: '/usr/sbin/apache2'
Database logs
2021-09-20 10:45:13 48348 [Warning] Aborted connection 48348 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got timeout reading communication packets)
2021-09-20 10:45:43 48349 [Warning] Aborted connection 48349 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got timeout reading communication packets)
2021-09-20 10:46:13 48350 [Warning] Aborted connection 48350 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got timeout reading communication packets)
2021-09-20 10:46:14 48351 [Warning] Aborted connection 48351 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got timeout reading communication packets)
2021-09-20 10:46:43 48352 [Warning] Aborted connection 48352 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got timeout reading communication packets)
2021-09-20 10:47:53 43895 [Warning] Aborted connection 43895 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got timeout reading communication packets)
2021-09-20 10:49:13 48353 [Warning] Aborted connection 48353 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got timeout reading communication packets)
2021-09-20 14:47:33 48450 [Warning] Aborted connection 48450 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got an error reading communication packets)
2021-09-20 14:47:37 48471 [Warning] Aborted connection 48471 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got an error reading communication packets)
2021-09-20 14:47:39 48472 [Warning] Aborted connection 48472 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got an error reading communication packets)
2021-09-20 14:47:42 48473 [Warning] Aborted connection 48473 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got an error reading communication packets)
2021-09-20 14:47:44 48474 [Warning] Aborted connection 48474 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got an error reading communication packets)
2021-09-20 14:47:47 48475 [Warning] Aborted connection 48475 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got an error reading communication packets)
2021-09-20 14:47:50 48476 [Warning] Aborted connection 48476 to db: 'nextcloud' user: 'ncadmin' host: 'localhost' (Got an error reading communication packets)
Nextcloud logs
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Repair::step: Repair step: Reset generated avatar flag","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"no app in context","method":"","url":"--","message":"Deprecated event type for \\OC\\Repair::step: Symfony\\Component\\EventDispatcher\\GenericEvent is used","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Repair::step: Repair step: Keep legacy encryption enabled","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"no app in context","method":"","url":"--","message":"Deprecated event type for \\OC\\Repair::step: Symfony\\Component\\EventDispatcher\\GenericEvent is used","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Repair::step: Repair step: Check encryption key format","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"no app in context","method":"","url":"--","message":"Deprecated event type for \\OC\\Repair::step: Symfony\\Component\\EventDispatcher\\GenericEvent is used","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Repair::step: Repair step: Remove old dashboard app config data","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"no app in context","method":"","url":"--","message":"Deprecated event type for \\OC\\Repair::step: Symfony\\Component\\EventDispatcher\\GenericEvent is used","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Repair::step: Repair step: Add job to cleanup the bruteforce entries","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"no app in context","method":"","url":"--","message":"Deprecated event type for \\OC\\Repair::step: Symfony\\Component\\EventDispatcher\\GenericEvent is used","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Repair::step: Repair step: Queue a one-time job to check for user uploaded certificates","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"no app in context","method":"","url":"--","message":"Deprecated event type for \\OC\\Repair::step: Symfony\\Component\\EventDispatcher\\GenericEvent is used","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Repair::step: Repair step: Repair DAV shares","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"no app in context","method":"","url":"--","message":"Deprecated event type for \\OC\\Repair::step: Symfony\\Component\\EventDispatcher\\GenericEvent is used","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Repair::step: Repair step: Add background job to set the lookup server share state for users","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:51:50+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Updater::startCheckCodeIntegrity: Starting code integrity check...","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:52:04+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Updater::finishedCheckCodeIntegrity: Finished code integrity check","userAgent":"--","version":"20.0.11.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:52:04+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Updater::updateEnd: Update successful","userAgent":"--","version":"21.0.4.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:52:04+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Updater::maintenanceDisabled: Turned off maintenance mode","userAgent":"--","version":"21.0.4.1"}
{"reqId":"n7ZLxDslKkTm7BHagGbb","level":1,"time":"2021-09-20T12:52:04+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Updater::resetLogLevel: Reset log level to Warning(2)","userAgent":"--","version":"21.0.4.1"}
This was a curl install on armbian.
Prior to the update I had to kill (9) a cron.php job that got stuck and ate all the resources. Prior to this issue I had enabled the preview generator app. I found (among other threads mentioning this issue) this one:
mentioning some permissions issue causing the preview generator to stall.
I changed to setting up systemd to run the cron jobs instead of just the crontab, which should help prevent too many cron jobs (I had the stuck one and 7 other ones when I noticed the issue)
to prevent this happening in the future.
Anyway, since then there seems to be something up with access to the database with this error message:
“Got an error reading communication packets”
I have the server setup behind a reverse proxy with SSL termination.
nc is in its standard location on a btrfs root
ncdb is similarly in its usual spot
there are no btrfs subvolumes on /
ncdata is on an external btrfs drive mounted at /data with btrfs subvolumes for ncdata, backups etc
To update I commented out the btrf check in ncp-update-nc then ran ncp-update-nextcloud from ncp-config.
(also at the end of the update there was an error message about push notifications not working due to being behind a reverse proxy but don’t think this is related to the main issue of not getting anything online due to comms errors with the database)
