Error after ncp-update v1.39.19

ℹ️ Support 📦 Appliances (Docker, Snappy, VM, NCP, AIO)
, , ,
1

Hi,
I thought I give the updater a try and wanted to update to the latest NCP-version. But that didn’t go so well.

Downloading updates
Performing updates
Cannot load Zend OPcache - it was already loaded
Cannot load Zend OPcache - it was already loaded
Cannot load Zend OPcache - it was already loaded
Cannot load Zend OPcache - it was already loaded
Installing nc-restore
AH00526: Syntax error on line 5 of /etc/apache2/sites-enabled/ncp.conf:
SSLCertificateFile: file '/etc/letsencrypt/live/<TLD>-0001/fullchain.pem' does not exist or is empty
Action '-k graceful' failed.
The Apache error log may have more information.
Error while applying update 1.40.0. Exiting...

(replaced TLD)

Since then my Nextcloud is unresponsive. I run NCP in a docker container behind Caddy. I already restarted the server, Caddy and the container. nc-info throws this:

Running nc-info
Gathering information...
Cannot load Zend OPcache - it was already loaded
Cannot load Zend OPcache - it was already loaded
NextCloudPi version  v1.39.19

NextCloudPi image    NextCloudPi_docker_09-20-21
OS                   Debian GNU/Linux 10. 5.10.60-meson64 (aarch64)
automount            no
USB devices          sda sdb
datadir              /data/nextcloud/data
data in SD           no
data filesystem      btrfs
data disk usage      2.1T/3.7T
rootfs usage         7.5G/29G
swapfile             none
dbdir                /data/database
Nextcloud check      error
HTTPD service        down
PHP service          up
MariaDB service      down
Redis service        up
HPB service          down
Postfix service      up
internet check       ok
port check 80        open
port check 443       open
IP                   172.17.0.2
gateway              172.17.0.1
interface            eth0
certificates         none
NAT loopback         no
uptime               14min

Cannot load Zend OPcache - it was already loaded

You should run Lets Encrypt for trusted encrypted access

You should enable automount to use your plugged in USB drive
Done. Press any key...

I had the Cannot load Zend OPcache-error before and tried to fix it, but didn’t help. the issue is (most likely) not related to that, as I have it for a while now. If anyone has a trick for that, I’m happy as well, but it’s not the major concern right now.

HPB wasn’t working before either, as I’m behind Caddy I wasn’t sure how to do that.

This comes when I try start a ncc-command:

An unhandled exception has been thrown:
Doctrine\DBALception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2002] Connection refused in /data/nextcloud/lib/private/DB/Connection.php:85
Stack trace:
#0 /data/nextcloud/3rdparty/doctrine/dbal/src/Connection.php(1486): OC\DB\Connection->connect()
#1 /data/nextcloud/3rdparty/doctrine/dbal/src/Connection.php(1014): Doctrine\DBAL\Connection->getWrappedConnection()
#2 /data/nextcloud/lib/private/DB/Connection.php(226): Doctrine\DBAL\Connection->executeQuery('SELECT * FROM `...', Array, Array, NULL)
#3 /data/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php(210): OC\DB\Connection->executeQuery('SELECT * FROM `...', Array, Array)
#4 /data/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php(287): Doctrine\DBAL\Query\QueryBuilder->execute()
#5 /data/nextcloud/lib/private/AppConfig.php(345): OC\DB\QueryBuilder\QueryBuilder->execute()
#6 /data/nextcloud/lib/private/AppConfig.php(110): OC\AppConfig->loadConfigValues()
#7 /data/nextcloud/lib/private/AppConfig.php(301): OC\AppConfig->getApps()
#8 /data/nextcloud/lib/private/legacy/OC_App.php(965): OC\AppConfig->getValues(false, 'installed_versi...')
#9 /data/nextcloud/lib/private/Server.php(682): OC_App::getAppVersions()
#10 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(155): OC\Server->OC\{closure}(Object(OC\Server))
#11 /data/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#12 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(122): Pimple\Container->offsetGet('OC\Memcache\Fac...')
#13 /data/nextcloud/lib/private/ServerContainer.php(137): OC\AppFramework\Utility\SimpleContainer->query('OC\Memcache\Fac...', false)
#14 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query('OC\Memcache\Fac...')
#15 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(177): OC\AppFramework\Utility\SimpleContainer->get('OC\Memcache\Fac...')
#16 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(155): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(OC\Server))
#17 /data/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(114): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#18 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(122): Pimple\Container->offsetGet('OCP\ICacheFacto...')
#19 /data/nextcloud/lib/private/ServerContainer.php(137): OC\AppFramework\Utility\SimpleContainer->query('OCP\ICacheFacto...', false)
#20 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query('OCP\ICacheFacto...')
#21 /data/nextcloud/lib/private/Server.php(1031): OC\AppFramework\Utility\SimpleContainer->get('OCP\ICacheFacto...')
#22 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(155): OC\Server->OC\{closure}(Object(OC\Server))
#23 /data/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#24 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(122): Pimple\Container->offsetGet('OCP\Lock\ILocki...')
#25 /data/nextcloud/lib/private/ServerContainer.php(137): OC\AppFramework\Utility\SimpleContainer->query('OCP\Lock\ILocki...', false)
#26 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query('OCP\Lock\ILocki...')
#27 /data/nextcloud/lib/private/Server.php(1990): OC\AppFramework\Utility\SimpleContainer->get('OCP\Lock\ILocki...')
#28 /data/nextcloud/lib/private/Files/View.php(119): OC\Server->getLockingProvider()
#29 /data/nextcloud/lib/private/Server.php(430): OC\Files\View->__construct()
#30 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(155): OC\Server->OC\{closure}(Object(OC\Server))
#31 /data/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#32 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(122): Pimple\Container->offsetGet('OC\Files\Node\H...')
#33 /data/nextcloud/lib/private/ServerContainer.php(137): OC\AppFramework\Utility\SimpleContainer->query('OC\Files\Node\H...', false)
#34 /data/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query('OC\Files\Node\H...')
#35 /data/nextcloud/lib/private/Server.php(1353): OC\AppFramework\Utility\SimpleContainer->get('OC\Files\Node\H...')
#36 /data/nextcloud/lib/base.php(588): OC\Server->boot()
#37 /data/nextcloud/lib/base.php(1076): OC::init()
#38 /data/nextcloud/console.php(49): require_once('/data/nextcloud...')
#39 /data/nextcloud/occ(11): require_once('/data/nextcloud...')
#40 {main}

Thanks in advance!

2

Content of ncp.conf:

Listen 4443
<VirtualHost _default_:4443>
  DocumentRoot /var/www/ncp-web
  SSLEngine on
  SSLCertificateFile /etc/letsencrypt/live/<private>-0001/fullchain.pem
  SSLCertificateKeyFile /etc/letsencrypt/live/<private>-0001/privkey.pem
  <IfModule mod_headers.c>
    Header always set Strict-Transport-Security "max-age=15768000; includeSubDomains"
  </IfModule>

  # 2 days to avoid very big backups requests to timeout
  TimeOut 172800

  <IfModule mod_authnz_external.c>
    DefineExternalAuth pwauth pipe /usr/sbin/pwauth
  </IfModule>

</VirtualHost>
<Directory /var/www/ncp-web/>

  AuthType Basic
  AuthName "ncp-web login"
  AuthBasicProvider external
  AuthExternal pwauth

  SetEnvIf Request_URI "^" noauth
  SetEnvIf Request_URI "^index\.php$" !noauth
  SetEnvIf Request_URI "^/$" !noauth
  SetEnvIf Request_URI "^/wizard/index.php$" !noauth
  SetEnvIf Request_URI "^/wizard/$" !noauth

  <RequireAll>

   <RequireAny>
      Require host localhost
      Require local
      Require ip 192.168
      Require ip 172
      Require ip 10
      Require ip fe80::/10
      Require ip fd00::/8
   </RequireAny>

   <RequireAny>
      Require env noauth
      Require user ncp
   </RequireAny>

  </RequireAll>

</Directory>
3

does this file exist? the problem seems to be your database though, check systemctl status mylqsd

4

Never used Caddy, so cant help you with that.
Maybe try to pull latest image, rm old and start a new container.
Restore from backup or -v mount volume to use same data and db.

The Zend OPcache message is a known issue, search this forum or Github, it has been solved and resolved many times, it doesn’t affect anything tho, just annoying.

5

No the file doesn’t exist anymore. I know it existed before, but I shouldn’t need it as I’m having Caddy in front…

I ran your command in the container and outside, tried even mysqld in case that’s a typo:

odroidn2:hannes:# systemctl status mysqld
Unit mysqld.service could not be found.
odroidn2:hannes:# systemctl status mylqsd
Unit mylqsd.service could not be found.
odroidn2:hannes:# docker exec -it ncp bash
root@01bc8618bcad:/# systemctl status mylqsd
bash: systemctl: command not found
6

oh you are in docker, does mysqld come back if you restart the container?

7

Yup, that’s my last resort to pull another image and see if it fixes it. But sometimes that creates even bigger issues :smiley: So I try that way first.

8

Nope, doesn’t show up after restarting the container.

9

hang on, let’s try to understand what happened first if you don’t mind, so what do you have under /etc/letsencrypt/live?

10

anything if you just try to run it manually (mysqld) or anything in the error log? /var/log/mysql/error.log?

11

~ /live is empty

12 
root@01bc8618bcad:/# mysqld
2021-10-20 16:10:17 0 [Note] mysqld (mysqld 10.3.31-MariaDB-0+deb10u1) starting as process 544 ...
13

Seems to hang up after starting mysqld. Can’t exit with CTRL-C either.

14

well, it’s not exiting or crashing, anything in the error log? you can open another terminal session and docker exec -ti nextcloudpi bash again in another terminal window

15 
ls -lR `/etc/letsencrypt/live`

if it’s empty it doesn’t make sense that the script tries to use it

16 
ERROR: 1932  Table 'information_schema.(temporary)' doesn't exist in engine
2021-10-10  6:29:36 0 [ERROR] Aborting

Warning: Memory not freed: 520
2021-10-20 16:10:17 0 [Warning] The parameter innodb_file_format is deprecated and has no effect. It may be removed in future releases. See https://mariadb.com/kb/en/library/xtradbinnodb-file-format/
2021-10-20 16:10:17 0 [Note] InnoDB: Using Linux native AIO
2021-10-20 16:10:17 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2021-10-20 16:10:17 0 [Note] InnoDB: Uses event mutexes
2021-10-20 16:10:17 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-10-20 16:10:17 0 [Note] InnoDB: Number of pools: 1
2021-10-20 16:10:17 0 [Note] InnoDB: Using generic crc32 instructions
2021-10-20 16:10:17 0 [Note] InnoDB: Initializing buffer pool, total size = 896M, instances = 1, chunk size = 128M
2021-10-20 16:10:17 0 [Note] InnoDB: Completed initialization of buffer pool
2021-10-20 16:10:17 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2021-10-20 16:10:18 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=35318922505
2021-10-20 16:10:20 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2021-10-20 16:10:20 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2021-10-20 16:10:20 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-10-20 16:10:20 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-10-20 16:10:20 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-10-20 16:10:20 0 [Note] InnoDB: 10.3.31 started; log sequence number 35318922514; transaction id 51850519
2021-10-20 16:10:20 0 [Note] InnoDB: Loading buffer pool(s) from /data/database/ib_buffer_pool
2021-10-20 16:10:20 0 [Note] Plugin 'FEEDBACK' is disabled.
2021-10-20 16:10:20 0 [Note] Recovering after a crash using tc.log
2021-10-20 16:10:20 0 [Note] Starting crash recovery...
2021-10-20 16:10:20 0 [Note] Crash recovery finished.
2021-10-20 16:10:20 0 [Note] Server socket created on IP: '127.0.0.1'.
2021-10-20 16:10:21 0 [Note] Reading of all Master_info entries succeeded
2021-10-20 16:10:21 0 [Note] Added new Master_info '' to hash table
2021-10-20 16:10:21 0 [Note] mysqld: ready for connections.
Version: '10.3.31-MariaDB-0+deb10u1'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Debian 10
2021-10-20 16:11:09 0 [Note] InnoDB: Buffer pool(s) load completed at 211020 16:11:09
2021-10-20 16:13:25 8 [Warning] Access denied for user 'ncadmin'@'localhost' (using password: YES)
2021-10-20 16:13:25 9 [Warning] Access denied for user 'ncadmin'@'localhost' (using password: YES)
2021-10-20 16:13:26 10 [Warning] Access denied for user 'ncadmin'@'localhost' (using password: YES)
2021-10-20 16:13:26 11 [Warning] Access denied for user 'ncadmin'@'localhost' (using password: YES)
17

Red flag here, did you forcefully stop or have a power cut?

18

I think that could be around the time when I did the update. Not sure though.

I did a restart of the whole server after I lost access to Nextcloud, just in case and I restarted the container multiple times, but none of that is in the logs.

19

can you run ncp-update devel and see if apache comes back up?

20 
root@01bc8618bcad:/# ncp-update devel
INFO: updating to development branch 'devel'
Downloading updates
Performing updates
Cannot load Zend OPcache - it was already loaded
Cannot load Zend OPcache - it was already loaded
Cannot load Zend OPcache - it was already loaded
Cannot load Zend OPcache - it was already loaded
Installing nc-restore
AH00526: Syntax error on line 5 of /etc/apache2/sites-enabled/ncp.conf:
SSLCertificateFile: file '/etc/letsencrypt/live/<private>-0001/fullchain.pem' does not exist or is empty
Action '-k graceful' failed.
The Apache error log may have more information.
Error while applying update 1.40.0. Exiting...

Can I remove the letsencrypt part from ncp.config?