Can't log in anymore after upgrade 21.0.7 -> 22.2.3 -> 23.0.0

ℹ️ Support
1

Nextcloud version (eg, 20.0.5): 23.0.0
Operating system and version (eg, Ubuntu 20.04): Debian 8
Apache or nginx version (eg, Apache 2.4.25): Apache 2
PHP version (eg, 7.4): 7.4.3
MySQL version: 5.7.25

The issue you are facing:

My Nextcloud instance is running on shared hosting, I’m doing administration work from SSH, but have very limited access there I just upgraded first from 21.0.7 to 22.2.3 after the update server was reachable again. However after the first update the database can’t be reached anymore:

php -d memory_limit=384M updater.phar
Nextcloud Updater - version: v20.0.0beta4-11-g68fa0d4

Current version is 22.2.3.

Update to Nextcloud 22.2.3 available. (channel: "stable")
Following file will be downloaded automatically: https://download.nextcloud.com/server/releases/nextcloud-22.2.3.zip
Open changelog ↗

Steps that will be executed:
[✔] Check for expected files
[✔] Check for write permissions
[✔] Create backup
[✔] Downloading
[✔] Verify integrity
[✔] Extracting
[✔] Enable maintenance mode
[✔] Replace entry points
[✔] Delete old files
[✔] Move new files in place
[✔] Done

Continue update? [y/N] y

Info: Pressing Ctrl-C will finish the currently running step and then stops the updater.

[✔] Check for expected files
[✔] Check for write permissions
[✔] Create backup
[✔] Downloading
[✔] Verify integrity
[✔] Extracting
[✔] Enable maintenance mode
[✔] Replace entry points
[✔] Delete old files
[✔] Move new files in place
[✔] Done

Update of code successful.

Should the "occ upgrade" command be executed? [Y/n] Y
An unhandled exception has been thrown:
Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2002] Connection refused in /var/www/webXXX/html/nextcloud/lib/private/DB/Connection.php:87
Stack trace:
#0 /var/www/webXXX/html/nextcloud/3rdparty/doctrine/dbal/src/Connection.php(1519): OC\DB\Connection->connect()
#1 /var/www/webXXX/html/nextcloud/3rdparty/doctrine/dbal/src/Connection.php(1041): Doctrine\DBAL\Connection->getWrappedConnection()
#2 /var/www/webXXX/html/nextcloud/lib/private/DB/Connection.php(236): Doctrine\DBAL\Connection->executeQuery('SELECT * FROM `...', Array, Array, NULL)
#3 /var/www/webXXX/html/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php(345): OC\DB\Connection->executeQuery('SELECT * FROM `...', Array, Array)
#4 /var/www/webXXX/html/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php(287): Doctrine\DBAL\Query\QueryBuilder->execute()
#5 /var/www/webXXX/html/nextcloud/lib/private/AppConfig.php(344): OC\DB\QueryBuilder\QueryBuilder->execute()
#6 /var/www/webXXX/html/nextcloud/lib/private/AppConfig.php(109): OC\AppConfig->loadConfigValues()
#7 /var/www/webXXX/html/nextcloud/lib/private/AppConfig.php(300): OC\AppConfig->getApps()
#8 /var/www/webXXX/html/nextcloud/lib/private/legacy/OC_App.php(971): OC\AppConfig->getValues(false, 'installed_versi...')
#9 /var/www/webXXX/html/nextcloud/lib/private/Server.php(687): OC_App::getAppVersions()
#10 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(160): OC\Server->OC\{closure}(Object(OC\Server))
#11 /var/www/webXXX/html/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#12 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(127): Pimple\Container->offsetGet('OC\\Memcache\\Fac...')
#13 /var/www/webXXX/html/nextcloud/lib/private/ServerContainer.php(136): OC\AppFramework\Utility\SimpleContainer->query('OC\\Memcache\\Fac...', false)
#14 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query('OC\\Memcache\\Fac...')
#15 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(182): OC\AppFramework\Utility\SimpleContainer->get('OC\\Memcache\\Fac...')
#16 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(160): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(OC\Server))
#17 /var/www/webXXX/html/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(114): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#18 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(127): Pimple\Container->offsetGet('OCP\\ICacheFacto...')
#19 /var/www/webXXX/html/nextcloud/lib/private/ServerContainer.php(136): OC\AppFramework\Utility\SimpleContainer->query('OCP\\ICacheFacto...', false)
#20 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query('OCP\\ICacheFacto...')
#21 /var/www/webXXX/html/nextcloud/lib/private/Server.php(1064): OC\AppFramework\Utility\SimpleContainer->get('OCP\\ICacheFacto...')
#22 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(160): OC\Server->OC\{closure}(Object(OC\Server))
#23 /var/www/webXXX/html/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#24 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(127): Pimple\Container->offsetGet('OCP\\Lock\\ILocki...')
#25 /var/www/webXXX/html/nextcloud/lib/private/ServerContainer.php(136): OC\AppFramework\Utility\SimpleContainer->query('OCP\\Lock\\ILocki...', false)
#26 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query('OCP\\Lock\\ILocki...')
#27 /var/www/webXXX/html/nextcloud/lib/private/Server.php(2024): OC\AppFramework\Utility\SimpleContainer->get('OCP\\Lock\\ILocki...')
#28 /var/www/webXXX/html/nextcloud/lib/private/Files/View.php(118): OC\Server->getLockingProvider()
#29 /var/www/webXXX/html/nextcloud/lib/private/Server.php(435): OC\Files\View->__construct()
#30 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(160): OC\Server->OC\{closure}(Object(OC\Server))
#31 /var/www/webXXX/html/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#32 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(127): Pimple\Container->offsetGet('OC\\Files\\Node\\H...')
#33 /var/www/webXXX/html/nextcloud/lib/private/ServerContainer.php(136): OC\AppFramework\Utility\SimpleContainer->query('OC\\Files\\Node\\H...', false)
#34 /var/www/webXXX/html/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query('OC\\Files\\Node\\H...')
#35 /var/www/webXXX/html/nextcloud/lib/private/Server.php(1387): OC\AppFramework\Utility\SimpleContainer->get('OC\\Files\\Node\\H...')
#36 /var/www/webXXX/html/nextcloud/lib/base.php(594): OC\Server->boot()
#37 /var/www/webXXX/html/nextcloud/lib/base.php(1089): OC::init()
#38 /var/www/webXXX/html/nextcloud/console.php(48): require_once('/var/www/webXXX...')
#39 /var/www/webXXX/html/nextcloud/occ(11): require_once('/var/www/webXXX...')
#40 {main}
Keep maintenance mode active? [y/N] y
Maintenance mode kept active

If tested it with PHP version 7.4.3 as well as 8.0.1 and I can log in to MySQL via

mysql -u webXXX -p

So the issue shouldn’t be on the MySQL side.

Update:
A manual

php -d memory_limit=384M occ upgrade

call worked out fine. No clue why. However the upgrader now finds NC 23.0.0 but when leaving the maintenance mode I get an error message on the webend:

Internal Server Error
The server was unable to complete your request.

If this happens again, please send the technical details below to the server administrator.

More details can be found in the server log.

Technical details
Remote Address: X.X.X.X
Request ID: x9M7TSBxv834sDiEYgu3

Update 2:
I now updated to version 23.0.0 and at first glance in the backend everything is fine now:

php -d memory_limit=384M updater.phar
Nextcloud Updater - version: v20.0.0beta4-11-g68fa0d4

Current version is 23.0.0.

No update available.

Nothing to do.

The backend also looks fine at first glance, but whenever someone tries to log in with correct credentials the above “Internal server error” is returned.

Trying to find the issue I watched the log while trying to log in. The log is pasted here: https://pastebin.com/X3g6sdt2

The error seems to be similar as after the first update attempt above:

  Error    index              Doctrine\DBAL\Exception\DriverException: An exception 2022-01-25T21:27:17+00:00
                              occurred while executing a query: SQLSTATE[HY001]:
                              Memory allocation error: 1038 Out of sort memory,
                              consider increasing server sort buffer size at
                              .../dbal/src/Driver/API/MySQL/ExceptionConverter.php
                              line 114

What can I do here? I don’t have access to any deeper server settings and the maximum memory I get for PHP scripts is 384 MB. However I would think that that should be enough to query the database?

2

Hi @Eriksson

the error messages you are getting now do not seem to be connected to the database, at least in my experience if the database could not be reached, there is no request Id generated and shown. Could you check the Nextcloud log files for the error associated to the request ID?

/S

3

Ah, you were quicker…

the issue is not PHP memory but MySQL memory as far as I understand the error message. Maybe this helps?

/S

4

Sorry, I won’t update the above posting anymore - just did it as cleaner way until I had the first answer. The problem is that I can’t change any global database settings, since I have limited access to the server (shared hosting). I can see that the buffer size is not specified there, but I can’t change it.

5

Hi @Eriksson

sorry, I must have misinterpreted the “shared hosting” bit - so your database is on a shared MySQL server instance and you cannot change its configuration. Can you actually see what the relevant setting is, e.g. via SHOW GLOBAL STATUS;?

/S

6

Hi @simonspa
thanks, yes, I can execute SHOW GLOBAL STATUS, however it provides a lot of information, here’s everything that is sort-related:

| Sort_merge_passes                             | 20786398                                                                                                                                                                                                                                    |
| Sort_range                                    | 1512982874                                                                                                                                                                                                                                  |
| Sort_rows                                     | 152420683459                                                                                                                                                                                                                                |
| Sort_scan                                     | 387266600

I’ve read that the sort_merge_passes seems to be related to the sort buffer size. But I have no clue whether the value should be fine or not.

In parallel I also contacted my Hoster whether he could increase the sort buffer size. However I would be a bit surprised if he would change anything.

7

Dang. My hoster just told me that NC >= 22 required MySQL 8 which won’t be available on the server before Q2/22. So I would have to stick with NC 21 for now despite EOL in 02/22.

Is there any other way? Or do I really have to roll back to NC 21 via backups? Would be a pity.

8

I finally rolled back to NC 21 since there is no way to get MySQL 8.0 or MariaDB 10.2 on my webspace :frowning:
Waiting for my hoster to change that and hoping that NC 21 still works fine until then despite reaching EOL soon…