Upgrade to 20.0.11 fails with exception

Nextcloud version (eg, 20.0.5): 20.0.10
Operating system and version (eg, Ubuntu 20.04): Debian 9
Apache or nginx version (eg, Apache 2.4.25): Apache 2.4.25
PHP version (eg, 7.4): 7.4

The issue you are facing:

Using the web upgrader from 20.0.10 to 20.0.11 results in the following exception during the update of the Array (passwords) app:

Doctrine\DBAL\Exception\UniqueConstraintViolationException: An exception occurred while executing 'INSERT INTO `oc_passwords_challenge` (`id`, `user_id`, `uuid`, `type`, `secret`, `server_data`, `client_data`, `created`, `updated`, `deleted`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params ["1", "username", "uuid", "PWDv1r1", "mysecret|2", "myserver_data|2", "myclient_data|2", "1614196025", "1614196025", "0"]: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1' for key 'PRIMARY'

I do have the Passwords app installed and encryption is enabled (for my user profile only). I have performed updates previously (to 20.0.10 back in May) without issue.

I have attempted dropping rows from the oc_passwords_challenge table, but then I get the same error for other tables (oc_passwords_folder, oc_passwords_folder_rv, oc_passwords_keychain, oc_passwords_password, oc_passwords_password_rv). After this, there is some other error that I wasn’t able to capture before needing to roll-back to 20.0.10.

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

  1. Use Nextcloud for a long time and with lots of apps (haha).
  2. Keep up with upgrades up through 20.0.10.
  3. Upgrade to 20.0.11.

The output of your Nextcloud log in Admin > Logging /var/www/nextcloud/data/nextcloud.log (maintenance mode prevents access to web):

{"reqId":"EaOBlhoN0zM2x0HMVu3o","level":3,"time":"2021-07-06T20:14:21+00:00","remoteAddr":"x.x.x.x","user":"--","app":"core","method":"GET","url":"/core/ajax/update.php?requesttoken=x166EDTYz%2FbEUCv0CWNb%2BES5AxeRwLWSCHn%2FstyiOwk%3D%3AhT%2FPREG8%2BIWHAESebFEMvnSSYXSilNulai%2FKhrXoTVo%3D","message":{"Exception":"Doctrine\\DBAL\\Exception\\UniqueConstraintViolationException","Message":"An exception occurred while executing 'INSERT INTO `oc_passwords_challenge` (`id`, `user_id`, `uuid`, `type`, `secret`, `server_data`, `client_data`, `created`, `updated`, `deleted`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [\"1\", \"username\", \"d1f02677-1c4d-4a23-933e-f59c2cc4dcaa\", \"PWDv1r1\", \"mysecret|2\", \"myserver_data|2\", \"myclient_data|2\", \"1614196025\", \"1614196025\", \"0\"]:\n\nSQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1' for key 'PRIMARY'","Code":0,"Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php","line":169,"function":"convertException","class":"Doctrine\\DBAL\\Driver\\AbstractMySQLDriver","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php","line":149,"function":"wrapException","class":"Doctrine\\DBAL\\DBALException","type":"::"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1071,"function":"driverExceptionDuringQuery","class":"Doctrine\\DBAL\\DBALException","type":"::"},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":248,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/apps/passwords/lib/Migration/Version20210700.php","line":1510,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/MigrationService.php","line":498,"function":"postSchemaChange","class":"OCA\\Passwords\\Migration\\Version20210700","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/MigrationService.php","line":419,"function":"executeStep","class":"OC\\DB\\MigrationService","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_App.php","line":984,"function":"migrate","class":"OC\\DB\\MigrationService","type":"->"},{"file":"/var/www/nextcloud/lib/private/Updater.php","line":379,"function":"updateApp","class":"OC_App","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Updater.php","line":258,"function":"doAppUpgrade","class":"OC\\Updater","type":"->"},{"file":"/var/www/nextcloud/lib/private/Updater.php","line":130,"function":"doUpgrade","class":"OC\\Updater","type":"->"},{"file":"/var/www/nextcloud/core/ajax/update.php","line":214,"function":"upgrade","class":"OC\\Updater","type":"->"}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php","Line":55,"Previous":{"Exception":"Doctrine\\DBAL\\Driver\\PDOException","Message":"SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1' for key 'PRIMARY'","Code":"23000","Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1062,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDOStatement","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":248,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/apps/passwords/lib/Migration/Version20210700.php","line":1510,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/MigrationService.php","line":498,"function":"postSchemaChange","class":"OCA\\Passwords\\Migration\\Version20210700","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/MigrationService.php","line":419,"function":"executeStep","class":"OC\\DB\\MigrationService","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_App.php","line":984,"function":"migrate","class":"OC\\DB\\MigrationService","type":"->"},{"file":"/var/www/nextcloud/lib/private/Updater.php","line":379,"function":"updateApp","class":"OC_App","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Updater.php","line":258,"function":"doAppUpgrade","class":"OC\\Updater","type":"->"},{"file":"/var/www/nextcloud/lib/private/Updater.php","line":130,"function":"doUpgrade","class":"OC\\Updater","type":"->"},{"file":"/var/www/nextcloud/core/ajax/update.php","line":214,"function":"upgrade","class":"OC\\Updater","type":"->"}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","Line":129,"Previous":{"Exception":"PDOException","Message":"SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1' for key 'PRIMARY'","Code":"23000","Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","line":127,"function":"execute","class":"PDOStatement","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1062,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDOStatement","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":248,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/apps/passwords/lib/Migration/Version20210700.php","line":1510,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/MigrationService.php","line":498,"function":"postSchemaChange","class":"OCA\\Passwords\\Migration\\Version20210700","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/MigrationService.php","line":419,"function":"executeStep","class":"OC\\DB\\MigrationService","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_App.php","line":984,"function":"migrate","class":"OC\\DB\\MigrationService","type":"->"},{"file":"/var/www/nextcloud/lib/private/Updater.php","line":379,"function":"updateApp","class":"OC_App","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Updater.php","line":258,"function":"doAppUpgrade","class":"OC\\Updater","type":"->"},{"file":"/var/www/nextcloud/lib/private/Updater.php","line":130,"function":"doUpgrade","class":"OC\\Updater","type":"->"},{"file":"/var/www/nextcloud/core/ajax/update.php","line":214,"function":"upgrade","class":"OC\\Updater","type":"->"}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","Line":127}},"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36","version":"20.0.11.1"}
{"reqId":"EaOBlhoN0zM2x0HMVu3o","level":3,"time":"2021-07-06T20:14:21+00:00","remoteAddr":"x.x.x.x","user":"--","app":"updater","method":"GET","url":"/core/ajax/update.php?requesttoken=x166EDTYz%2FbEUCv0CWNb%2BES5AxeRwLWSCHn%2FstyiOwk%3D%3AhT%2FPREG8%2BIWHAESebFEMvnSSYXSilNulai%2FKhrXoTVo%3D","message":"\\OC\\Updater::failure: Doctrine\\DBAL\\Exception\\UniqueConstraintViolationException: An exception occurred while executing 'INSERT INTO `oc_passwords_challenge` (`id`, `user_id`, `uuid`, `type`, `secret`, `server_data`, `client_data`, `created`, `updated`, `deleted`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [\"1\", \"username\", \"d1f02677-1c4d-4a23-933e-f59c2cc4dcaa\", \"PWDv1r1\", \"mysecret|2\", \"myserver_data|2\", \"myclient_data|2\", \"1614196025\", \"1614196025\", \"0\"]:\n\nSQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1' for key 'PRIMARY'","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36","version":"20.0.11.1"}
{"reqId":"EaOBlhoN0zM2x0HMVu3o","level":3,"time":"2021-07-06T20:14:21+00:00","remoteAddr":"x.x.x.x","user":"--","app":"updater","method":"GET","url":"/core/ajax/update.php?requesttoken=x166EDTYz%2FbEUCv0CWNb%2BES5AxeRwLWSCHn%2FstyiOwk%3D%3AhT%2FPREG8%2BIWHAESebFEMvnSSYXSilNulai%2FKhrXoTVo%3D","message":"\\OC\\Updater::updateEnd: Update failed","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36","version":"20.0.11.1"}
{"reqId":"EaOBlhoN0zM2x0HMVu3o","level":1,"time":"2021-07-06T20:14:21+00:00","remoteAddr":"x.x.x.x","user":"--","app":"updater","method":"GET","url":"/core/ajax/update.php?requesttoken=x166EDTYz%2FbEUCv0CWNb%2BES5AxeRwLWSCHn%2FstyiOwk%3D%3AhT%2FPREG8%2BIWHAESebFEMvnSSYXSilNulai%2FKhrXoTVo%3D","message":"\\OC\\Updater::maintenanceActive: Maintenance mode is kept active","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36","version":"20.0.11.1"}
{"reqId":"EaOBlhoN0zM2x0HMVu3o","level":1,"time":"2021-07-06T20:14:21+00:00","remoteAddr":"x.x.x.x","user":"--","app":"updater","method":"GET","url":"/core/ajax/update.php?requesttoken=x166EDTYz%2FbEUCv0CWNb%2BES5AxeRwLWSCHn%2FstyiOwk%3D%3AhT%2FPREG8%2BIWHAESebFEMvnSSYXSilNulai%2FKhrXoTVo%3D","message":"\\OC\\Updater::resetLogLevel: Reset log level to Debug(0)","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36","version":"20.0.11.1"}
{"reqId":"G0JiR6o4jG8Nen0tbtHB","level":3,"time":"2021-07-06T20:14:40+00:00","remoteAddr":"x.x.x.x","user":"--","app":"PHP","method":"GET","url":"/index.php/apps/passwords/api/1.0/session/keepalive","message":{"Exception":"Error","Message":"Array to string conversion at /var/www/nextcloud/lib/private/legacy/template/functions.php#40","Code":0,"Trace":[{"file":"/var/www/nextcloud/lib/private/legacy/template/functions.php","line":40,"function":"onError","class":"OC\\Log\\ErrorHandler","type":"::"},{"file":"/var/www/nextcloud/core/templates/update.admin.php","line":14,"function":"p"},{"file":"/var/www/nextcloud/lib/private/Template/Base.php","line":180,"args":["/var/www/nextcloud/core/templates/update.admin.php"],"function":"include"},{"file":"/var/www/nextcloud/lib/private/Template/Base.php","line":150,"function":"load","class":"OC\\Template\\Base","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_Template.php","line":180,"function":"fetchPage","class":"OC\\Template\\Base","type":"->"},{"file":"/var/www/nextcloud/lib/private/Template/Base.php","line":132,"function":"fetchPage","class":"OC_Template","type":"->"},{"file":"/var/www/nextcloud/lib/base.php","line":404,"function":"printPage","class":"OC\\Template\\Base","type":"->"},{"file":"/var/www/nextcloud/lib/base.php","line":963,"function":"printUpgradePage","class":"OC","type":"::"},{"file":"/var/www/nextcloud/index.php","line":37,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/lib/private/Log/ErrorHandler.php","Line":91,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36","version":"20.0.11.1"}

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'updatechecker' => false,
  'instanceid' => '***',
  'passwordsalt' => '***',
  'secret' => '***',
  'trusted_domains' =>
  array (
    0 => 'my.external.domain',
    1 => 'my.internal.domain',
  ),
  'trusted_proxies' =>
  array (
    0 => 'x.x.x.x',
    1 => 'x.x.x.y',
    2 => 'x.x.x.z',
  ),
  'datadirectory' => '/var/www/nextcloud/data',
  'overwrite.cli.url' => 'https://my.external.domain',
  'dbtype' => 'mysql',
  'version' => '20.0.11.1',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'nextcloud',
  'dbpassword' => '***',
  'installed' => true,
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'mail_smtpmode' => 'smtp',
  'mail_from_address' => 'cloud',
  'mail_domain' => 'my.mail.domain',
  'maintenance' => false,
  'loglevel' => 0,
  'theme' => '',
  'mail_smtphost' => 'my.smtp.host',
  'mail_smtpport' => '25',
  'mail_smtpdebug' => true,
  'mail_sendmailmode' => 'smtp',
  'app_install_overwrite' =>
  array (
    0 => 'calendar',
    1 => 'spreed',
    2 => 'drawio',
  ),
  'mysql.utf8mb4' => true,
  'updater.release.channel' => 'stable',
  'encryption.legacy_format_support' => false,
  'encryption.key_storage_migrated' => false,
  'updater.secret' => '***',
);

The output of your Apache/nginx/system log in /var/log/____:

x.x.x.x - - [06/Jul/2021:16:14:12 -0400] "GET / HTTP/1.1" 200 6045
x.x.x.x - - [06/Jul/2021:16:14:13 -0400] "GET /core/img/background.png?v=2 HTTP/1.1" 304 126
x.x.x.x - - [06/Jul/2021:16:14:13 -0400] "GET /core/img/favicon.ico HTTP/1.1" 304 125
x.x.x.x - - [06/Jul/2021:16:14:13 -0400] "GET /core/img/manifest.json HTTP/1.1" 304 124
x.x.x.x - - [06/Jul/2021:16:14:15 -0400] "GET /index.php/apps/photos/service-worker.js HTTP/1.1" 302 878
x.x.x.x - - [06/Jul/2021:16:14:19 -0400] "GET /core/ajax/update.php?requesttoken=x166EDTYz%2FbEUCv0CWNb%2BES5AxeRwLWSCHn%2FstyiOwk%3D%3AhT%2FPREG8%2BIWHAESebFEMvnSSYXSilNulai%2FKhrXoTVo%3D HTTP/1.1" 200 7841
x.x.x.x - - [06/Jul/2021:16:14:23 -0400] "GET /ocs/v2.php/apps/notifications/api/v2/notifications HTTP/1.1" 503 1019