SOLVED: Occ upgrade trouble

ℹ️ Support
, ,
1

I posted before about database troubles when upgrading through a few major versions (here). Now, I am just doing a minor upgrade and am getting errors. I must have an unorthodox database…

I’m hoping someone might be able to offer advice as to how to prevent these kinds of problems in the future.

The Events

I ran sudo -u www-data php /var/www/nextcloud/updater/updater.phar --no-interaction --no-backup
Result:

Updater run in non-interactive mode - will start "occ upgrade" now.

Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Setting log level to debug
Repair step: Repair MySQL collation
Repair info: All tables already have the correct collation -> nothing to do
Repair step: Copy data from accounts table when migrating from ownCloud
Repair step: Drop account terms table when migrating from ownCloud
Updating database schema
Exception: Database error when running migration 13000Date20170718121200 for app core
The column "scope" on table "oc_authtoken" already exists.
Update failed
Maintenance mode is kept active
Resetting log level

Updater run in non-interactive mode - will disable maintenance mode now.
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Maintenance mode disabled

Maintenance mode is disabled

So, I tried a

USE nextcloud;
ALTER TABLE oc_authtoken DROP COLUMN scope;

Followed by:

occ upgrade

But then I get

The column "uid_lower" on table "oc_users" already exists

I could keep going, but I’m not sure how much data I’m losing by continually deleting DB columns!

Is there a more sustainable way?

System info

Open here

Software versions

  • Nextcloud Server version: 31.0.2 → 31.0.4
  • Operating system and version: Ubuntu 24.04.2 LTS
  • Web server and version: Apache/2.4.58
  • PHP version: 8.3.6 (using fpm)
  • Installation method: manual
  • Are you using CloudfIare, mod_security, or similar? No

Log entries

The logs just repeat the command output above.

Configuration

Nextcloud

"system": {
    "dbtype": "mysql",
    "version": "31.0.2.1",
    "htaccess.RewriteBase": "\/",
    "dbport": "",
    "dbtableprefix": "oc_",
    "mysql.utf8mb4": true,
    "installed": true,
    "skeletondirectory": "\/var\/www\/skeleton",
    "trashbin_retention_obligation": "1, 2",
    "ldapUserCleanupInterval": "30",
    "log_type": "syslog",
    "logfile": "",
    "loglevel": 1,
    "syslog_tag": "Nextcloud",
    "enable_previews": false,
    "log.condition": {
        "apps": [
            "admin_audit"
        ]
    },
    "memcache.local": "\\OC\\Memcache\\APCu",
    "filelocking.enabled": "true",
    "memcache.locking": "\\OC\\Memcache\\Redis",
    "redis": {
        "port": 0,
        "dbindex": 0,
        "timeout": 1.5
    },
    "maintenance": true,
    "lost_password_link": "disabled",
    "simpleSignUpLink.shown": false,
    "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
    "activity_expire_days": 30,
    "files.chunked_upload.max_size": 0,
    "theme": ""
}

Apps

Enabled:
  - activity: 4.0.0
  - admin_audit: 1.21.0
  - bruteforcesettings: 4.0.0
  - cloud_federation_api: 1.14.0
  - dav: 1.33.0
  - federatedfilesharing: 1.21.0
  - files: 2.3.1
  - files_external: 1.23.0
  - files_pdfviewer: 4.0.0
  - files_sharing: 1.23.1
  - files_trashbin: 1.21.0
  - lookup_server_connector: 1.19.0
  - oauth2: 1.19.1
  - password_policy: 3.0.0
  - profile: 1.0.0
  - provisioning_api: 1.21.0
  - related_resources: 2.0.0
  - serverinfo: 3.0.0
  - settings: 1.14.0
  - text: 5.0.0
  - theming: 2.6.1
  - twofactor_backupcodes: 1.20.0
  - user_ldap: 1.22.0
  - viewer: 4.0.0
  - workflowengine: 2.13.0
Disabled:
  - app_api: 5.0.2 (installed 5.0.2)
  - circles: 31.0.0 (installed 22.1.1)
  - comments: 1.21.0 (installed 1.12.0)
  - contacts: 7.0.4 (installed 7.0.4)
  - contactsinteraction: 1.12.0 (installed 1.3.0)
  - dashboard: 7.11.0 (installed 7.2.0)
  - encryption: 2.19.0
  - federation: 1.21.0 (installed 1.12.0)
  - files_downloadactivity: 1.16.0 (installed 1.16.0)
  - files_downloadlimit: 4.0.0 (installed 4.0.0)
  - files_reminders: 1.4.0 (installed 1.4.0)
  - files_versions: 1.24.0 (installed 1.15.0)
  - firstrunwizard: 4.0.0 (installed 2.11.0)
  - logreader: 4.0.0 (installed 2.7.0)
  - nextcloud_announcements: 3.0.0 (installed 1.11.0)
  - notifications: 4.0.0 (installed 2.10.1)
  - photos: 4.0.0-dev.1 (installed 2.3.0)
  - privacy: 3.0.0 (installed 3.0.0)
  - recommendations: 4.0.0 (installed 1.1.0)
  - sharebymail: 1.21.0 (installed 1.12.0)
  - support: 3.0.0 (installed 1.5.0)
  - survey_client: 3.0.0 (installed 1.10.0)
  - suspicious_login: 9.0.1
  - systemtags: 1.21.1 (installed 1.12.0)
  - twofactor_nextcloud_notification: 5.0.0
  - twofactor_totp: 13.0.0-dev.0
  - updatenotification: 1.21.0 (installed 1.15.0)
  - user_status: 1.11.0 (installed 1.2.0)
  - weather_status: 1.11.0 (installed 1.2.0)
  - webhook_listeners: 1.2.0 (installed 1.2.0)
2

Digging in a bit further…

There is some migration code like this:

class Version31000Date20240814184402 extends SimpleMigrationStep {
        public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
                /** @var ISchemaWrapper $schema */
                $schema = $schemaClosure();

                $table = $schema->getTable('preferences');
                $table->addColumn('lazy', Types::SMALLINT, ['notnull' => true, 'default' => 0, 'length' => 1, 'unsigned' => true]);

But the addColumn throws an exception if the column exists, which stops the whole migration. This seems like a bug in a few ways…

Do I need to run these migration scripts for an upgrade from 31.0.2 → 31.0.4?

3

Current theory is that the migrater is supposed to know which migration scripts it needs to run and mine has forgotten and is running all of them. Is this possible?

4

Yes it is! Fixed by adding the necessary migration DB table. Case closed.