Update to NC 17.0.1 failed

Hi all!
About a week ago I got an alert about NC17.0.1 was available for me.
A few hours ago I started an upgrade on my current NC 16.0.6-installation, only to fail utterly.

I had some database problems a while ago, but that was resolved after installing MariaDB and changing tables etc to utf8mb4 as per the documentation. Has been working quite nicely since then.

Anyway, the upgrade failed, I got this:

Updating <files_trashbin> …
Doctrine\DBAL\Exception\DriverException: An exception occurred while executing ‘CREATE INDEX id_index ON oc_files_trash (id)’:
SQLSTATE[HY000]: General error: 1709 Index column size too large. The maximum column size is 767 bytes.
Update failed

Any ideas what might’ve failed and why the files_trashbin app seems to have problems?
As can be see from the phpmyadmin pic-dump mb4 is used there too.

What gives? Hints appreciated as well as any for resolving this.
Thanks.

From nextcloug.log:

{“reqId”:“YSMyYEh22g614URCj3Ba”,“level”:4,“time”:“2019-11-17T12:25:10+00:00”,“remoteAddr”:"",“user”:"–",“app”:“core”,“method”:"",“url”:"–",“message”:“Failed to update database structure (PDOException: SQLSTATE[HY000]: General error: 1709 Index column size too large. The maximum column size is 767 bytes. in /var/www/html/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:88\nStack trace:\n#0 /var/www/html/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php(88): PDO->query(‘CREATE INDEX id…’)\n#1 /var/www/html/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(1010): Doctrine\DBAL\Driver\PDOConnection->query(‘CREATE INDEX id…’)\n#2 /var/www/html/nextcloud/lib/private/DB/Migrator.php(260): Doctrine\DBAL\Connection->query(‘CREATE INDEX id…’)\n#3 /var/www/html/nextcloud/lib/private/DB/Migrator.php(83): OC\DB\Migrator->applySchema(Object(Doctrine\DBAL\Schema\Schema))\n#4 /var/www/html/nextcloud/lib/private/DB/MDB2SchemaManager.php(123): OC\DB\Migrator->migrate(Object(Doctrine\DBAL\Schema\Schema))\n#5 /var/www/html/nextcloud/lib/private/legacy/db.php(188): OC\DB\MDB2SchemaManager->updateDbFromStructure(’/var/www/html/n…’)\n#6 /var/www/html/nextcloud/lib/private/legacy/app.php(903): OC_DB::updateDbFromStructure(’/var/www/html/n…’)\n#7 /var/www/html/nextcloud/lib/private/Updater.php(373): OC_App::updateApp(‘files_trashbin’)\n#8 /var/www/html/nextcloud/lib/private/Updater.php(252): OC\Updater->doAppUpgrade()\n#9 /var/www/html/nextcloud/lib/private/Updater.php(128): OC\Updater->doUpgrade(‘17.0.1.1’, ‘16.0.6.1’)\n#10 /var/www/html/nextcloud/core/Command/Upgrade.php(258): OC\Updater->upgrade()\n#11 /var/www/html/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OC\Core\Command\Upgrade->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#12 /var/www/html/nextcloud/3rdparty/symfony/console/Application.php(908): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#13 /var/www/html/nextcloud/3rdparty/symfony/console/Application.php(269): Symfony\Component\Console\Application->doRunCommand(Object(OC\Core\Command\Upgrade), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#14 /var/www/html/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#15 /var/www/html/nextcloud/lib/private/Console/Application.php(213): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#16 /var/www/html/nextcloud/console.php(97): OC\Console\Application->run()\n#17 /var/www/html/nextcloud/occ(11): require_once(’/var/www/html/n…’)\n#18 {main}\n\nNext Doctrine\DBAL\Driver\PDOException: SQLSTATE[HY000]: General error: 1709 Index column size too large. The maximum column size is 767 bytes. in /var/www/html/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:90\nStack trace:\n#0 /var/www/html/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(1010): Doctrine\DBAL\Driver\PDOConnection->query(‘CREATE INDEX id…’)\n#1 /var/www/html/nextcloud/lib/private/DB/Migrator.php(260): Doctrine\DBAL\Connection->query(‘CREATE INDEX id…’)\n#2 /var/www/html/nextcloud/lib/private/DB/Migrator.php(83): OC\DB\Migrator->applySchema(Object(Doctrine\DBAL\Schema\Schema))\n#3 /var/www/html/nextcloud/lib/private/DB/MDB2SchemaManager.php(123): OC\DB\Migrator->migrate(Object(Doctrine\DBAL\Schema\Schema))\n#4 /var/www/html/nextcloud/lib/private/legacy/db.php(188): OC\DB\MDB2SchemaManager->updateDbFromStructure(’/var/www/html/n…’)\n#5 /var/www/html/nextcloud/lib/private/legacy/app.php(903): OC_DB::updateDbFromStructure(’/var/www/html/n…’)\n#6 /var/www/html/nextcloud/lib/private/Updater.php(373): OC_App::updateApp(‘files_trashbin’)\n#7 /var/www/html/nextcloud/lib/private/Updater.php(252): OC\Updater->doAppUpgrade()\n#8 /var/www/html/nextcloud/lib/private/Updater.php(128): OC\Updater->doUpgrade(‘17.0.1.1’, ‘16.0.6.1’)\n#9 /var/www/html/nextcloud/core/Command/Upgrade.php(258): OC\Updater->upgrade()\n#10 /var/www/html/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OC\Core\Command\Upgrade->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#11 /var/www/html/nextcloud/3rdparty/symfony/console/Application.php(908): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#12 /var/www/html/nextcloud/3rdparty/symfony/console/Application.php(269): Symfony\Component\Console\Application->doRunCommand(Object(OC\Core\Command\Upgrade), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#13 /var/www/html/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#14 /var/www/html/nextcloud/lib/private/Console/Application.php(213): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#15 /var/www/html/nextcloud/console.php(97): OC\Console\Application->run()\n#16 /var/www/html/nextcloud/occ(11): require_once(’/var/www/html/n…’)\n#17 {main}\n\nNext Doctrine\DBAL\Exception\DriverException: An exception occurred while executing ‘CREATE INDEX id_index ON oc_files_trash (id)’:\n\nSQLSTATE[HY000]: General error: 1709 Index column size too large. The maximum column size is 767 bytes. in /var/www/html/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php:106\nStack trace:\n#0 /var/www/html/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php(169): Doctrine\DBAL\Driver\AbstractMySQLDriver->convertException(‘An exception oc…’, Object(Doctrine\DBAL\Driver\PDOException))\n#1 /var/www/html/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php(145): Doctrine\DBAL\DBALException::wrapException(Object(Doctrine\DBAL\Driver\PDOMySql\Driver), Object(Doctrine\DBAL\Driver\PDOException), ‘An exception oc…’)\n#2 /var/www/html/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(1012): Doctrine\DBAL\DBALException::driverExceptionDuringQuery(Object(Doctrine\DBAL\Driver\PDOMySql\Driver), Object(Doctrine\DBAL\Driver\PDOException), ‘CREATE INDEX id…’)\n#3 /var/www/html/nextcloud/lib/private/DB/Migrator.php(260): Doctrine\DBAL\Connection->query(‘CREATE INDEX id…’)\n#4 /var/www/html/nextcloud/lib/private/DB/Migrator.php(83): OC\DB\Migrator->applySchema(Object(Doctrine\DBAL\Schema\Schema))\n#5 /var/www/html/nextcloud/lib/private/DB/MDB2SchemaManager.php(123): OC\DB\Migrator->migrate(Object(Doctrine\DBAL\Schema\Schema))\n#6 /var/www/html/nextcloud/lib/private/legacy/db.php(188): OC\DB\MDB2SchemaManager->updateDbFromStructure(’/var/www/html/n…’)\n#7 /var/www/html/nextcloud/lib/private/legacy/app.php(903): OC_DB::updateDbFromStructure(’/var/www/html/n…’)\n#8 /var/www/html/nextcloud/lib/private/Updater.php(373): OC_App::updateApp(‘files_trashbin’)\n#9 /var/www/html/nextcloud/lib/private/Updater.php(252): OC\Updater->doAppUpgrade()\n#10 /var/www/html/nextcloud/lib/private/Updater.php(128): OC\Updater->doUpgrade(‘17.0.1.1’, ‘16.0.6.1’)\n#11 /var/www/html/nextcloud/core/Command/Upgrade.php(258): OC\Updater->upgrade()\n#12 /var/www/html/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OC\Core\Command\Upgrade->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#13 /var/www/html/nextcloud/3rdparty/symfony/console/Application.php(908): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#14 /var/www/html/nextcloud/3rdparty/symfony/console/Application.php(269): Symfony\Component\Console\Application->doRunCommand(Object(OC\Core\Command\Upgrade), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#15 /var/www/html/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#16 /var/www/html/nextcloud/lib/private/Console/Application.php(213): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))\n#17 /var/www/html/nextcloud/console.php(97): OC\Console\Application->run()\n#18 /var/www/html/nextcloud/occ(11): require_once(’/var/www/html/n…’)\n#19 {main})”,“userAgent”:"–",“version”:“16.0.6.1”}

[details=“Support intro”]

Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 82320
Server version: 5.5.66-MariaDB MariaDB Server

root@cyndane2:~ # php -v
PHP 7.3.11 (cli) (built: Oct 22 2019 12:34:50) ( NTS )
Copyright © 1997-2018 The PHP Group
Zend Engine v3.3.11, Copyright © 1998-2018 Zend Technologies
with Zend OPcache v7.3.11, Copyright © 1999-2018, by Zend Technologies
with Xdebug v2.8.0, Copyright © 2002-2019, by Derick Rethans

root@cyndane2:/var/www/html # httpd -v
Server version: Apache/2.2.15 (Unix)
Server built: Jun 19 2018 15:45:13

root@cyndane2:/var/www/html # cat /etc/centos-release
CentOS release 6.10 (Final)

Nextcloud 16.0.6

root@cyndane2:/var/www/html/nextcloud # sudo -u apache php occ config:list system
{
“system”: {
“passwordsalt”: “REMOVED SENSITIVE VALUE”,
“secret”: “REMOVED SENSITIVE VALUE”,
“trusted_domains”: [
“srbu.se”,
“www.srbu.se”,
“192.168.0.8”
],
“datadirectory”: “REMOVED SENSITIVE VALUE”,
“overwrite.cli.url”: “http://localhost”,
“dbtype”: “mysql”,
“version”: “16.0.6.1”,
“dbname”: “REMOVED SENSITIVE VALUE”,
“dbhost”: “REMOVED SENSITIVE VALUE”,
“dbport”: “”,
“dbtableprefix”: “oc_”,
“dbuser”: “REMOVED SENSITIVE VALUE”,
“dbpassword”: “REMOVED SENSITIVE VALUE”,
“logtimezone”: “UTC”,
“installed”: true,
“instanceid”: “REMOVED SENSITIVE VALUE”,
“mail_from_address”: “REMOVED SENSITIVE VALUE”,
“mail_smtpmode”: “sendmail”,
“mail_domain”: “REMOVED SENSITIVE VALUE”,
“maintenance”: false,
“theme”: “”,
“loglevel”: 2,
“filelocking.enabled”: true,
“memcache.local”: “\OC\Memcache\APCu”,
“memcache.locking”: “\OC\Memcache\APCu”,
“trashbin_retention_obligation”: “15, 20”,
“enable_previews”: true,
“mail_smtphost”: “REMOVED SENSITIVE VALUE”,
“mail_smtpport”: “587”,
“auth.bruteforce.protection.enabled”: false,
“mail_sendmailmode”: “smtp”,
“mail_smtpdebug”: true,
“mail_smtpauthtype”: “LOGIN”,
“mail_smtpsecure”: “ssl”,
“mail_smtpauth”: 1,
“mysql.utf8mb4”: true,
“twofactor_enforced”: “false”,
“twofactor_enforced_groups”: [],
“twofactor_enforced_excluded_groups”: [],
“app_install_overwrite”: [
“activitylog”,
“impersonate”
]
}
}
root@cyndane2:/var/www/html/nextcloud #

root@cyndane2:/var/www/html/nextcloud # sudo -u apache php occ app:list
Enabled:

  • activity: 2.9.1
  • admin_audit: 1.6.0
  • cloud_federation_api: 0.2.0
  • dav: 1.9.2
  • federatedfilesharing: 1.6.0
  • files: 1.11.0
  • files_pdfviewer: 1.5.0
  • files_rightclick: 0.15.1
  • files_sharing: 1.8.0
  • files_texteditor: 2.8.0
  • files_trashbin: 1.6.0
  • files_versions: 1.9.0
  • files_videoplayer: 1.5.0
  • gallery: 18.3.0
  • impersonate: 1.3.0
  • lookup_server_connector: 1.4.0
  • nextcloud_announcements: 1.5.0
  • notifications: 2.4.1
  • oauth2: 1.4.2
  • privacy: 1.0.0
  • provisioning_api: 1.6.0
  • serverinfo: 1.6.0
  • text: 1.0.2
  • twofactor_backupcodes: 1.5.0
  • updatenotification: 1.6.0
  • workflowengine: 1.6.0
    Disabled:
  • accessibility
  • comments
  • encryption
  • federation
  • files_external
  • firstrunwizard
  • logreader
  • password_policy
  • recommendations
  • sharebymail
  • support
  • survey_client
  • systemtags
  • theming
  • user_ldap
  • viewer
    root@cyndane2:/var/www/html/nextcloud #

root@cyndane2:/var/www/html/nextcloud # sudo -u apache php occ maintenance:repair

  • Repair MySQL collation
    • All tables already have the correct collation -> nothing to do
  • Repair mime types
  • Clean tags and favorites
    • 0 tags of deleted users have been removed.
    • 0 tags for delete files have been removed.
    • 0 tag entries for deleted tags have been removed.
    • 0 tags with no entries have been removed.
  • Repair invalid shares
  • Remove shares of a users root folder
  • Move .step file of updater to backup location
    • .step file exists
    • .step-previous-update removed
    • .step file moved to .step-previous-update
  • Fix potential broken mount points
    • No mounts updated
  • Repair invalid paths in file cache
  • Add log rotate job
  • Clear frontend caches
    • Image cache cleared
    • SCSS cache cleared
    • JS cache cleared
  • Clear every generated avatar on major updates
  • Add preview background cleanup job
  • Queue a one-time job to cleanup old backups of the updater
  • Repair pending cron jobs
    • No need to repair pending cron jobs.
  • Extract the vcard uid and store it in the db
  • Cleanup invalid photocache files for carddav
  • Add background job to cleanup login flow v2 tokens
  • Remove potentially over exposing share links
    • No need to remove link shares.
  • Clear access cache of projects
  • Cleanup cypress files from viewer app
  • Fix component of birthday calendars
    • 0 birthday calendars updated.
  • Regenerating birthday calendars to use new icons and fix old birthday events without year
    • Repair step already executed
  • Fix broken values of calendar objects
    0 [>---------------------------]
  • Registering building of calendar search index as background job
    • Repair step already executed
  • Registering background jobs to update cache for webcal calendars
    • Added 0 background jobs to update webcal calendars
  • Clean up orphan event and contact data
    • 0 events without a calendar have been cleaned up
    • 0 properties without an events have been cleaned up
    • 0 changes without a calendar have been cleaned up
    • 0 cached events without a calendar subscription have been cleaned up
    • 0 changes without a calendar subscription have been cleaned up
    • 0 contacts without an addressbook have been cleaned up
    • 0 properties without a contact have been cleaned up
    • 0 changes without an addressbook have been cleaned up
  • Remove activity entries of private events
    • Removed 0 activity entries
  • Fix the share type of guest shares when migrating from ownCloud
  • Copy the share password into the dedicated column
  • Update OAuth token expiration times
  • Add background job to check for backup codes
    root@cyndane2:/var/www/html/nextcloud #

1 Like

Yes, this is the error:

SQLSTATE[HY000]: General error: 1709 Index column size too large. The maximum column size is 767 bytes.

Using google I was able to find a fix on the second returned result.

A small hint, if you would insert a line before and after the log lines, each containing “```”, they’re better readable :wink:

I hear you. :slight_smile:
Tried cleaning ut up a bit, but the log itself is quite messed up too.

You mean this one?

I’ve tried that before, nothing happened.

You missunderstood me, I meant three backticks “```”, not multiple “*” :wink:

Mmm. We’re going about this the wrong way.

What steps have you taken to try fixing this?

Have you checked the source code on NC 17 and verified your database configuration is correct?

innodb_default_row_format=dynamic
innodb_file_format=barracuda
innodb_file_per_table=true
innodb_large_prefix=true

If I add this to /etc/my.cnf, the database won’t start, BTW.

And… what error do you see?

I restored the nc16-db from backup and fired it up again. Worked fine. Thank the gods I had them. :slight_smile:

I’m not a coder. I’d have no idea what to look for.
The database should be fine. I use it for Piwigo and Joomla too. No problems there.
Anything in particular with the db-config I should be aware of?

root@cyndane2:/var/www/html/nextcloud # service mysql restart
Shutting down MariaDB… [ OK ]
Starting MariaDB.191117 19:21:43 mysqld_safe Logging to ‘/var/log/mysqld.log’.
191117 19:21:43 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
…/etc/init.d/mysql: line 265: kill: (9319) - No such process
[FAILED]

That’s cool, you don’t need to be a programmer to understand mysql or how it’s created in php.

Now, scripting the actual code might be more of a challenge.

Glad your upgrade worked, though!

Ah, but it didn’t.

Running sudo -u apache php occ upgrade gave me this;

Updating <files_trashbin> ...
Doctrine\DBAL\Exception\DriverException: An exception occurred while executing 'CREATE INDEX id_index ON oc_files_trash (`id`)':

SQLSTATE[HY000]: General error: 1709 Index column size too large. The maximum column size is 767 bytes.
Update failed

Yay, so going back to my question - what error do you see when you fire up mysql with those changes?

I noticed you ignored the error log entirely and just posted the systemctl message…

Sorry. Misunderstood. Was putting the kids to bed at the same time. I’m a bit split ATM…

191117 19:51:46 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
191117 19:51:46 [Note] /usr/sbin/mysqld (mysqld 5.5.66-MariaDB) starting as process 16459 …
191117 19:51:46 InnoDB: The InnoDB memory heap is disabled
191117 19:51:46 InnoDB: Mutexes and rw_locks use GCC atomic builtins
191117 19:51:46 InnoDB: Compressed tables use zlib 1.2.3
191117 19:51:46 InnoDB: Using Linux native AIO
191117 19:51:46 InnoDB: Initializing buffer pool, size = 128.0M
191117 19:51:46 InnoDB: Completed initialization of buffer pool
191117 19:51:46 InnoDB: highest supported file format is Barracuda.
191117 19:51:46 InnoDB: Waiting for the background threads to start
191117 19:51:47 Percona XtraDB (http://www.percona.com) 5.5.61-MariaDB-38.13 started; log sequence number 33314883177
191117 19:51:47 [Note] Plugin ‘FEEDBACK’ is disabled.
191117 19:51:47 [ERROR] /usr/sbin/mysqld: unknown variable ‘innodb_default_row_format=dynamic’
191117 19:51:47 [ERROR] Aborting

191117 19:51:47 [ERROR] /usr/sbin/mysqld: unknown variable ‘innodb_default_row_format=dynamic’

This is the error.

This is the correct fix:

ALTER TABLE tablehere ROW_FORMAT=DYNAMIC;

Here is a guide made by mariadb for reference if the above fails:

Hmm… I’ve seen that somewhere.

The “tablehere” - what table would that be?

Hmm. Whichever table you’re having problems with.

I actually can’t say, because it’s possible you’re going to have another related error in the upgrade process once that table is fixed.

The joy of modifying your own database and deviating from the norms, no?

An exception occurred while executing ‘CREATE INDEX id_index ON oc_files_trash ( id )’:\n\nSQLSTATE[HY000]: General error: 1709 Index column size too large. The maximum column size is 767 bytes.

According to the error above, it’s likely the oc_files_trash table.
Your suggested command would then be:

ALTER TABLE oc_files_trash ROW_FORMAT=DYNAMIC;

Deviating from the defaults? I don’t think so, at least I hope not. Followed a guide, which has since vanished in the mists of memories, years ago when I first started using Owncloud and Nextcloud.
I made a point of using defaults and default settings, in order to not have any weird bugs bite my on the road later.

I’ll give this a go and report back.

Thanks for now!