Lift and shift upgrade from NC 17 to NC 24.03

Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version (eg, 20.0.5): 17.?
Operating system and version (eg, Ubuntu 20.04): Ubuntu 18.04.6 LTS
Apache or nginx version (eg, Apache 2.4.25): Apache/2.4.29 (Ubuntu)
PHP version (eg, 7.4): PHP 8.1.2 (cli) (built: Jan 24 2022

The issue you are facing:
Continuous issues trying to update this server.
Actual Nextcloud operation has been relatively trouble free otherwise.
Can I backup the database and just move it to a newly built Ubuntu 22.04 with Nextcloud 24.03?

Current error:
Doctrine\DBAL\Exception\DriverException: An exception occurred while executing ‘ALTER TABLE oc_addressbooks CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;’: SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 767 bytes

Previous error:
# Cannot declare class OCA\Talk\Migration\Version2000Date20170707093535

I keep
Is this the first time you’ve seen this error? (Y/N): I get a new error everytime I resolve one error and rerun the upgrade.

Steps to replicate it:

  1. Launch update

The output of your Nextcloud log in Admin > Logging:

server currently in maintenance

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):


<?php
$CONFIG = array (
  'instanceid' => 'ffggffbg tv gdd',
  'passwordsalt' => ‘dgbdaxxfgfxsst5)',
  'secret' => 'zzdhdvvhhnngf',
  'trusted_domains' =>
  array (
    0 => '
    1 => 'localhost',
    2 => '127.0.0.1',
    3 => 
    4 => '
    5 => '
    6 => 'files',
    7 => '',
    8 => '
  ),
  'datadirectory' => '/var/www/nextcloud/data',
  'overwrite.cli.url' => 'https://192.168.1.1/nextcloud',
'dbtype' => 'mysql',
  'version' => '17.0.10.1',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' =>
  'dbpassword' => 
  'installed' => true,
  'mail_smtpmode' => 'smtp',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_from_address' => 'cloud',
  'mail_domain' => 
  'mail_smtpauth' => 1,
  'maintenance' => false,
  'updater.release.channel' => 'stable',
  'theme' => '',
  'loglevel' => 0,
  'mail_smtphost' => 'mail.office365.com',
  'mail_sendmailmode' => 'smtp',

The output of your Apache/nginx/system log in /var/log/____:

no system log. just access.log and error.log

output from error.log:
/var/log/apache2$ more error.log
[Wed Aug 24 06:25:04.414329 2022] [ssl:warn] [pid 41522] AH01909: 127.0.1.1:443:0 serve
r certificate does NOT include an ID which matches the server name
[Wed Aug 24 06:25:04.414765 2022] [mpm_prefork:notice] [pid 41522] AH00163: Apache/2.4.
29 (Ubuntu) OpenSSL/1.1.1g configured -- resuming normal operations
[Wed Aug 24 06:25:04.414780 2022] [core:notice] [pid 41522] AH00094: Command line: '/us
r/sbin/apache2'

PASTE HERE


Output errors in nextcloud.log in /var/www/ or as admin user in top right menu, filtering for errors. Use a pastebin service if necessary.

Nothing .log in  /var/nextcloud

ummmm… you run your server locally, apparently. probably even without any valid ssl-certificate.

soooooo… it might happen that your instance can’t access neccessary files from the internet to be able doing the download and update correctly.

further on you claim that you’re running nc17.x and that you have php8.1 running. NC17 isn’t php 8.1 ready. Pls check system requirements for NC18 before updating and make sure your setup is fitting the requirements.

well you could try to make a DB backup and restore it to the DB of a newly installed NC24 or whatever. But what happens with the changes that has appeared to the structure of DB between NC 17 and every later version?

Thanks Jimmy,

On-prem install - yep

SSL - running certbot using certbot with DNS approval. The Nextcloud server is not public facing, so auto renewal is not an option. Why would SSL cert bound to httpd service make a difference for outbound system or app calls? Not discounting this point, just would like to understand it better. I may be missing something important.

php8.1 running - that is what is running. PHP was upgraded based on proposed fixes for another error received during a previous upgrade attempt. Honestly NC 17 didn’t seem to mind it much (it’s been a minute between attempts) and seemed to be relatively happy except for upgrade issues, of course. For NC 18, php version preferred is 8.0 but v8.1 is listed as supported. I forget exactly why I went to php 8.1 but I think there was some drama with getting and/or installing v8.0

structure of DB in NC17 vs NC24 and beyond? - a sage question my fine fellow, to whom should we inquire with to find an answer? For tis this question, and it’s ilk, that I am here to ask!
Looking for the greater experience and guidance of others far more knowledgeable then myself. That and the backup and restore documentation definitely have that “fare thee well brave sir, and giving you full manual control” (in Douglas Adam’s Heart of Gold kind of voice) vibe to them for this scenario.

SSL - i can’t tell you in detail becaus I dunno… I just experienced that there could arise problems from using a self-signed certificate (or none at all) - just by reading through the forum.

but this is a clear hint on a problematic ssl-certificate (or none), though.

php 8.1 - well referring to my (official) sources, NC18-NC20 aren’t even php 8.0 ready… and no, it’s not recommended either https://docs.nextcloud.com/server/18/admin_manual/installation/system_requirements.html
for NC21,22,23 php 8.0 is recommended (7.4 still supported), 8.1 not supported.
these are the facts. But you’re free to believe whatever you want to believe.

DB-structure… I doubt you’re gonna find someone here being able (or willing) telling you the different structure of DB from NC18 on, I’m afraid. If you’d do the upgrade-process as recommended you’d likely could do neccessary structure-upgrades as well right after upgrading (if it wasn’t done automatically).

On the other hand you could try yourself what would happen if you’d do it the way you want. But you better have actual backups ready to restore back… just in case

Well maybe you should document changes like this.

It’s not about drama or about preference or wishful thinking. It’s about what is supported by the respective Nextcloud version you are using.

Well there you have it. How are we supposed to support your instance, when not even you seem to know what changes were made when and why. Also you choose to ignore the documentation by deliberatly installing dependencies that deviate from the requirements that were originally stated. How would one know whether the wrong PHP version, the database or something else causes the issue with the updater.

Don’t you think that there are good reasons, why the Nextcloud documentation explicitly says not to skip versions? I mean feel free to test what happens when you upgrade straight to 24.0.3. But I would recommend you to stick to the documentation, especially if it is a production instance. https://docs.nextcloud.com/server/latest/admin_manual/maintenance/upgrade.html#

2 Likes

problematic ssl - its true that the ssl does not match the server (host) name. It does, however, match the site name.
php 8.12 - downgraded to php 7.3 via sudo update-alternatives --config php and reboted. New error:

Doctrine\DBAL\Exception\DriverException: An exception occurred while executing ‘ALTER TABLE oc_addressbooks CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;’: SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 767 bytes.

DB-structure - No doubt it’ll be a process of patience and continuing to ask questions. But don’t you always have to climb the mountain to find the guru? In regards to “If you’d do the upgrade-process as recommended you’d likely could do necessary structure-upgrades as well right after upgrading”, not to be flippant but I’m here because the upgrade process ain’t working. I’d gladly perform the update as intended.
Also, I’m going to double dip on the “you’d likely could do necessary structure-upgrades as well right after upgrading” and parse your wording a bit. You seem to be suggesting there’s a secondary process for DB structure updates that can be run separately from the upgrade process. Wondering if you could expound a bit on that? Might be able to leverage that.

Well now, that is a contribution, isn’t it?

Tried to fix the UTF8mb4 issue per https://www.tutorialrepublic.com/faq/how-to-start-stop-mysql-server-on-ubuntu.php. Completed all the steps but go the following output when I ran sudo -u www-data php occ maintenance:repair:

Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade

  • Repair MySQL collation
    • Change row format for oc_addressbooks …
    • Change collation for oc_addressbooks …

In AbstractMySQLDriver.php line 106:

An exception occurred while executing ‘ALTER TABLE oc_addressbooks CONVER
T TO CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;’:

SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was t
oo long; max key length is 767 bytes

In PDOStatement.php line 119:

SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was t
oo long; max key length is 767 bytes

In PDOStatement.php line 117:

SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was t
oo long; max key length is 767 bytes

maintenance:repair [–include-expensive]

Please check in the documentation linked below whether you have made all the required configuration changes for your particular version of MySQL or MariaDB:

https://docs.nextcloud.com/server/17/admin_manual/configuration_database/mysql_4byte_support.html#enabling-mysql-4-byte-support

sure… after each upgrade to a new major version I recommend to run
sudo -u www-data php occ db:add-missing-indices as well as
sudo -u www-data php occ db:add-missing-columns
where www-data is the webuser owning NC-files and folders (there could be different names for webuser but this one is standard)

if you want to be extra hard working check setup messages after each upgrade and try to solve these ahead of upgrading to the next version. Though it’s not mandatory… but at some point those messages should be solved. Usually they help you running your instance smoother and/or more secure.

awww and it could turn out to be helpful if you’d search the forum for hints on your problems as well since I think everything has been already discussed up and down here… :wink:

Thanks for that link.
Ran show variables like 'innodb_file_format and innodb_file_format was Antelope.
Ran SET GLOBAL innodb_file_format=Barracuda; successfully
Ran show variables like ‘innodb_file_per_table’ and it was already set to ON.
Restarted MariaDB with systemctl restart mariadb
Ran ALTER DATABASE nextcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci; with no issues.
Updated /var/www/nextcloud/config/config.php with sudo -u www-data php occ config:system:set mysql.utf8mb4 --type boolean --value=“true” also with no issues.
Errors running sudo -u www-data php occ maintenance:repair
Doesn’t matter if /var/www/nextcloud/config/config.php has “maintenance mode” set to “true” or “false” (restarting apache after it is changed each time).
I think the pending update status is blocking it.

Did you try to stop the upgrade process and the start it again from the command line…?

cd /var/www/nextcloud
sudo -u www-data php occ maintenance:mode --off
sudo -u www-data php occ upgrade

https://docs.nextcloud.com/server/latest/admin_manual/maintenance/manual_upgrade.html#troubleshooting

sudo -u www-data php occ db:add-missing-indices as well as sudo -u www-data php occ db:add-missing-columns - Thanks for that. Looks like there’s at least a “bigint” update to run occasionally as well. Maybe more. I’ll take a look.
“everything has been already discussed up and down here” - fully agreed that is the due diligence owed before posting and I have been whacking away error by error over a long period of time. But there comes a point where you are reaching death of a thousand cuts, at least in time spent. That’s pretty much where I am at. So do I double down and keep going, and may give the same amount of time ( or more) to get to NC 18? and then maybe have to devote the equal amount of time to get to 19? Hopefully not but no guarantees, because caveat emptor when it’s FOSS and community editions.
So from a pragmatic point of view, taking your data and just starting in a new install on a fully updated system and instance is generally an option in systems administration world. That’s what isn’t springing forth from the boards and prompted this entry.

I did not. Everyone references "maintenance: mode true or false in config.php. First time I am seeing sudo -u www-data php occ maintenance:mode --off to abort an upgrade.
I’ll give it a try.

i got your drift… and yes I can follow you in terms of “death of 1000 cuts” (that image made me smile somehow)

Well usually updates run pretty smoothly for the broad majority of (home)admins. At times there are certain bugs, though but usually not after x.0.2 versions.
Your setup is pretty unique and rather uncommon. That might be a reason for the problems you’re experiencing.

After you managed to get your instance up to nc18 and got rid of all problems upgrading to 19 should be working better.

Of course you could start over again with a new version… depending on how many users you have on your instance it could take a while as well.

Good luck!