CLI update hang

[executive]

I tried running the latest update from CLI.
I wrote a script to timestamp the log file every 5 minutes so I could keep track:

Sat Dec 16 19:55:03 PST 2023
Nextcloud Updater - version: v27.1.0rc3-15-gfa7af53

Current version is 27.1.5.

Update to Nextcloud 28.0.0 available. (channel: "stable")
Following file will be downloaded automatically: https://download.nextcloud.com/server/releases/nextcloud-28.0.0.zip
Open changelog ↗

Updater run in non-interactive mode.

Start update

Info: Pressing Ctrl-C will finish the currently running step and then stops the updater.

[ ] Check for expected files ...
e[2K[✔] Check for expected files
[ ] Check for write permissions ...
e[2K[✔] Check for write permissions
[ ] Create backup ...Sat Dec 16 20:00:02 PST 2023
Sat Dec 16 20:05:17 PST 2023
Sat Dec 16 20:10:17 PST 2023
Sat Dec 16 20:15:02 PST 2023
Sat Dec 16 20:20:06 PST 2023
Sat Dec 16 20:25:01 PST 2023
Sat Dec 16 20:30:01 PST 2023
Sat Dec 16 20:35:02 PST 2023
Sat Dec 16 20:40:01 PST 2023
Sat Dec 16 20:45:05 PST 2023
Sat Dec 16 20:50:02 PST 2023

e[2K[✔] Create backup
[ ] Downloading ...Sat Dec 16 20:55:01 PST 2023

e[2K[✔] Downloading
[ ] Verify integrity ...
e[2K[✔] Verify integrity
[ ] Extracting ...Sat Dec 16 21:00:02 PST 2023
Sat Dec 16 21:05:01 PST 2023

e[2K[✔] Extracting
[ ] Enable maintenance mode ...
e[2K[✔] Enable maintenance mode
[ ] Replace entry points ...
e[2K[✔] Replace entry points
[ ] Delete old files ...
e[2K[✔] Delete old files
[ ] Move new files in place ...
e[2K[✔] Move new files in place
[ ] Done ...
e[2K[✔] Done

Update of code successful.
Updater run in non-interactive mode - will start "occ upgrade" now.

Sat Dec 16 21:10:03 PST 2023
Sat Dec 16 21:15:03 PST 2023
Sat Dec 16 21:20:03 PST 2023
Sat Dec 16 21:25:02 PST 2023
Sat Dec 16 21:30:03 PST 2023
Sat Dec 16 21:35:02 PST 2023
Sat Dec 16 21:40:02 PST 2023
Sat Dec 16 21:45:02 PST 2023
Sat Dec 16 21:50:03 PST 2023
Sat Dec 16 21:55:03 PST 2023
Sat Dec 16 22:00:04 PST 2023

it’s been hanging on this “will start occ upgrade now” step for over an hour now.
Is this normal?

[ernolf]

@executive

I wrote a script

@executive

Is this normal?

Without knowledge of that script, one can not know, if it is normal.

Why do you do it by script, do you have no CLI access to your server?

Much luck,
ernolf

[executive]

@ernolf

Without knowledge of that script, one can not know, if it is normal.

all the script does is write the timestamps that you see (eg Sat Dec 16 20:30:01 PST 2023) to the log file every 5mins that’s it.
Nothing to do with NC whatsoever. Shouldn’t interfere.

@ernolf

Why do you do it by script, do you have no CLI access to your server?

I don’t have shell. So I launch the update using cron (–no-interaction) and pipe the output to a log file. It has work ok so far.

[executive]

Well I got impatient and launched ‘occ upgrade’ manually.
That did not go well …

Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Setting log level to debug
Updating database schema
Exception: Database error when running migration 28000Date20230906104802 for app core
Index name "nid" for table "oc_social_3_stream" collides with the constraint on table "oc_social_3_cache_actor".
Update failed
Maintenance mode is kept active
Resetting log level

[ernolf]

@executive

Shouldn’t interfere.

What exactly stands in the script please?

@executive

So I launch the update using cron

OK. But HOW did you launch it.

Again: Without the knowledge of the invoked script, it is impossible to know, what exactly is happening or how you could change to get a better result.

Much luck,
ernolf

[executive]

@ernolf

OK. But HOW did you launch it.

via cron:
php -f ${cloud}/updater/updater.phar -- --no-interaction >> update.log 2>&1

I ran the upgrade this way last time and I didn’t have this issue.

@ernolf

What exactly stands in the script please?

the script is completely irrelevant.

TZ='America/Vancouver' date >> ${logfile}

[executive]

still no luck. I tried running the web updater instead and still get the

Exception: Database error when running migration 28000Date20230906104802 for app core Index name "nid" for table "oc_social_3_stream" collides with the constraint on table "oc_social_3_cache_actor".

How do I recover from this? Honestly this software has the worst update process I’ve ever seen.

EDIT: looks like I’m not the only one with this problem. It’s an open bug caused by some garbage tables left over from some “social” app which I never installed.
I thought 28 was “stable” but I should have waited before updating

[Bug]: Database error when running migration 28000Date20230906104802

[ernolf]

@executive

Honestly this software has the worst update process I’ve ever seen.

Excuse me, but Why do you update now already since you know about all of this problems? The Forum and the issue tracker of the server is full of messages about this problems. I myself have Nextcloud 28 only on some testing environments but my productive servers are running 27.1

@executive

I thought 28 was “stable” but I should have waited before updating

exactly.

@executive

Exception: Database error when running migration 28000Date20230906104802 for app core Index name “nid” for table “oc_social_3_stream” collides with the constraint on table “oc_social_3_cache_actor”.

That tables look verry strange, since I have the social app installed and on my server the table has other names (without the ‘3’ after social)

@executive

EDIT: looks like I’m not the only one with this problem. It’s an open bug caused by some garbage tables left over from some “social” app which I never installed.

I understand that as a user there is always a tendency to blame the software for errors and difficulties but those tables do not come there without some interaction from your side. Your computer never forgets nothing. Every single step you did, even when you reverted it, leaves some traces in the database or elsewhere.
But since you do not use the social app and it is disabled or even removed from your server, then you can drop all social related tables complete:

DROP TABLE IF EXISTS nextcloud.oc_social_3_%;

(after you made a backup of your database) and then the updater will not complain about some double available indices.

Much luck,
ernolf

[executive]

I don’t have time to read issue trackers. Just like most people don’t research developer history when updating phone apps.

I dropped ALL oc_social* tables per these instructions.

[fixed] Upgrade 28.0.0 stable ... is NOT stable - was broken index data from old "Social APP"

Same Problem here :-/ Update of code successful. Should the “occ upgrade” command be executed? [Y/n] Y Nextcloud or one of the apps require upgrade - only a limited number of commands are available You may use your browser or the occ upgrade command to do the upgrade Setting log level to debug Repair step: Repair MySQL collation Repair info: All tables already have the correct collation → nothing to do Repair step: Copy data from accounts table when migrating from ownCloud Repair step:…

[ernolf]

@executive

because I don’t have time to read issue trackers.

Then you should have time for the problems.

@executive

I dropped ALL oc_social tables

Exactly what I advised you to do:

@ernolf

But since you do not use the social app and it is disabled or even removed from your server, then you can drop all social related tables complete:

DROP TABLE IF EXISTS nextcloud.oc_social_3_%;

Much luck,
ernolf

[executive]

@ernolf

Then you should have time for the problems.

I don’t . And if stable releases consistently have problems like this, then I don’t have time for NC either.

@ernolf

Exactly what I advised you to do:

No, you didn’t. You advised dropping oc_social_3_*
Solution is to drop oc_social_*
Luckily I fixed it myself before your advice, but thank you for your effort.

[executive]

@ernolf

Then you should have time for the problems.

Imagine I was a paying customer, and I had to waste time chasing bugs like this on what is supposed to be a finished product. Possibly even losing money. Not the best business practice I think.

[ernolf]

@executive

Imagine I was a paying customer,

Paying customers do not get the “latest” version but get it when it is considered stable.

@executive

what is supposed to be a finished product.

My friend, I am just trying to advise you and am writing out of my experience. It is the cutting edge, far away from stable, even though it is not in beta status.
It is not a good idea to do major upgrades on a productive system as long as that version is not yet tested thoroughly. It is better to wait until the first three or four bug releases are released.

Much luck,
ernolf

[executive]

you know, it’s funny. Normally I do wait a bit before upgrading to the latest and greatest. But many times when I come here to forum for help, the first response will be “your version is too old you need to update!!”. So … can’t be too old … can’t be too new … are there any versions that actually work properly??

[ernolf]

@executive

are there any versions that actually work properly??

27.1.* - Hub6

Much luck,
ernolf

[jtr]

@executive

But many times when I come here to forum for help, the first response will be “your version is too old you need to update!!”. So … can’t be too old … can’t be too new … are there any versions that actually work properly??

Don’t confuse being out of date just due to needing to bump to the latest maintenance patch level[1] with running out-of-date/no longer unsupported major versions.

At any given point in time at least two, and often three, major versions are supported (currently: NC26, NC27, NC28)[2].

If someone suggests updating, they generally mean make sure you have the latest maintenance patch level and are running a currently supported major version[2].

So if you were to post today saying you were running v23.0.3 someone would probably rightfully say “update to a supported version >=NC26”.

If you said you were running v27.0.4 you might get nudged to bump up to the latest NC27.x.x patch level (currently v27.1.5).

Minor (maintenance) version updates are generally low risk.

But running NC28 (v28.0.0) is bleeding edge (it came out <7 days ago).

• NC26 is the most mature and robust
• NC27 is getting there
• NC28 is brand-new

[1] Maintenance and release schedule — Nextcloud latest Administration Manual latest documentation
[2] Maintenance and Release Schedule · nextcloud/server Wiki · GitHub

[executive]

@jtr

But running NC28 (v28.0.0) is bleeding edge (it came out <7 days ago).

OK, I was fooled by the “stable” descriptor. From now on I’ll stay one version behind. Thank you.

[executive]

@executive

it’s been hanging on this “will start occ upgrade now” step for over an hour now.
Is this normal?

update 28.0.1
Again ‘upgrade’ command is not launched automatically

Still have some PHP errors during the update

[22-Dec-2023 01:01:24 UTC] PHP Fatal error:  Uncaught Error: Call to undefined function NC\Updater\system() in phar:///home/mountain/public_html/cloud/updater/updater.phar/lib/UpdateCommand.php:310
Stack trace:
#0 phar:///home/mountain/public_html/cloud/updater/updater.phar/vendor/symfony/console/Command/Command.php(255): NC\Updater\UpdateCommand->execute()
#1 phar:///home/mountain/public_html/cloud/updater/updater.phar/vendor/symfony/console/Application.php(1021): Symfony\Component\Console\Command\Command->run()
#2 phar:///home/mountain/public_html/cloud/updater/updater.phar/vendor/symfony/console/Application.php(275): Symfony\Component\Console\Application->doRunCommand()
#3 phar:///home/mountain/public_html/cloud/updater/updater.phar/vendor/symfony/console/Application.php(149): Symfony\Component\Console\Application->doRun()
#4 phar:///home/mountain/public_html/cloud/updater/updater.phar/updater.php(10): Symfony\Component\Console\Application->run()
#5 /home/mountain/public_html/cloud/updater/updater.phar(14): require('phar:///home/mo...')
#6 {main}
  thrown in phar:///home/mountain/public_html/cloud/updater/updater.phar/lib/UpdateCommand.php on line 310