Nextcloud 30 been rolled out over the "stable" channel although known to be unstable

Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version (eg, 29.0.5): 30.0.0
Operating system and version (eg, Ubuntu 24.04): Debian 12 (bookworm)
Apache or nginx version (eg, Apache 2.4.25): Apache 2.4.62
PHP version (eg, 8.3): 8.2.20

The issue you are facing:

I am involved in the maintenance of multiple Nextcloud instances, including one of a large german parent association. All servers are updated automatically, and the “stable” update channel is selected.

Unfortunately, Nextcloud does not separate security and feature updates (as e.g. the Debian project does). At the same time, easy rollbacks after upgrades are not supported and there appear to be no plans to support them:

Downgrade from Nextcloud 30 to 29

This is a very bad combination IMHO. In consequence, however, users must (at least) be able to rely on the fact that “stable” updates do not contain serious, known issues.

A few days ago, Nextcloud 30.0.0 has been rolled out to many servers previously running (in our case) NC 29.0.x, although it is known, for example, that the OnlyOffice connector is not working:

OnlyOffice-Nextcloud Connector not working in Nextcloud 30.0 (Connecting Refused) · Issue #1020 · ONLYOFFICE/onlyoffice-nextcloud · GitHub
Reported as not compatible with NC 30.0 · Issue #1022 · ONLYOFFICE/onlyoffice-nextcloud · GitHub

(both still open as of now)

In our case, at least the following “featured” apps were disabled during the upgrade, because they are “not compatible with your Nextcloud version”:

  • ONLYOFFICE (9.3.0)
  • Forms (4.2.4)
  • Maps (1.4.0)

Several more reports on update issues can be found in this forum.

To resolve this, I honestly propose to:

  1. Resolve the issues with these apps at high priority with joint effort.
  2. In the future, check the “featured” apps for compatibility before rolling out a new release via the stable channel and only roll out if the “featured” apps are known to work.
  3. Provide and/or document an easy way for rolling back to the previous stable NC release, so that also in the case of unexpected issues, server admins are able to safely and quickly get back to a running system.

Digital sovereignty is a very hot and IMHO important topic today. I would be very unhappy to see the reputation of such a great project and companies like Nextcloud GmbH beeing damaged due to update issues that are clearly avoidable.

Thank you for consideration.

The details below mainly refer to the OnlyOffice connector issues after enabling it manually.

Is this the first time you’ve seen this error? (Y/N): N

Steps to replicate it:

  1. Install a NC 30.0.0
  2. As an admin user, check the status of the “featured” apps “ONLYOFFICE” (9.3.0), “Forms” (4.2.4) or “Maps” (1.4.0). At the time of this writing, they are all labelled “This app is not compatible with your Nextcloud version…” in the tooltip of their “Allow untested app” button.

The output of your Nextcloud log in Admin > Logging:

(see below)

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'XXX',
  'passwordsalt' => 'XXX',
  'secret' => 'XXX',
  'trusted_domains' => 
  array (
    0 => 'cloud.XXX.de',
  ),
  'datadirectory' => '/var/local/nextcloud-data',
  'dbtype' => 'mysql',
  'version' => '30.0.0.14',
  'overwrite.cli.url' => 'https://cloud.XXX.de',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud',
  'dbpassword' => 'XXX',
  'installed' => true,
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'ssl',
  'mail_sendmailmode' => 'smtp',
  'mail_domain' => 'XXX.de',
  'mail_smtpauthtype' => 'PLAIN',
  'mail_smtpauth' => 1,
  'mail_from_address' => 'nextcloud',
  'mail_smtphost' => 'mail.XXX.de',
  'mail_smtpport' => '465',
  'mail_smtpname' => 'nextcloud',
  'mail_smtppassword' => 'XXX',
  'default_locale' => 'de_DE',
  'default_language' => 'de_DE',
  'default_phone_region' => 'DE',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'ldapProviderFactory' => 'OCA\\User_LDAP\\LDAPProviderFactory',
  'app_install_overwrite' => 
  array (
    0 => 'ownpad',
    1 => 'ldapcontacts',
    2 => 'onlyoffice',
  ),
  'simpleSignUpLink.shown' => false,
  'profile.enabled' => false,
  'maintenance' => false,
  'loglevel' => 2,
  'theme' => '',
);

The output of your Apache/nginx/system log in /var/log/____:

(nothing relevant found)

Output errors in nextcloud.log in /var/www/ or as admin user in top right menu, filtering for errors. Use a pastebin service if necessary.

{"reqId":"Zulb6Y1PQ3bJkDlR0maufgAAAAQ","level":3,"time":"2024-09-17T10:37:30+00:00","remoteAddr":"62.144.230.160","user":"Admin","app":"no app in context","method":"GET","url":"/index.php/settings/ajax/checksetup","message":"Exception running check OCA\\User_LDAP\\SetupChecks\\LdapConnection: The arguments array must contain 2 items, 1 given","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0","version":"30.0.0.14","exception":{"Exception":"ValueError","Message":"The arguments array must contain 2 items, 1 given","Code":0,"Trace":[{"file":"/var/www/nextcloud/lib/private/L10N/L10NString.php","line":68,"function":"vsprintf"},{"file":"/var/www/nextcloud/lib/private/L10N/L10N.php","line":107,"function":"__toString","class":"OC\\L10N\\L10NString","type":"->"},{"file":"/var/www/nextcloud/lib/private/L10N/LazyL10N.php","line":38,"function":"n","class":"OC\\L10N\\L10N","type":"->"},{"file":"/var/www/nextcloud/apps/user_ldap/lib/SetupChecks/LdapConnection.php","line":87,"function":"n","class":"OC\\L10N\\LazyL10N","type":"->"},{"file":"/var/www/nextcloud/lib/private/SetupCheck/SetupCheckManager.php","line":34,"function":"run","class":"OCA\\User_LDAP\\SetupChecks\\LdapConnection","type":"->"},{"file":"/var/www/nextcloud/apps/settings/lib/Controller/CheckSetupController.php","line":147,"function":"runAll","class":"OC\\SetupCheck\\SetupCheckManager","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":208,"function":"check","class":"OCA\\Settings\\Controller\\CheckSetupController","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":114,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":161,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":302,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1001,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":24,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/lib/private/L10N/L10NString.php","Line":68,"message":"Exception running check OCA\\User_LDAP\\SetupChecks\\LdapConnection: The arguments array must contain 2 items, 1 given","exception":[],"CustomMessage":"Exception running check OCA\\User_LDAP\\SetupChecks\\LdapConnection: The arguments array must contain 2 items, 1 given"},"id":"66e95bf17170d"}

2 Likes

What can we learn from this: Do not install version x.0.0 on the production server

1 Like

Hello @gkiefer ,

welcome to the community of Nextcloud.

Why do you have automatic updates enabled if you are involved in multiple instances?

Please prove your process and have a working backup strategy which allows you a roll back.

3 Likes

@gkiefer Hey there and welcome to the wonderful world of the nextcloud community.

that’s clearly not the business and not the means of this forum. we are here to help mostly homeusers with their problems. companies should have a prosupport-contract with NC Inc.

On the other hand, my dear, you are apparently one of the admins of several NC-instances. And someone must have set them up to automatically update. You don’t need to do that. It’s a free decision of yourself. Just stop automatic updating and wait until your needs in terms of apps are fulfilled and then do the upgrading.

this is a problem of this app. not neccessarily of NC.

“stable” release means: the core serversoftware will run without known major errors/bugs/mistakes.
It’s not the purpose of NC (server) to wait until this or that app are ready to perform updates. this is only within the focus of the server admin - which means: you, my dear.

So please don’t try to hold NC server responsible for problems that you caused yourself. Though I’m really sorry that you were running into problems.

You are right, downgrading isn’t supported but the solution is easy: restore your latest backup (which is usually taken automatically right before the upgrade will happen, unless you deactivated this feature). If you don’t know how that would work the forum will help you… after you used the built-in search-function.

Apart from that… it’s known from the beginning on that you can’t downgrade NC.
It’s as well known that some apps won’t be compatible with new versions right after a release of a new major-version (the last app I was needing for N29 just became compatible a week ago).

apart from all of this well.known facts and circumstances… NC still is a great piece of work. Though it needs to be maintained and watched over.

So please review your own setups first… and just let NC be responsible for the things it was made for.

2 Likes

Hi @gkiefer

What you describe here is unfortunately the reality in the Nextcloud universe. There are a lot of people who gloss over it, but it’s not great. What I have learnt in the last 7 years:

  • Always use the latest patch release of the second last major version. Never ever use the latest major version. Older major versions will continue to receive security updates.
  • Do not use automatic updates unless the update script takes this into account.
  • Always test major updates (or all updates) on a test installation first. There are always dozens of apps that are not yet compatible or various Nextcloud bugs.

Personally, I am surprised that Nextcloud is pushing nc 30 to 100% of users in the stable channel. In the past, they waited longer or rolled out less per cent.

Best,
Markus

1 Like

They are not. It is 30%: 30.0.0 by Altahrim · Pull Request #1119 · nextcloud-releases/updater_server · GitHub

You’re right, I was looking in the wrong line.

I still think it’s crazy in view of the many bugs. Even Nextcloud AIO wants to wait for the next patch. But yes, that doesn’t change the fact that I was looking in the wrong line.

Yes indeed. AIO always waits for at least one patch.

But then why upset 30 % of users without Nextcloud AIO? I don’t understand it.

I agree there’s room for improvement (there always is).

I’ve spent a lot of time in this area of things so here are a few thoughts…

My take has always been stable means a stable API/feature-set (i.e. the API within v28 is locked in and won’t change within any v28 minor/maintenance releases); that’s all it means technically.

Unfortunately “stable” means a different thing to operators/admins than it may to developers.

Also, Nextcloud is not consumer software. It’s more like MariaDB/MySQL. Those that rely on their database understand the release model and choose what’s appropriate (e.g. LTS releases versus short-term releases, newer innovation releases if specific features are more important than robustness, etc.). And those where reliability is extremely important, pay up to get advice about what release to run for their particularly needs (and also for personalized help if important things break). :slight_smile:

I’ve been trying, however, to work on some ways to address the gap for everyone:

  • revising the documentation to more thoroughly describe the release process, cadence, and backporting approach (e.g. Maintenance and release schedule — Nextcloud latest Administration Manual latest documentation - though I already want to rewrite this from scratch since it turned out more long-winded than necessary in hindsight)
  • making it clearer that we support multiple major releases, with differing levels of maturity, simultaneously so that the operator can choose whether the latest features are more important than time-in-the-field
  • refining the updatenotification app - which provides the overview about versions and updates that is visible within Server - by enabling the operator to be able to both better understand as well as define their local “update policy” on each instance - e.g. see https://github.com/nextcloud/server/issues/45402)

One area that is more challenging still: new installs. This is because, by default, we don’t use an installer where one selects the version/etc. Instead the installer+version are embedded together as one package. What you download is what you install. And new operators just grab what we offer as the “latest” download (understandably). There are also numerous ways of “installing” Nextcloud so there are differences in terms of the initial experience merely by virtue of what method was chosen initially (e.g. AIO versus Archive versus Community Docker versus Snap versus etc etc).

Archive: We could probably do a better job in how we present the different (supported) major versions that are available to download. Even if we implement a great mechanism to let them define a suitable update policy, that only benefits them down the road since they’ve effectively installed the wrong initial release for their needs. After all, we spend lots of extra time maintaining multiple major versions simultaneously, but that isn’t helpful to a new operator that deploys “the latest major” because that’s all that that we presented them with.

This is solved somewhat in solutions like AIO, because AIO effectively has its own installer and part of the AIO value-add is that releases are held back/pushed based on judgement that probably more closely resembles what a “typical” operator cares about: showstopper bug prevalence / code maturity.

The Snap and Community Docker also have flexibility in this area (though the latter tries to stick to the % of users to determine what’s labeled as “stable” so it’s “update policy” closely follows an Archive deployment; for new installs/deployments it’s effectively the same since the operator simply gets the latest if the operator doesn’t specify a stable Docker tag).

I personally would like to see us split out the stable channel and define it basically from the operator perspective:

  • latest (N; latest stable branch)
  • stable (N-1)
  • oldstable (N-2) [whenever not EoL]

As the time of writing this, this would mean:

  • latest = v30
  • stable = v29
  • oldstable = v28

We could probably make this change entirely in the updatenotification app and in how we present the releases available for download. Internally (development-wise) we don’t have to change anything. We still handle our branches the same and the release cadence doesn’t change. Downstream packaging (i.e. anything other than Archive-based install methods; so basically AIO, the various Docker images, Snap, VMs, etc.) would map this to whatever is appropriate for their own release models, target user bases, etc.

I also would love to see us better organize beta/RC programs by essentially building out a team of operators/community members/contributors that commit to testing out each release. And I think we could do a better job highlighting areas of highest risk within each new beta/RC, so that testers know where to focus their efforts.

What no one can fix is those that can’t resist deploying the latest and greatest, but also have no internal staging/testing strategy nor a back-out strategy. The latest and greatest of anything (even heavily pre-tested software) is always going to have greater risk than someone that’s had more time in the field. And that’s true regardless of what we call a given release.

But we should:

  • make it clearer what we’re doing / what we mean
  • make it easier to make an initial choice (at download and/or install time)
  • make it easier to define a locally suitable update policy
6 Likes

Actually we do:

  • a given major branch (e.g. stable29 = aka: v29) is by definition only going to receive bug/security fixes. Once a branch is tagged as “stable” it doesn’t receive new features. It will only receive bug and security fixes (and will for at least one year).
  • we support at least two - and often three - simultaneously major stable branches at a given point in time (and for 12 months each). Currently that means we’re still supporting v28, v29, and now v30. This gives operators a lot of flexibility to decide between the trade-offs of more recent features/optimizations versus stability/proven-time-in-the-field.

And this is only the community (free) side of things. My understanding is that enterprise (paid) support can be much longer.

So the just tagged stable30 branch (aka: v30) will only receive bug/security fixes (via approximately monthly maintenance releases) from now until that branch reaches end-of-life.

We’re already making numerous changes in the future v31 branch (master / main until tagged as stable). When we fix important bugs, we backport those (sometimes easily, but also often manually because we need to adapt them to differences in the code base) to all supported relevant majors.

More conservative environments - and this is just my opinion - should probably stick to N-1 or even N-2. But even that has exceptions depending on local requirements/needs.

1 Like

Additional channel "prevstable’ would be cool. Maybe someone could write an issue.

1 Like

Tip: The update for 30 is offered to instances running 29.0.7.1. It’s not necessary to install the update right away. Once we publish 29.0.8.x it will be offered to you.

Ref: updater_server/config/config.php at 6c9083e9d759f2c40d4048485532cdddbe883f84 · nextcloud-releases/updater_server · GitHub

Hi @jtr,

thanks four your detailed answer. I really appreciate that.

For me, “stable” is a production release, the one which is recommendet for most users. For example, debian stable is what you want to use if you use debian because it is solid and has the longest support until it is End-of-life. At least that’s what I would understand as behind stable. Same with e.g. Chrome. I believe that you are right with this observation.

I think that could prevent some discontent. I would very much welcome that :slightly_smiling_face:

Hello everybody,

thank you for the warm welcome and the quick replies.

@jtr: Thank your very much for the detailed explanations!

Thank you, this is exactly what I was looking for and certainly a great scheme. However, what I am missing is a way to select these branches for automated updates. In the adminstration panel and on the website (Nextcloud release channels and how to track them - Nextcloud) I can only see three “release channels”, named Production (for commercial customers), Stable and Beta.

I apologize, if this a stupid questions, but I was not able to find it: How can I make updater/updater.phar to follow a branch like “v29” or do something equivalent?

Software that is exposed to the internet and capable of storing much and potentially sensitive data must be upgraded regularly for security reasons. I have seen private installations that have not been updated for years with lots of private files including sensitive data stored. We all know that this is not secure, but in practice, this happens if there is no easy way to setup automatic updates with as little trouble as possible.

Just to clarify: We are not a company, but a non-profit association. The Nextcloud instance I was talking about is not a very big one (yet) and can easily be self-maintained. (And of course, we have regular backups.) Another reason why we maintain it ourselves is to have practical experience useful for consulting our members who ask us about software recommendations for schools, parents’ councils or private use.

Thank you again for the great work.

1 Like

This is not a stupid question at all, and I think it would be a nice feature if you could set the updater to stay on the current release branch, and not offer updates for the next major release until the admin manually switches to it, or the current release is EOL. But as far as I know this is not possible.

So I’d say to get what you want, at least for the time being, you’d have to write your own script that compares the major version offered by the updater with the one currently installed, and only proceeds if those versions match.

Oh, and whether you do the upgrades automatically or manually, make sure you always have a current backup of your instance from before the upgrade, just in case something goes wrong. The updater is pretty reliable, and I haven’t had any issues with it in a long time, but better safe than sorry :wink:

4 Likes

re 1: NC does that since the beginning. The forum is full of that.
re 2 & re 3: these are very good suggestions. but…

2 Likes

right. But I bet debian isn’t waiting on Php nor Apache nor any other software to fit their (new) needs. They release when the core-product is ready. All other things needs to adapt to that. And not vice versa.
And exactly this is the case with NC-Server as well. It’s not waiting on apps to meet the new goals. The core apps will meet the new goals and be released while installing the new version. (What is core? I dunno, I estimate: files, calendar, contacts, sharing, security) - All beyond “core” needs to adapt to NC-Server and not vice versa.
Or your other example “Crome” - does crome takes care about if a new release would disable your most loved add-on? No, it doesn’t and it never will. (I do remember clearly that some new versions screwed my fav add-ons at times)

so you might need to adapt your definition a bit.

1 Like

you might wanna ask for a personal quote, though? Afaik there are great offers for non-profit organizations. And after all asking is still for free.

2 Likes

My 5 cents to this topic…

While Debian’s stable releases are of course very “solid” in the sense that things are well tested and therefore provide a stable experience, the primary meaning of “stable” in Debian is not “solid”, but “does not change”.

Of course, providing a stable experience is one of Debian’s primary goals, which they tend to achieve quite well, and certainly this is Nextcloud’s goal as well. However, Nextcloud would probably be more successful in achieving this goal if they did not subordinate everything to fixed release dates. :wink:

EDIT:
One thing to keep in mind though is that the release model of Nextcloud’s Community Edition is not directly comparable to Debian’s release model, as they don’t offer LTS releases for it, while Debian only offers LTS releases.

So a better comparison would be with the Ubuntu interim releases, which are not quite as ‘solid’ as their LTS releases either, although this comparison is not entirely accurate either, as the Nextcloud releases overlap more and are supported a bit longer than the Ubuntu interim releases.

Also, the Nextcloud releases tend to get pretty ‘solid’ after the first or second point release. :slight_smile: