NC 20: Too many incompatible apps: What to do?

[Spartachetto]

I’ll try not to be harsh, but It is a bit strange to read the same comments at every new release…

The same day that Nextcloud 20 was released I read complaints that not all the apps were updated. I cannot say I was astonished.

“everything is available right now”, they said

We know how the apps are developed:

• Nextcloud (the firm) develops a set of core apps, which ARE available
• other societies (OpenOffice, Collabora, Moodle and more…) codevelop some important apps. These societies were at the Conference and these apps either are ready or should be ready really soon
• some apps are developed by “the community” for free. I think that the presence of several developer (e.g.: @Rello ) as speakers in the conference and the fact that the apps are ready for Nextcloud 20 testifies a continuous effort towards a good synchronization.

If some community developer is not ready with a new version the exact day of the release I think that we can have a bit of comprehension… They are working for free!

---

Another point: the rapid pace of new versions. As far as I understood they just released a new set of APIs for the front end and they are improving the already available ones. This should allow the app developers to better integrate an app in Nextcloud but also to have an easier and more predictable developing experience…
This was also discussed here and on Github, and several suggestions from community developers were listened

---

Last point: about stability vs. new features. Here again, it was evident and also in some cases clearly explained, that often it was necessary to review and improve the software architecture to develop new features. Not all the planned improvement are done yet, but if one follows Github it is evident that continuous efforts are spent toward this goal.

[StartupStack]

Note that I wasn’t complaining about apps that haven’t updated yet. I was complaining that they shouldn’t have to.

This is a solved problem. Semantic versioning along with strong respect for what the versions imply in regards to an API contract takes care of this. It’s not necessarily easy, and is definitely not free, but it can be done, and has been done by many other products.

---

I’m also not complaining about API changes. What I’ve seen are improvements. It’s also a solved problem to be able to evolve APIs allowing updated apps to opt-in to new features, while retaining backward compatibility so as not to break every other 3rd party app with each change.

I think the current attitude at Nextcloud accepts breakage too easily. I don’t think this is a lack of skill or ability on their part, but a cultural choice. And one I hope they fix.

Yes, maintaining backward compatibility takes extra work and adds complexity and maintenance burden to code. Over time older compatibility layers can be removed to ease that burden, once 3rd party apps have had time to update. That’s what major version changes are supposed to be for. At least give the apps a few releases in between before they break.

---

I’m also aware that there are fundamental architectural improvements in the works. Again, a very good thing. But this also requires a delicate balance and carefully testing to prevent breaking existing, working systems. The breakage I’ve been seeing creep in so far seems to be caused by incompatible API changes more than anything. See my second point above. This is avoidable.

With the large, and growing, number of 3rd party apps, the combinatorics for thorough testing are already too large. It’s just not possible to test every combination of installed apps and see where conflicts arise. Better API management is required here to keep the basics working in the face of change.

The rapid pace of releases is also a good thing. No complaints about that. Several major apps release even more often. Note that they generally don’t release breaking changes very often.

Nextcloud could release 20.1, 20.2, etc. on a monthly basis and I’d be thrilled. So long as each of the minor releases don’t break existing apps. When they absolutely have to break an API, then bump the major version and force all the apps to update (or at least test). And if marketing just loves their round numbers too much, that’s OK too. There can be a separate internal API version that apps key to rather than the marketing version.

[Freedim]

Sorry if I am not being relevant, but when I read such exchanges, I have two thoughts:

• I am really small and out;
• they seem really good but I feel I don’t have access.

I am not a developer, just a random IT in a random small organisation, who is convinced that open source software and privacy must win.

I think it is great that a lot of exchange and communication exists between Nextcloud and its “community”. It makes it live and thriving.

But I often feel I am excluded from the community as a mere user. A lot of communication is not mere user-friendly. Great features are announced as described as currently implemented, sometimes explicitly described as “production ready” or “enterprise ready” in front pages, then I need to spend days navigating bugs and deeper web pages, create accounts in github or here to request precisions, to finally find out that it is kind of obvious for knowledgeable people, the “community”, that the announcements should not be taken to the letter.

Maybe Nextcloud’s community, which is made of members that are visible to each other because they exchange views and meet at conferences, underestimate the vast amount of mere users. Those users have great hope they can leave Google, Microsoft etc. and that opt for a Nextcloud plan from a reseller for their private cloud and their employers cloud.

And by the way, leaving Big Tech is the main drive to many of them. No need to announce great features 6 months before they are ready (meaning “stable”) to convince them.

Bottom line: maybe having two clearly distinct channels of communication would help: one for the community, and one for the users. Even maybe a third one for resellers, because they often seem to have no clue about what is going on with new or announced features, known issues etc.

Thanks for your attention

[rollanders]

I use to upgrade every new version routinely. I have one experimental and one production site. I I always test the upgrade on the experimental site first, and usually wait for the first update of a new version before updating the production site. This has worked for me and probably for most users:

1. Make a backup or snapshot before you upgrade so that you can easily restore if anything goes wrong.
2. Update
3. Enable all incompatible or untested apps that you need. Usually you first need to enable it and then activate it. In most cases the “incompatible” app is accepted and working.
4. Check the log for errors related to the apps.
5. If errors are related to specific apps - disable them. Then you may have to wait for e new compatibel version. This has not happened to me except for the gallery app which has a solution that you can find in the forum.
6. If NC crashes (rarely happened) - go back to the snapshot.

[enoch85]

Actually, all apps are backed up during upgrade and end up in /mnt/NCBACKUP/.../apps

If you want to re-enable the (at your own risk) apps, then just copy over the missing apps to the live apps dir in Nextcloud, or try to enable them one by one in the GUI - red button in app store.

If you have many customizations or apps (or in general for that matter) you should always wait until the first bug fix is released, in this case 20.0.1, so that app maintainers have a chance to make their apps compatible.

In any case using the VM, nothing is lost - and we do have a check for incompatible apps which don’t install them if they are incompatible. But as I mentioned, they are backed up, and you can manually add them back if you want, or simply reinstall them when they show up in the app store again since the DB stuff aren’t removed either.

[dshirk]

Consider adding this information to a best practices section of documentation related to installation. Reading through hundreds of threads in community is really hard on mere “users.”

Maybe not necessary if you want NC to only be used by larger organizations that have IT staff in house that are knowledgeable programmers. Smaller shops don’t have time to keep up with all the noise. If all the chatter was about the RC that would be fine. But it’s about the stable release.

These are not new topics, it’s hard to determine whether any release is stable enough to adopt. That’s why my NC instance has remained a rest site for 5 years and only had production data for 6 months. My production site remains with a large commercial app.

I’ve always found the documentation light on explanation of what the intended use case is for a given feature and explanation of expected outcome. These may be obvious to developer but a little more time in documentation would save a lot more time in forum scouring and result in a lot more satisfaction from users. Make the jump from early adopters to mainstream user adoption requires that commitment.

Im not a developer per se but I was a functional architect for core enterprise software used by 3 Fortune 500 companies. All the functionality my team specified for developers took as much time to document as to design. And the most important and difficult work was on keeping what was expected where and when expected, not adding the new bells and whistles marketing teams could sell before they were stabilized.

[pandorra]

in terms of unexpected behaviour, just have a look what’s going on with photos/album

[Discussion] New Photo App in Nextcloud 18

Hi! The new Photo App (Gallery) is very bad… I have 20.000 Photos and 1000 Folders… When I open the Photo app, it shows me 20000 Photos without any possibility to sort them. When I look for the last photo (date), I must scroll down the whole day… And the worst is, that when I click on “Your Albums” it looks like my whole Apache crashes. After 10 Minutes it shows me all my 1000 Folders in color gray…and 10 Minutes later with thumbnails… Is there any possibility to fix it, short the photos …

[Rello]

Hi @nomad

Audio Player is available since yesterday by the way. I was busy finishing the conference video.
thank you…

[John30k]

Thank you @Rello and all the other developers that are putting so many hours in!

[DarkSteve]

@rakekniven

The updater shows a warning about apps which are not compatible.

Why did you proceed?

Because most people don’t use the internal updater. Depending on what OS you’re using, or whether you’re running a prepackaged version (e.g. NextcloudPi), updating doesn’t show any warning about incompatible apps. I use FreeBSD, and all my updating is done through ports, which tells me nothing about which NC apps will be disabled.

Combined with the NC company pushing immediate updates, that’s why people proceed.

@rakekniven

The scripts should be adapted to warn you before silently upgrading.

It’s got nothing to do with silently upgrading, the end user can be intentionally upgrading. The issue is that Nextcloud exists in a larger ecosystem, and it’s putting an unnecessary burden on community developers and maintainers.

@anon71540698

There is [a warning that there might be incompatible apps] in the GUI, here is an example

Your example shows that all apps are compatible with a minor version (i.e. 19.0.4RC1). On FreeBSD, I don’t get the option of going to a minor new version, I’m upgraded to the next major version once it’s available. Your example tells me ALL apps are compatible with the new version (19.0.4), but then I’m upgraded to version 20.0, breaking apps.

This is the issue people are having. We’re all happy to have a rapid development pace, but it’s just not necessary to break compatibility with existing apps, especially ones that were working fine until yesterday.

@StartupStack

Nextcloud needs to adopt semantic versioning, at least for internal APIs. What they call releases for marketing purposes can be orthogonal.

This appears to be the most reasonable approach. Nextcloud has been WAY better at managing it’s updates/upgrades than ownCloud ever did, and I have far fewer issues with upgrades than I used to. But that doesn’t mean it can’t be further improved!

[nomad]

Great, thanks a lot!

[kesselb]

@DarkSteve

I use FreeBSD, and all my updating is done through ports, which tells me nothing about which NC apps will be disabled.

The FreeBSD ports are not maintained by Nextcloud. Talk about that issue with the port maintainer.

I guess it’s possible to expose the all apps are compatible via occ. Is there already an issue on GitHub?

[DarkSteve]

@kesselb

The FreeBSD ports are not maintained by Nextcloud.

I didn’t say it was, I was explaining to somebody else why people upgrade without knowing what apps will break.

Since I only use official apps, I don’t tend to have problems with upgrades. However, the reason I only use official apps is because of a history of third party apps breaking with every update. That’s not ideal for a healthy open source ecosystem.

It’s possible to manage updates and upgrades better than they currently are.

[rollanders]

@DarkSteve

Because most people don’t use the internal updater.

I question that. I for one use it and it works without problem except I get a warning and I have to activate the noncompatible apps manually from within the NC user interface.

[DarkSteve]

@rollanders

I question that. I for one use it

Well, I for one don’t use it.

But whatever. There are plenty of people in this very forum that also don’t, and have questions about disabled apps or when their upgrading goes awry.

How about I rephrase. There is a significantly large number of people that don’t use the internal updater.

I don’t think it changes my point at all. It also doesn’t change the fact that a lot of people unexpectedly lose apps while upgrading.

[JimmyKater]

I’m answering directly to the question from threadtitle:
“What to do?”

It’s as easy: Nobody can force you to do something you don’t want to do… So best you can do is: waiting.
Yes, even if NC GmbH thinks it’s better to update immediately. It’s your own responsibilty doing it or not.

[kesselb]

I don’t understand why some of you hijack this thread with the photos app topic again. It’s a completely different story. This topic was about apps that are disabled after upgrade because they are not updated yet. Gallery is not developed anymore and has been replaced by Photos which some of you (for good reasons of course) don’t like. But that’s something to discuss at: New Photo App in Nextcloud 18.

[jospoortvliet]

@StartupStack

This is a solved problem. Semantic versioning along with strong respect for what the versions imply in regards to an API contract takes care of this. It’s not necessarily easy, and is definitely not free, but it can be done, and has been done by many other products.

I don’t think so. Now I’m not super technical, but if you followed the conference you might have seen that we just introduced versioned API’s for Javascript - my understanding is that until now, this was not guaranteed. And it will require changes in the apps to make it work, like bundling stuff JS libraries.

Here’s Christoph talking about that:

Livestream of the Nextcloud Conference 2020 Day 2

So yes, maybe in the future, we can make it so that apps will keep running.

On the general outdated-apps issue.

We can’t indeed expect all app authors to update their apps every 4 months. That is why we did the ‘enable untested app’ button - yes, it means after a major upgrade you have to click that a few times to get the untested/un-updated apps enabled, but that isn’t THAT much work, 3x a year… Enabling them automatically and potentially breaking systems seems a less than optimal solution.

With more stable API’s in the future, this issue will get smaller. But honestly, if this is your biggest issue with Nextcloud, having to click the red ‘untested app’ button, I think we did a fine job

[efftee]

Thank you for this enlightening comment.
This new development (API) sounds extremely promising and exciting for future interactions.
Beyond apps, I hope it will also empower admins to use it for new use cases…
It must have taken some guts to make the leap.
Considering this change I fear we might have to wait a little longer than usual to get all the available apps we hope for, before moving to production with NC20.
I’ve just switched to 19.0.4 in production, and for some reason I’m amazed by what seems like a big efficiency increase and responsiveness. Great. Really glad !
Can’t wait to move to NC20, but also very conscious and respectful of the necessary work from all the apps developers.
Thank you !

[ruppscloudadmin]

I rolled back to 1:21:00 to see the Moodle presentation. I’m blown away. I’ve run a fairly undiscovered Moodle site for a couple years now and had no idea how much cool stuff was coming next! Amazing. I think I went from a appreciative user/admin to a zealot by watching that.