NC 20: Too many incompatible apps: What to do?

Just installed after the exciting announcement at the conference. Maybe it would have been better to wait. Many, partly essential apps seem to be incompatibe, among others Only Office Document server, Music, Social, Extract, Mindmap, Carnet and many more. I am trying to reactivate them anyway and to see what will happen.

I remember at a previous update it often was enough to change the version number inside one of the appsā€™ config file. Is this recommended?

No, changing the version number inside the app isnā€™t recommended or necessary. You can just press the ā€œenable untested appā€ button in the apps admin page (unless that got changed for 20).

In the past, itā€™s been best to wait a few weeks, if not until the first dot release to upgrade if you rely on your apps. They often take a while to update for new major releases.

2 Likes

The updater shows a warning about apps which are not compatible.

Why did you proceed?

3 Likes

Itā€™s a bit an oldfashioned oppinion but if your app is not compatible then thereā€™s a reason for that. If you really need to use a specific app very hard. You need to have an patching strategy. For example.

First try an update on a testmachine, cheap raspberry will do it, and than roal out on your productive system.

Besides allways insure that your are able to roalback a critical system to the last functional stack.

NC itself may donā€™t have the options for that but there are many options on the OS layer to do that for example Acronis etc.

1 Like

It is not recommended but some times it works. I recommend to search from forum app by app information how to get them work. Example for gallery app there is thread in forum how to get it work. It was quite regression when they ā€œupdateā€ it to photos app.

I used the VM installscript for VPS and there the update works via command line and I was not able to see any warnings.

This was not my main instance, I am testing the VM as future main productive system. If I was a sysadmin I would not have upgraded of course.

I wrote this post not because of my situation but wanted to give a general feedback from a normal home user, not sysadmin. From this perspective predictability is important, being able to rely on our apps, not being afraid of upgrades. In this regard - to make home usersā€™ life easier - I would wish a better coordination with developpers of core apps, OnlyOffice document server for example or the Music / Audio player apps. I would only release a new version if core apps are compatible. Also for new users who just installed NC 20 after the exciting keynote, a look at the apps is not especially inviting. So much red in the description, so many apps that cannot be installed!

The message of having to wait for some weeks also clashes with the message of NCs CEO Frank Karlitschek who actually encouraged us to upgrade. In contrast to other companies, he said, NC does not like pre-announcements, everything is available right now!

That was only a feedback from a specific perspective, not an attack. The developper perspective will differ of course. NCā€™s success depends on having both perspectives in mind all the time, I think

1 Like

Good catch. The scripts should be adapted to warn you before silently upgrading.

3 Likes

I, for my part are totally with you!

1 Like

Would you mind filing an issue at https://github.com/nextcloud/vm/issues

Even better would be to search for related issues first :slight_smile:

1 Like

Will check. I am not sure if this is a bug, though, maybe I was not attentive enough regarding command-line messages, might have been my fault also, it was my first upgrade with the VM, sofar everything worked great, and I am very impressed by the VM

Afaik, the update scrupt in the Nextcloud VM doesnā€™t check for incompatible apps before upgrading. But this is also not possible to check via cli, so a limitation of the occ command, afaik. Maybe there should be a warning that there might be incompatible apps before upgrading to a major version?

1 Like

There is in the GUI, here is an example

image

In this case all apps have updates. Those that donā€™t will be listedā€¦

Why blindly run scripts the first day of an update release?

we have been talking about the VM where the update is done via command line and not via the web interface, we do not have this gui there

1 Like

Very much agreed. Nextcloud has mixed-messaging regarding upgrades. The marketing arm encourages rapid upgrades, yet operationally they encourage people to wait for the first minor release and their upgrader doesnā€™t even make new versions available to all users at once (with good reason).

While I also very much appreciate the rapid development of new features, I feel their versioning and compatibility story leaves much to be desired.

Shipping a new release every few months is great. Shipping a new major release every few months causes chaos. Each major release breaks compatibility with all 3rd party apps until they submit an update, even when there are no compatibility issues with a given app. There are apps in common use that have still not updated to version 19 and now we have version 20 to deal with.

Nextcloud needs to adopt semantic versioning, at least for internal APIs. What they call releases for marketing purposes can be orthogonal.

When the API surface for apps breaks in a non-backward compatible way, then they should increment the major version number. When apps donā€™t need to be updated, they should be incrementing only the minor version number.

This, of course, also implies that they need to be more careful about ensuring API compatibility in future releases. They may need to put some focus on API maturity and stability to be able to do this. Also, when adding to their core APIs to enable new features, they need to do this in a backward compatible way whenever possible, even if itā€™s not convenient for core developers.

This would allow new releases without requiring every 3rd party app to update in lockstep. It would also increase customer confidence and allow users to get the latest features faster without sacrificing existing ones.

Especially given that most apps are open source efforts with unpaid contributors, putting the compatibility burden on them is not fair, and clearly does not scale.

When Nextcloud has a release which is really mostly an update of their built-in apps, that should be considered a minor release from a versioning perspective. e.g. if this release were 19.1.0, your apps would still work.


Frankly, I also have concerns about the current pace of development. While itā€™s great to see the new features and improved user experience, Iā€™m concerned that this is coming at a cost of stability.

Iā€™m seeing more and more errors in my logs due to internal software issues, and have been starting to see issues with basic functionality like file syncing. If Nextcloud breaks its core feature set in an effort to add the next new shiny thing quickly, they are doing a disservice to their users and hurting their product immensely.

Sometimes in software development itā€™s necessary to take a brief step back and focus on stability and cleaning up the foundations of your product before you start the next round of building on top of what you have.

I very much want Nextcloud to continue to succeed. Unfortunately I see them heading down a path that has doomed other products in the past. Please, letā€™s not repeat easily avoided mistakes.

13 Likes

Iā€™ll try not to be harsh, but It is a bit strange to read the same comments at every new releaseā€¦

The same day that Nextcloud 20 was released I read complaints that not all the apps were updated. I cannot say I was astonished.

ā€œeverything is available right nowā€, they said

We know how the apps are developed:

  • Nextcloud (the firm) develops a set of core apps, which ARE available
  • other societies (OpenOffice, Collabora, Moodle and moreā€¦) codevelop some important apps. These societies were at the Conference and these apps either are ready or should be ready really soon
  • some apps are developed by ā€œthe communityā€ for free. I think that the presence of several developer (e.g.: @Rello ) as speakers in the conference and the fact that the apps are ready for Nextcloud 20 testifies a continuous effort towards a good synchronization.

If some community developer is not ready with a new version the exact day of the release I think that we can have a bit of comprehensionā€¦ They are working for free!


Another point: the rapid pace of new versions. As far as I understood they just released a new set of APIs for the front end and they are improving the already available ones. This should allow the app developers to better integrate an app in Nextcloud but also to have an easier and more predictable developing experienceā€¦
This was also discussed here and on Github, and several suggestions from community developers were listened


Last point: about stability vs. new features. Here again, it was evident and also in some cases clearly explained, that often it was necessary to review and improve the software architecture to develop new features. Not all the planned improvement are done yet, but if one follows Github it is evident that continuous efforts are spent toward this goal.

2 Likes

Note that I wasnā€™t complaining about apps that havenā€™t updated yet. I was complaining that they shouldnā€™t have to.

This is a solved problem. Semantic versioning along with strong respect for what the versions imply in regards to an API contract takes care of this. Itā€™s not necessarily easy, and is definitely not free, but it can be done, and has been done by many other products.


Iā€™m also not complaining about API changes. What Iā€™ve seen are improvements. Itā€™s also a solved problem to be able to evolve APIs allowing updated apps to opt-in to new features, while retaining backward compatibility so as not to break every other 3rd party app with each change.

I think the current attitude at Nextcloud accepts breakage too easily. I donā€™t think this is a lack of skill or ability on their part, but a cultural choice. And one I hope they fix.

Yes, maintaining backward compatibility takes extra work and adds complexity and maintenance burden to code. Over time older compatibility layers can be removed to ease that burden, once 3rd party apps have had time to update. Thatā€™s what major version changes are supposed to be for. At least give the apps a few releases in between before they break.


Iā€™m also aware that there are fundamental architectural improvements in the works. Again, a very good thing. But this also requires a delicate balance and carefully testing to prevent breaking existing, working systems. The breakage Iā€™ve been seeing creep in so far seems to be caused by incompatible API changes more than anything. See my second point above. This is avoidable.

With the large, and growing, number of 3rd party apps, the combinatorics for thorough testing are already too large. Itā€™s just not possible to test every combination of installed apps and see where conflicts arise. Better API management is required here to keep the basics working in the face of change.

The rapid pace of releases is also a good thing. No complaints about that. Several major apps release even more often. Note that they generally donā€™t release breaking changes very often.

Nextcloud could release 20.1, 20.2, etc. on a monthly basis and Iā€™d be thrilled. So long as each of the minor releases donā€™t break existing apps. When they absolutely have to break an API, then bump the major version and force all the apps to update (or at least test). And if marketing just loves their round numbers too much, thatā€™s OK too. There can be a separate internal API version that apps key to rather than the marketing version.

Sorry if I am not being relevant, but when I read such exchanges, I have two thoughts:

  • I am really small and out;
  • they seem really good but I feel I donā€™t have access.

I am not a developer, just a random IT in a random small organisation, who is convinced that open source software and privacy must win.

I think it is great that a lot of exchange and communication exists between Nextcloud and its ā€œcommunityā€. It makes it live and thriving.

But I often feel I am excluded from the community as a mere user. A lot of communication is not mere user-friendly. Great features are announced as described as currently implemented, sometimes explicitly described as ā€œproduction readyā€ or ā€œenterprise readyā€ in front pages, then I need to spend days navigating bugs and deeper web pages, create accounts in github or here to request precisions, to finally find out that it is kind of obvious for knowledgeable people, the ā€œcommunityā€, that the announcements should not be taken to the letter.

Maybe Nextcloudā€™s community, which is made of members that are visible to each other because they exchange views and meet at conferences, underestimate the vast amount of mere users. Those users have great hope they can leave Google, Microsoft etc. and that opt for a Nextcloud plan from a reseller for their private cloud and their employers cloud.

And by the way, leaving Big Tech is the main drive to many of them. No need to announce great features 6 months before they are ready (meaning ā€œstableā€) to convince them.

Bottom line: maybe having two clearly distinct channels of communication would help: one for the community, and one for the users. Even maybe a third one for resellers, because they often seem to have no clue about what is going on with new or announced features, known issues etc.

Thanks for your attention

7 Likes

I use to upgrade every new version routinely. I have one experimental and one production site. I I always test the upgrade on the experimental site first, and usually wait for the first update of a new version before updating the production site. This has worked for me and probably for most users:

  1. Make a backup or snapshot before you upgrade so that you can easily restore if anything goes wrong.
  2. Update
  3. Enable all incompatible or untested apps that you need. Usually you first need to enable it and then activate it. In most cases the ā€œincompatibleā€ app is accepted and working.
  4. Check the log for errors related to the apps.
  5. If errors are related to specific apps - disable them. Then you may have to wait for e new compatibel version. This has not happened to me except for the gallery app which has a solution that you can find in the forum.
  6. If NC crashes (rarely happened) - go back to the snapshot.

Actually, all apps are backed up during upgrade and end up in /mnt/NCBACKUP/.../apps

If you want to re-enable the (at your own risk) apps, then just copy over the missing apps to the live apps dir in Nextcloud, or try to enable them one by one in the GUI - red button in app store.

If you have many customizations or apps (or in general for that matter) you should always wait until the first bug fix is released, in this case 20.0.1, so that app maintainers have a chance to make their apps compatible.

In any case using the VM, nothing is lost - and we do have a check for incompatible apps which donā€™t install them if they are incompatible. But as I mentioned, they are backed up, and you can manually add them back if you want, or simply reinstall them when they show up in the app store again since the DB stuff arenā€™t removed either.

Consider adding this information to a best practices section of documentation related to installation. Reading through hundreds of threads in community is really hard on mere ā€œusers.ā€

Maybe not necessary if you want NC to only be used by larger organizations that have IT staff in house that are knowledgeable programmers. Smaller shops donā€™t have time to keep up with all the noise. If all the chatter was about the RC that would be fine. But itā€™s about the stable release.

These are not new topics, itā€™s hard to determine whether any release is stable enough to adopt. Thatā€™s why my NC instance has remained a rest site for 5 years and only had production data for 6 months. My production site remains with a large commercial app.

Iā€™ve always found the documentation light on explanation of what the intended use case is for a given feature and explanation of expected outcome. These may be obvious to developer but a little more time in documentation would save a lot more time in forum scouring and result in a lot more satisfaction from users. Make the jump from early adopters to mainstream user adoption requires that commitment.

Im not a developer per se but I was a functional architect for core enterprise software used by 3 Fortune 500 companies. All the functionality my team specified for developers took as much time to document as to design. And the most important and difficult work was on keeping what was expected where and when expected, not adding the new bells and whistles marketing teams could sell before they were stabilized.

1 Like