Boosting community app development

What motivates you to create apps?

I maintain two apps and have two very exciting ones in the pipeline, all of them were created from a personal need for the functionality and I try to generalize them so other people can profit from them.

What are your main blockers to get your apps published to the app store?

The process isnā€™t very easy to understand in the beginning, but once you get the hang of it, the signing and uploading process is manageable.

What really slowed you down when you just started developing Nextcloud apps?

Nextcloud has a strong backend API and great vue components for the frontend, but there is a missing link in the chain of the data flow.
There is no library to handle the collection of data from the backend to the frontend, it is all manual axios http calls.
If you want anything more than that, you need to abstract it away yourself.
This is in my opinion a missed opportunity, it would be an awesome time saver, if nextcloud had a library that could automatically map backend entities to frontend entities. As most nextcloud apps already use the vue ecosystem and vuex (which will need to be migrated to its sucessor Pinia) these frontend entities would ideally be mapped into Pinia ORM entities.
This could allow for FAR faster development in my opinion, as data structures could just be defined in the backend and automatically exist in the frontend.

Normally this might not be possible to implement, because the data of the entities in the backend travels through Mappers, Services and Controllers, but thankfully appinfo/routes.php definitions already have the concept of resources, which combine the index, show, create, update and destroy actions which would be all, that is necessary for basic backend entitites to frontend entities mapping.

This is the kind of thing only a fully fledged Ecosystem like nextcloud could deliver and it would open the door to MANY cool features, that are not only good for the developer experience, but also for the user experience.
For example: If many apps used the described library, it would be quite easy to implement a Offline First experience for the nextcloud platform, which would make read-only access to app content possible in the browser, while the device is offline and make content loading FAR faster while online, because at first the stored offline copy of data is presented until the newest online version is downloaded and the state is synchronized. This is clearly the way the web is heading (see any google product) and I would like to see nextcloud modernizing in this direction. All that would be neccessary to implement offline first for apps using the above described library would be the pinia persisted state, a Service Worker to cache static files and small modifications of the apps to disable elements that modify data while the data synchonizes or the device is offline.

I would also like to use the opportunity to answer a question, that wasnā€™t asked :smile: , which is what would make me consider moving my dev efforts from the nextcloud plattform to a different one.

One killer feature of an alternative open source plattform, that would make me switch to developing apps for it basically instantly is, if we could get away from data polling from the frontend.
I want to play with the cool new stuff, or in this case the cool new stuff from 12 years ago: Server Sent Events.

Polling is inefficient on traffic and takes up valuable resources on the backend and is a pain to implement, which is why quite a few nextcloud apps only update their data when a context is switched (calendar: deselecting a calender) or only on page reload (passwords app).
This also ties in to my earlier proposal for automatic entities in the frontend as the library could also listen for changes via a Server Sent Event Stream from the Backend (more complicated to implement, as the resources routes ā€œtrickā€ is not applicable).

SSE in nextcloud is not a new idea, it was partially implemented in 2021 (https://github.com/nextcloud/push/pull/27), but sadly (as far as i can tell) never finished and abandonned (nextcloud/push is archived and I could not find any information, if nextcloud/notify_push is meant to be itā€™s successor).
Collaboration is a huge feature for nextcloud, but sadly only the really big apps have the capabilities to implement it as Realtime Collaboration, often relying on external dependencies (such as the onlyoffice documentserver docker) for the realtime communication part over WebSockets or SSE. Built in SSE could make this available to many more apps.

Generally as a developer I would appreciate more architectual improvements to nextcloud. I know features are what customers are paying for, but the nextcloud architecture has to keep up with the time. I hope nextcloud gmbh can get around to more of them, but I know you are trying, as can be seen with stuff like the sync clients no longer having to poll and sync 2.0.

There have been recent advances in getting more programming languages in the nextcloud backend (MediaDC), but I donā€™t think these implementations can get generalized enough (like a whole framework for creating nextcloud apps with python / JS) without backing from the Nextcloud gmbh, so maybe you can allocate a few resources to that :slight_smile: .

Thanks for this opportunity to get these thoughts out there, I planned on creating a forum thread about them for months, but this is better :slight_smile:

6 Likes

Itā€™s a good idea to link the dedicated page for developers in the app developer dropdown of the apps store. Thank you for sharing.

1 Like

Thank you so much for taking the time to share your thoughts with me, and learning more about what you think of the Nextcloud architecture. THANKS!! :slight_smile:

2 Likes

I think thatā€™s a very important point to boost app development efforts: To have an ā€œeasyā€ to run development environment without the need to go through manual installation of all dependencies. I donā€™t think Iā€™ve seen the nextcloud-docker-dev container mentioned anywhere in the documentation?!

I also like to mention @szaimenā€™s nextcloud-easy-test-container: https://github.com/szaimen/nextcloud-easy-test. Itā€™s intended to test different server branches with different app branches (in contrast to juliusā€™s container where the focus is more on development and ā€œbiggerā€ setups). It greatly helps you to get things up and running to test.

5 Likes

as this was referenced

@Daphne may you want to check out Joplinā€™s community.
Plug-In system was introduced in late 2020 and it is steadily growing joplin/plugins: Joplin official plugin repository. For issues, please use https://github.com/laurent22/joplin/issues. I see good activities in Latest Development topics - Joplin Forum (joplinapp.org) which keeps the project alive through a lot of volunteers.
I might be not as complex as NC but it is the momentum in the community that makes it fun to contribute.
You may get some inspiration there. Documentation is not perfect but GSoC said, that is quite helpful w^cominbed with the in-code documentation.

p.s. Iā€™m a GSoC mentor at Joplin (there was even some exchange with NC two years ago)

2 Likes

See this State of app development topic from 2019, which was linked earlier in this thread from the Joplin dev. Post begins:

Also in 2019, there was the TahoeLAFS plugin developer
There was also the TahoeLAFS plugin dev who was upset after being unable to find developer support
Snippet from that discussion:

I have developed a fairly large new application for Nextcloud with essentially no support from the Nextcloud community.

You have responded to my question with a link to a Nextcloud developer that has not responded to me or this thread.

You have marked this as a ā€œsolutionā€ in your response.

Is there any support in this community for developers? If so, what process should I use to ask questions and actually get a response?

1 Like

there was some direct contact with NC as there was one possible project to develop a NC integration but not enough slots were granted by Google to make it happen and NC did not fill the gap.

2 Likes

Your comment give rise to points very interesting !

I didnā€™t know this library and I have to be attentive in future developments of Vue.js

1 Like

New remarks for the documentation.

I have to create a new section tab from files app on Groupfolders only.

But there is no documentation on this.
I had to search in the existing projects to know how I put a new section here.
Andā€¦ I saw I should use the OCA library JavaScript side.
Butā€¦ there is no documentation on this library.

So, I have to copy the code from existing project. Itā€™s like the SOD (Stack Overflow Driven) for me :confused:

1 Like

Maybe some app dev creates a topic to collect all things what are not in the documentation?

1 Like

It would definitely be interesting to learn what caused problems there and what the simplified approach is youā€™ve taken. Iā€™d always be happy to improve the documentation on my setup or adjust if we can make things easier.

2 Likes

This does exist. There are Nextcloud Talk Chatrooms on https://cloud.nextcloud.com. You need somebody with the needed privileges to invite you to be able to join. I remeber that i was invited by Jan-Christoph Borchardt quite a while ago. It should not be hard to get an invite, you probably just need to ask.

The solution I found for me was https://www.gitpod.io. It lets you spin up dev environments in ā€œthe cloudā€ with full root access and Visual Studio Code in mere seconds. If you add a gitpod configuration file to your repository, you can make it automatically start a nextcloud server.
Gitpod is like Github Workspaces, but Open Source, exists longer than Workspaces, can be self hosted and the hosted version has a free version and cheaper prices.
If you want to go this route you can use my Gitpod config for nextcloud apps by copying my .gitpod.yml config file and the gitpod directory next to it and modifying it to fit your own app.

2 Likes

This is fantastic news, @Daphne :tada: Welcome! :hugs:

Iā€™ve been waiting for an initiative like this for a long time, so I just want to add a few words.

I maintain two apps, TimeManager and Link Editor. The biggest issue for me is lacking documentation on almost everything.

My tooling / setup and struggles it addresses

This is what has made life easier for me after a lot of trial & error, in no particular order:

  1. Releases ā€” I wrote a webhook server to publish releases to the appstore that I host on Uberspace to automate the process of publishing releases through a Github webhook. It has the added benefit that my apps are an exact copy of what can be found in my repositories, since no manual steps are involved in the publish process. Before I had this script, it took me a lot of time to publish releases properly, without missing any steps. I think it should be a no-brainer to add publishing tools to the Nextcloud developer ecosystem. Publishing manually is a recipe for disaster, especially for less experienced developers.
  2. Development environment ā€” I use the official nextcloud docker container for development after years of barely getting by with MAMP. How to set up a proper development environment on Windows, Mac and Linux should be in the developer documentation, too. Especially since it needs some hand-holding to get up-and-running properly. Iā€™d be more than happy if the documentation and docker-compose file that I use for my apps could be re-used in Nextcloudā€™s docs.
  3. End-to-end tests ā€” The docker-compose setup I also use to run cypress end-to-end tests in Gitlab CI. Itā€™s not at all perfect, but gives me more confidence when testing with new Nextcloud releases. There seems to be no documentation on end-to-end tests, even though ā€“ in my opinion ā€“ they are much more useful for UI-heavy Nextcloud apps than unit tests. We need documentation on how to write end-to-end tests and how to set up CI for them.
  4. PHP stubs and editor Intellisense ā€” Iā€™ve started using Intelephense for Visual Studio Code and included my local Nextcloud git, as well as some stubs by @ChristophWurst, that can be found in composer, in its config. See this post. We urgently need documentation on how to set up most used editors for development. Iā€™d never have guessed some deprecations or specific call signatures that I now get to see with this setup.
  5. PHP API ā€” This brings me to my last point: There appears to be no public roadmap for PHP APIs. I can never be sure what the latest and greatest way of doing things is (like loading assets, loading the current user object, routing, query building etc.), because things change all the time (with most of the legacy stuff staying around for a while, to be fair). The way I adapted to this, is to browse the core apps every once in a while to figure out what they changed. Also I subscribe to the Github issues on the server repository that mention breaking changes in each release. Finally, Iā€™ve added a step to my CI tests that dump the nextcloud.log file to see if I missed any deprecation warnings or obscure errors during development. To fix this, we urgently need good documentation for all PHP APIs and some info on when theyā€™re no longer recommended to be used and what to use instead and how to use it.
  6. Missing basics ā€” Part of my problem is that I donā€™t understand Symfony very well, which I think most Nextcloud APIs are based on? The developer documentation should link to the Symfony documentation or any other 3rd-party library used where necessary and explain common concepts of that library (like naming, mappers, available super classes etc).

Nextcloud, the company, and developer relations

Iā€™d be happy to help with this effort, but I also see Nextcloud as a company in (deep) debt here, because only they have the resources to hire someone like you to take a first shot at this. And since the whole ecosystem profits from good apps, I think there should be incentive to invest much more into developer relations, but I can remember that we mentioned this avenue to Frank a few years ago and he didnā€™t seem impressed ā€¦ so thereā€™s that :sweat_smile:

Letā€™s look around for good examples

Lastly, I want to suggest to not re-invent the wheel. We should take a look at other projects that have a successful (and diverse in terms of levels, from beginner to expert) app developer community. Iā€™m not saying itā€™s a prime example, but take a look at how good the WordPress documentation has gotten. It has examples on almost all methods, hooks, actions etc. It also has guides now to cover basic concepts and those guides are exhaustive.

I know the Nextcloud developer documentation has guides too, and kudos to the person who wrote them, but theyā€™re really only scratching the surface if weā€™re being honest.


This got much longer than I intended ā€“ hope itā€™s useful regardless.

ā€“ Thomas

6 Likes

Putting it through nextcloudā€™s mechanism for software distribution is just too much overhead to deal with. Iā€™m happy to work on my projects and even share via public git servers (like gitlab), but thatā€™s as far as I go.

As far as Iā€™m concerned, if youā€™d like to publish my work on your system, then provide a form where I can submit a link to a project on GIT (and nothing else), and you can get to scan and parse the project and handle publishing it.

2 Likes

One thing I noticed: Why do I have to use a web form (Generate app - App Store - Nextcloud) to generate a skeleton app to start developing? (You canā€™t even select NC24 as a target). I guess it would make sense (especially after reading the comments above) to offer some kind of CLI tools to: create a skeleton app, register an in the app store, publish an app, etc.
Then suddenly thereā€™s GitHub - ChristophWurst/krankerl: A CLI helper to manage, package and publish Nextcloud apps, which looks like it would solve some of the publishing problems mentioned above, but (much like the docker dev container) itā€™s quite inofficial and not mentioned in the documentation. There are a few good enhancement requests in the repo there:
Add commands for certificate management
Have a GitHub action

I guess this would be really appreciated to have.

Btw, the web form for the skeleton app is not correctly translated ā€¦

2 Likes

@juliushaertl : Yes, I have to take time to explain my problems to use your repo :confused:
But, yes, I imagine you would be happy if we give you feedbacks.

@all : What do you think if we create a new subject on docker, in general not only on Juliusā€™, to improve the development process ?

@TessyPowder : I am in this chatrooms and, Iā€™m sorry, but the Talk app is not like a ā€œDiscord-likeā€ or ā€œSlack-likeā€. You cannot create several channels (you have to invite the person), you cannot write markdown, you cannot create threads (bonus) and so on.
One more, with Talk you joign one group and not a server with channels to exchange.

This app is like Skype, Messenger (Facebook) or MSN Messenger (for old :older_man:).

I think a Mattermost, RocketChat or a Discord-like itā€™s better than Talk app.
But, I like Talk app ! Itā€™s just an app to exchange with one person or a group limited to 5 or 10 people.
By the way, I think itā€™s a good idea to have an app on desktop :wink:

Iā€™m sorry, I donā€™t understand why Gitpod it helped you :thinking:
For me, Gitpod itā€™s just a VScode through the web, not more. No ?

@te-online : I agree with you on everything you said :wink: :clap: :rocket:

@SysKeeper : Oooh ! Very interesting ! I donā€™t know, this project by Christoph Wurst :slight_smile:
But, yes, I think itā€™s very cool to have a command line to generate a skeleton like Laravel or any other tool. In general, I agree that we need a command line to :

  • Generate a project
  • Generate migration files
  • Generate seed files
  • Publish our app
  • etc.

Like artisan of Laravel or Symfony :sweat_smile:

2 Likes

Yes, Talk is not like Discord, but I didnā€™t write about them in my answer, because they are like ā€¦

but because they are used for the ā€¦

like you asked for.
I did not know, that you were already in these Talk rooms.

I think that the Talk rooms fulfill every function, that a discord server would, in terms of actual chatting, not function in terms of chat software features.
I think it is only logical for the nextcloud gmbh to use their own chat tool for their company and community chats.

I would rather propose making their Talk chats easier to use by splitting them up more (there are already a few, but i donā€™t think there is a list, so I only of the ones I am in) and creating a Community Hub Channel with a list and a way to ask/automated way for joining the specific community channels.

Well kind of, but if you screw up your dev environment you can spawn a new one and if you configured it to automatically set up, you will basically never have to debug your dev environment.

Also if you want to work on multiple branches concurrently that require changes to the dev environment you can just spawn another workspace and use both concurrently.

I use this often to test an app on multiple nextcloud versions, I do not have to spin up multiple nextcloud servers on my local machine (annoying port conflicts, clutters your storage, takes a while to install multiple times, download times limited to your internet speed), but just spawn a throwaway workspace and have a quick script, that changes the nextcloud version. After that i can just close the mess and return to a clean workspace.

It was basically the solution to dependency hell for me, which I find the most annoying about dev environments, especially if you work on mutliple things concurrently.

If you donā€™t run into this problem you of course donā€™t need a solution for it :slight_smile:

At least I never want to have a nginx nextcloud dev server on my laptop again.

You can of course recreate this on a local machine by using multiple dockers, but that exactly is what gitpod does, but more quickly and easier.

There would also be benefits to Gitpod, not for personal projects, but for the big community/official nextcloud projects: It makes it incredibly easy for first time contributers to get started. No need to set anything up, just change something in the code and see the result. I think that is quite powerful.

(I would never recommend gitpod if it was closed source btw)

2 Likes

Hmmmā€¦ Yes and No ^^
Thatā€™s not what I meant. I meant, itā€™s better to have a platform, not only exchange message, but help each other with differents channels as I quoted here :

The main of which is to be able to share our code (with Markdown), share our ideas on UX or UI, help each other if we have a problem in our code and so on. All this, with a better flow than Talk app.

I donā€™t agree with you. Itā€™s very complicated to share my code if the markdown itā€™s not present.
One more, exchange on UI, UX, Front-End, Back-End, Database, and so on. On one groupā€¦ Itā€™s a bad idea for fluidity.
Someone who is UX or UI Designer would not talk in the channel of Talk. Because, either they will be drowned out by the messages or because itā€™s a group for developers only.
One more, as a Front-End or Back-End developers, she/he would also feel drowned by the messages.

Maybeā€¦ Or maybe not. Currently, Talk app doesnā€™t meet the requirements I mentioned.
After all, all depends on its orientation, either as a messaging tool like Skype, or like Slack where you can join different channels.
One more, in Talk you can invite one person on a group. If we want to stay on Talk, this means adding people one by one in each group.
If you add a UX or UI designer only in a Talk UI/UX group. You exclude them from other groups and therefore from other exchanges :confused:

But beware, this is just my opinion on the matter :sweat_smile:

In this sense, I agree. But, itā€™s very time-consuming and not user friendly.
Again, it depends on the orientation of the app.

Oooh I see ! Itā€™s very surprising ! :smiley:

The idea is not bad :wink:

But, I think itā€™s heavy to use to create an app quickly.
Today, I use 2 dockers : One on the release 22 and the other on version 23.

I think a small docker based solution is a good idea and if we want to go further, we can propose Gitpod ?

I read prerequisites to install Gitpod Self-Hosted, and I feel like itā€™s a lot to handle, no ?

I havenā€™t used Kubernetes yet ^^ā€™

1 Like

It is not one group, i think you are in just one and donā€™t know of the other rooms. I am for example also in the VueJS Chat, the News App Chat and Frontend/Design Chat, all in different rooms on the same nextcloud instance.
Which just further illustrates the need for a way to discover these groups. If you can only know of them if you are a member, that is quite impractical for new developers.

Absolutely, Nextcloud development of course should be easy, regardless of if you want to use Gitpod.

Until a few months ago, it was basically impossible to understand how to install it. Now they have good documentation.
I have not tried for myself yet, because it is not cost efficient for me to self host gitpod, as the limits of the free hosted version are pretty generous.

If you want self host: definitely, for beginners I meant the hosted version.

1 Like

agree here. It has to be more than simple text.
Things have to be easy to use. A continuous surge of simple text makes conversation quite challenging. In particular, if you begin to share lings URLs, quotes not talking about code yet.
You should give people a chance to leave for a couple of days but to come back on an open conversation.
(that is how I would get attracted to use it :slight_smile: )

1 Like