Confusion about Controllers, Routes & Responses

đź’» Development
, , , ,
1

Hey there,

currently I’m having a go on trying to implement a custom NextCloud app - native, not ExApp. While I already got some things working, I have a hard time understanding how to correctly use Controllers, Routes & Responses. Basically, I’d like to know when to use which of the respectively available subtypes and how to combine them.

I’ll explain in more detail below and also link to the available resources & discussions that I’ve already found. If I’ve overlooked some available documentation or thread that already clarifies these questions, I’d be glad to receive some pointers. Moreover, if I can get some answers in this thread, I’ll try to suggest improvements to the docs on nextcloud/documentation: :blue_book: Nextcloud documentation, but it didn’t seem wise to start that discussion over there.

Okay, hereI we go. I’m aware of the following things.

Controllers

AFAIK there are 5 controller classes available, as documented here:

I already found some discussions about ApiController vs OCSController, namely in Dev manual is contradicting re: OCS versus REST - :laptop: Development, which also lead to dev-doc updates in Issue #12157 · nextcloud/documentation / Pull Request #12264 · nextcloud/documentation with the changes now public here. TL;DR: OCS is preferred over REST and hence, OCSController should be preferred should be preferred. The page also explains how using OCSController affects any DataResponse return values:

Methods from Controller classes can return DataResponse objects similar to OCSController class methods. For methods of a Controller class, the data of this response is sent e.g. as JSON as you provide it. Basically, the output is very similar to what json_encode would do. In contrast, the OCSController will encapsulate the data in an outer shell that provides some more (meta) information. For example a status code (similar to the HTTP status code) is transmitted at top level. The actual data is transmitted in the data property.

Routes

AFAIK there are 3 Route classe available, as documented here:

  • Route, base class: ref
  • FrontpageRoute: ref
  • ApiRoute: ref

Developer documentation for all three is mostly available here. Moreover, there is a comment in the app dev tutorial, specifically in part 6, which reads

There are 2 types of network APIs in Nextcloud:

the internal API (defined by the FrontpageRoute PHP attribute) and the OCS API (defined by the ApiRoute attribute).

The internal API is supposed to be created and consumed by the same developers. Since the API is not used by other developers, it does not have to be stable in time and developers can change their internal APIs whenever they wish.

The OCS APIs can be used internally but can also allow clients to interact with an app. Therefore, it must be stable to avoid breaking the clients. This is why we include the API version number in the endpoint paths. For example, any change breaking the version 1 of our API will lead to creating a version 2. This way we can keep version 1 untouched and old clients can still work because they still target version 1.

From this description, it’s not clear to me if

  • FrontpageRoute endpoints are protected at runtime to be accessible through the nextcloud frontend or
  • if this is simply a convention discouraging external services to use these endpoints, but no actual enforcement happens

Responses

AFAIK there are response classes available, as documented here and here:

The dev-docs of JSONResponse say

The usage of standard controller to access content data like JSON (no HTML) is considered legacy. Better use OCS for this type of requests.

Based on the comments on ApiController and OCSController, I would hence expect, that this hint boilds down to using OCSController with DataResponse. However, I noticed that the example snippets here use the plain Controller with JSONResponse instead of OCSController.

On REST APIs — Nextcloud latest Developer Manual latest documentation, the snippets suggest to use ApiController for REST APIs.

In part 6 of the tutorial linked above, they use DataResponse on a plain Controller and on a OCSController. Moreover, they use DataResponse both with FrontpageRoute and ApiRoute. TemplateResponse is used only with FrontpageRoute.

On OCS OpenAPI tutorial — Nextcloud latest Developer Manual latest documentation, the use TemplateResponse in some examples, but the overall guide says that OCSController should be used which to my understanding is meant to be used together with DataResponse.

On Routing — Nextcloud latest Developer Manual latest documentation it says

FrontpageRoute has to be used for routes that were in the routes section and ApiRoute has to be used for routes that were in the ocs section.

and

Registering of OCS routes requires to use a corresponding controller. Additionally, the route must be configured to be an OCS route in the router.

In addition to PublicTemplateResponse and TooManyRequestsResponse, I’ve also seen the attributes

And now what?

With the above summary I hope to illustrate the wide range of available controllers, routes, responses and even attributes and that from the available documentation resources it’s not straightforward to understand which of those objects should be used for which use cases in which combination.

I would very much like to get some guidelines on how to combine controllers, routes, responses & attributes for standard use cases, maybe similar to the ones listed here, which are:

  • Access from web frontend means the user is accessing the Nextcloud web frontend with a web browser.
  • Access from non-browser is if the user accesses the resource or page using something that is not a web browser, like an Android app or a curl command.
  • Access from external website means that the user browses some third party web site and data from your Nextcloud server appears. The other website has to embed/load/use images, JSON data, or other resources from a URL pointing to the Nextcloud server, to be able to do this.

The use cases of endpoints serving GUIs (e.g. the NextCloud frontend) and endpoints serving data/APIs (i.e. frontend-backend communication or public APIs) certainly need to be separated.

Additionally, I have some specific questions in addition to the general understanding

  • If certain combinations (e.g. router & controller, response type & router) are desired or even necessary, are there any runtime checks for these?
  • What is the difference between using the PublicPage attribute and returning a PublicTemplateResponse response?
  • What effects exactly do controllers, routes & attributes have on the endpoints in terms of
    • effective url
    • data format
    • CSRF, CORS and possibly other security mechanisms?
  • does FrontpageRoute indeed “hide” the endpoint from access outside the NextCloud API or is that a misunderstanding? If that’s not the case:
    • what’s the benefit using using DataResponse with FrontpageRoute as done in the tutorial - shouldn’t that rather be done through OCSController + ApiRoute, then?
    • What’s the recommended approach to split an internal API used for communication between frontend & backend from an external API that’s allowed to be used directly by users or 3rd parties?
  • what would be a good place to suggest improving the class documentation of the different PHP classes such that the API reference is more informative? The nextcloud/server repository or the nextcloud/documentation repository?

This has become quite a bit longer than I had anticipated. Nevertheless, I’m glad for any hints, tips, replies or redirections to existing threads. Please excuse me if I’ve missed some things that might be more intuitive after more experience with NextCloud development or web development in general.

Cheers :slight_smile:

2

Hi @Bibo-Joshi , as far as i know on this subject, the internal API from your app is your “free” api that you should use in your interface. The OCS Api is a commitment to sustainability that you could avoid, unless you need to make your app APIs public.

You had a lot of questions, i propose only some responses:
PublicPage is a MiddleWare attribute that let your internal api being un-authentified (your api might be rest/json by ex.)
PublicTemplateResponse is a “complete web html page” response. The purpose in general is to create a whole application page.

Most DataResponse derivated classes you’ve mentionned are in general subclasses of simple JSON responses. But using some standard formats. Use it when you can, and probably you’ll may use some Vue components adapted to these formats.

Yours,

3

Hey @smarinier thanks for your quick reply!

Do I understand correctly, that the split between private & public API is hence not enforced in any way but rather a concenvent? I.e. OCS endpoints should be considered public & stable, non-OCS endpoints are considered private and should not be assumed to have stability guarantees?

I see, thanks. What I’m still missing then is the relation to (Auth)PublicShareController. On Controllers — Nextcloud latest Developer Manual latest documentation, PublicTemplateResponse is used with a Controller without the #[PublicPage] attribute. Wouldn’t the attribute be required by your logic? On Public Pages — Nextcloud latest Developer Manual latest documentation they use #[PublicPage] even in combination with PublicShareController …

IISC the response classes mostly derive directly from Response, never from JSONResponse o.0
I understand that some responses are a bit self-explanatory, but others are confusing. What’s the difference between DataDisplayResponse and FileDisplayResponse? What’s the difference between DataDownloadRespone and DownloadResponse? The documentation is really descriptive IMHO :sweat_smile:

4

OK, let me chip in. I had to read through the text and found quite some confusion there. I see there are many open questions and I try to answer a few. I like that you find the new documentation useful (I rewrote the REST vs. OCP part recently). If I miss something, sorry for that. Just ask again.

OCS vs (plain) frontend routes

The difference between OCS and plain frontend routes is that OCS encapsulates the response into a structure like this:

{
  "meta": {
    "status": "ok",
    "statuscode": 200
  },
  "data": { /* Here comes your data */ }
}

Thus it makes no sense to put HTML there. You want the <html> tag to be the top element for the browser to show :wink:.

So, when you want to transmit data between backend and somewhere with logic attached (be it a browser with JS, a native mobile app, a desktop app, a webapp, or whatever), you can go with OCS. If you need to stick with a predefined API (defined by some other entity like e.g. you want to realize an OAuth scheme), you will probably have to go with plain controller routes.

There was a recent discussion about REST vs. OCS and the docs was even more contradicting than now. In the discussions I had with various people, I found that OCS is generally speaking the way to go as there are some minor differences that make them slightly more convenient as a programmer if you ever think about allowing 3rd party integrations (like mobile apps etc).

Controllers vs Routes

Addressing your main question here, you have to understand what routes and what controllers are:

  • Controllers are the classes that encapsulate the data as it is present in PHP as part of your business logic and prepare the data to be transmitted to the client. There are different controllers available with different features on board. Each method of a controller can serve data (generally speaking, could be real data, HTML, or whatever) to the user.
  • Responses are a way in the PHP/NC/Symphony world to be returned from a controller method. They contain the actual data to be transmitted but as well meta data like additional headers, cookies to be set, the status code, etc. Think of them as pure data classes holding all data requrired for the core to ship the answer to the user.
  • Routes link a controller method with an HTTP endpoint. So you can tell which HTTP request will be served by which method.

Let’s have a look at the various parts.

Routes

The legacy way of defining routes is/was to use the routes.php file. There you had to return an assoc array of arrays [see 1 and 2].

A route defined under routes is representing a plain route to /index.php/apps/<yourappname>/<url>. (This is true, even if your apps are installed in a subfolder custom_apps or similar!) If .htaccess or similar is in place, a prefix of /index.php can be neglected and you see typical routes like /apps/<yourappname>/<url>.

A route defined in ocs is representing an ocs route, that has a different prefix of /ocs/v2.php/<yourappname>/<url>.

There are actually two routes attributes (#[FrontendRoute] and #[ApiRoute]) available. These are just syntactic sugar. Instead of centrally defining the routes in the routes.php, the controller methods can get attributes to them to link the route implicitly with the controller method. The Route class is just a common abstract class if you wanted to define your own routes handling (you do not want that now).

Controllers and controller methods

Generally, a controller can only serve only OCS requests or only non-OCS requests. This is by definition and how the internal search algorithm works. So mixing them in one controller is not possible.

Depending on whether you want to use OCS or not, you must extent either OCSController or Controller. Obviously, is mokes no sense to combine a OCS route with a plain Controller or vice versa.

The PublicShareController and AuthPublicShareController, I have never used. As far as I can guess, these are used by files sharing and similar. But I have to admit, I would have to look out for them individually.

The ApiController is a somewhat extended Controller. It makes only sense for data-only responses. When you want to enable CORS, you need to have an ApiContoller which defines some headers when CORS is activated (preflighted).

Responses

There are some responses that you will, some that you might and some that you will not need.

You will probably need the TemplateResponse. This is the way to return HTML data to the user. By default, the HTML is pregenerated for you and all you have to provide is a stub that is inserted in the middle of the page. Header, footer and the like is handled by the Response object itself while rendering. You can prevent this, but… yeah.

The PublicTemplateResponse is the same as TemplateResponse but renders the public page (aka no head line etc).

Then, you have DataResponse. This is the typical way to answer to OCS requests. OCS has the special case that it can by default return JSON and XML depending on the users requests (and you do not have to do a thing about it). Therefore the data response is a quite common structure that can be marshalled as both JSON or XML.

If you use Controller or ApiController to return data, you might want to use JSONResponse to make clear this is a JSON and the corresponding header is set as well. I thing DataResponse will work here as well.

The others are quite uncommon, I guess. As far as I know, the DownloadResponse allows you to ship a file to the user so that he gets a save as dialog shown from the browser. The FileDisplayResponse is similar but the file is shown inside the browser. Think of a PDF that can be stored on disk or previewed in the browser.

Security and middlewares

Normally, a request from the user comes to the NC core. There the corresponding route is selected. Then all middlewares are run that are registered. These check for attributes (that is BTW how the routes are actually found but ignore this piece of magic for now) and eventually do additional stuff. E.g. by default a user is not allow to access any route. You have to define the attribute #[NoAdminRequired] to allow normal users to access a certain route. There is a middleware [3] that will handle just that.

Similarly, you need to allow public access (#[PublicPage]) or can configure the rate limit protection (#[UserRateLimit]).

Ok, sorry, I have to leave for now. It was a long question and possible multiple ansers are given. I wil come back. If you have more in depth questions, just post. I will try to answer as I can.

Chris

1 Like
5

Hello again,

I am back from yesterday’s session and I want to address dome of your direct questions.

Generally, this is hard to answer as I do not know the kind of app you have in mind. For the beginning, I see however only a few typical use cases, that can be described.

Description Example Route Controller Response Remarks
Serve HTML page Entrypoint for app FrontendRoute or routes Controller TemplateResponse
Serve public HTML page Public sharing for anonymous access FrontendRoute or routes Controller PublicTemplateResponse
Data transmission Send data between backend and frontend ApiRoute or ocs OCSController DataResponse opinionated preference
File transmission Send images between backend and frontend FrontendRoute or routes Controller FileResponse or whatever suits best
Data transmission Send data between backend and frontend FrontendRoute or routes Controller DataResponse or JSONResponse as a fallback
Data sharing Allow data to be accessed from foreign websites ApiRoute or ocs OCSController DataResponse Enable CORS (but understand the implications!)
File providing Access files from foreign website FrontendRoute or routes ApiController as needed Implement security as needed

Mainly you have to separate OCS from non-OCS. So, either route & controller are bothe OCS or neither is.

I would say to prefer OCS over non-OCS as there are some more options with OCS related to security. This makes live easier. The argument about consistent API, I would say, is not too pressing. You (as app dev) can define how stable a certain endpoint is.
If you want to provide a stable API, you could define a subset of your endpoints as stable, nevertheless.

I think there might be some checks but it could be that if you mix OCS with non-OCS, your routes might simply be not found/reachable. I’d say give it a try and see what happens.

PublicPage is just a security configuration. It means anyone can access the route, you do not need to be logged in.
PublicTemplateResponse is a Response, that renders a HTML page in the public style without navigation bar to all apps.

I think this should be clear after the last message, mostly.

URL is controlled by the corresponding route. The data format is controlled by the question of OCS/non-OCS and the Reponse object returned from the controller.

All endpoints are CSRF-protected by default. You could disable this using #[NoCSRFRequired]. But you shoud not do this. Instead work correctly with CSRF.

  • In the frontend, you can use the session CSRF token e.g. by using the @ nextcloud/axios package that handles CSRF for you. I heard that classical fetch would work as well but I would have to think it through.
  • If you have an OCS route, you just need to provide the OCS-APIRequest: true header to satisfy CSRF checks. Nice for apps etc. [1]

[1]: Remark: Since a recent version of NC released, also normal routes allow the OCS-APIRequest: true header. It sounds sort of strange but, hey…

The usage of CORS is easier with ApiController or OCSController as the (pre-flight) header handling is mostly included with them. You have to register the route for the pre-flight(s) nonetheless, though.

The #[CORS] attribute on the controller methods enables the CORS handling. Just be warned not to use it without thinking.

It is a misunderstanding. It just means a normal route based on the index.php file where you can configure the response freely.

As DataReposonse is automatically converted to JSON AFAIK, it is convenient as you can simply switch between OCS and non-OCS by just switching the controller base class and the routes’ attributes/settings in routes.php.

There is no recommended way. In the cookbook app (where I refactored the API quite some time ago), I have a documentation for devs [1, 2]. I have a prefix /webapp for internal API requests and /app for external ones. I do not like it that way as it doubles the work needed to maintain the app (both internal and external APIs need care).

I will migrate to a common OCS-based API in the long-term. Then, I can merge internal and external without a problem there. The point is that also in the internal API I do want stability once the app is out. So, I can still define e.g. /api/v1 for the first version and if I have breaking changes go to /api/v2.

That depends. The API reference is the condensed documentation. Although there have been enhancements recently, I still tend to look at the server code directly. For more complex descriptions, it might be better to go with the documentation as it is more feature rich (like real tables, cross-links, chapters/sections, …).

OCS ist not necessarily more stable. It depends on how you communicate stuff to fellow devs. You can happily use OCS internally. REST endpoints (non-OCS) can be perfectly stable (see cookbook).

I would say, the documentation tells you already quite some stuff. This is intended if your app should share stuff to the public. This can be either completely anonymously (PublicShareControlelr) or with a password (AuthPublichShareController). This again defines the styling of the page (and the fact that the user should type password).

One shows plain data (binary data in PHP) and the other returns the content of a file in the NC instance.

The DownloadResponse is a helper class that you should not use. It renders no content to the client.

1 Like
6

Hello @christianlupus ,

first of, let me thank you for putting the time into these detailed replies! I appreciate it :slight_smile:

I guess my takeaway is that the server interface is more flexible than my newbie-brain would like. In that flexibility it also allows configurations that are just not useful for any sensible use case. I’m sure that this as both many benefits and probably also some historic reasons.

What is extremely helpful IMO is your table. If it’s ok for you, I’ll suggest to include something like that in the development manual in the “Basics” section (as rule of thumb). The only edit I would suggest (based on your later comments) is to use (Auth)PublicShareController for “Serve public HTML page” instead of Controller (hope I got that right :sweat_smile: ).

Here I’m still not sure if I completely understand. I get that OCSController applies some additional OCS-logic to data. But is that depending on the URL (does /ocs/v2.php implement that logic?) or is it the controller itself that has the effect? Could I also use #[ApiRoute] with ApiController, resulting in an /ocs/v2.php/<appname>/urlthat does not serve OCS content but plain data content? The other way around (serving OCS content via #[FrontpageRoute] and Controller) is apparently not possible based on the documentation (“Additionally, the route must be configured to be an OCS route in the router.”)

I see. That means I could in theory I could also return a PublicTemplateResponse via Controller, leading to a logged-in user seeing a “public” page - which doesn’t make sense as use case, but is allowed. Defining a method without #[PublicPage] inside PublicTemplateResponseis possible as well, but doesn’t make either b/c then non-logged-in users can’t see the page.

I would have hoped there to be one, but fair enough. I guess in the end it always down to how you document and communicate things … I’ve also seen the “scope” parameter of the OpenAPI attribute. Maybe specifying something like internal there could be used as additional means to differentiate between internal & public API if necessary.

Uh, that’s interesting as DownloadResponse is explicitly listed at Controllers — Nextcloud latest Developer Manual latest documentation :hushed_face: In any case, for if something is not meant to be used by app developers, it would be nice IMHO if either it weren’t part of the public API at all or at least had a corresponding note in the class docs …

In general, I think the namings made it harder for me to get a grip of the setup. IMO:

  • ApiRoute should be named OCSRoute as it’s apparently intended to serve OCS data only
  • FrontpageRouteshould be named FrondendRoute or similar to avoid the impression that it’s only about “pages”, i.e. HTML websites
  • #[PublicPage]should be named #[PublicEndpoint] or similar for the same reason
7

One more follow-up question: What’s the use case for #[AuthorizedAdminSetting]?

From this statement, I understood that admin-access is opt-out rather than opt-in. But AuthorizedAdminSetting sounds like opt-in :melting_face:

PS: If I understand Settings — Nextcloud latest Developer Manual latest documentation correctly, then this attribute is required only if I’m using this delegated admin settings feature. In this case, admin-only controllers need to use #[AuthorizedAdminSetting] as otherwise, the frontend would not work for the delegated admins. Is that about right?

8

I have no problem with that. I am a bit busy ATM and cannot do much on the doc until December or January. After that, I might be able to help with the docs as well (again). Eventually ping me.

We should look at the details and differences between the various pages to make this very clear. I am not 100% sure as I did not use either of these controllers yet.

I can only tell you that the default apps all use no PublicSharingController except for the files_sharing.

OK, I give you only the statement: Don’t do it. It will not work. Either use OCS all-in or not at all for a single controller.

If we want to discuss this further down the route, open a new topic as this is then way more in-depth.

Exactly. :+1:

I suspect, you mean not inside a response but returning one, but yeah, it’s sort of the same statement.

Maybe, but I am unsure if the OpenAPI generator will work then. No clue, you would have to test it and not invest too much time into it.

The main question is: Why do you want to separate concerns? Believe me, it is a burden and imposes quite some hindering in terms of development progress as you need to keep both APIs in mind.

Yeah, that sounds strange. Maybe I misunderstood the source code and there is some magic in place? Test it out. Use an app or create one, install it and see what happens if you call the API endpoint. I guess, you will get the empty string but keep me posted.

If it is actually returning the empty string, probably the FileDownloadResponse was meant. We should then definitively adjust the docs.

I am thinking of something similar. In fact, I think these classes should be abstract. Then, you could not instantiate them directly and PHP would fail and you see a clear problem. I will ask this at appropriate location.

Unfortunately, this is out of scope. The current names are established and we would have to rewrite all apps out there as this might be a breaking change. The devs will love you for that.

This sis how I understand it as well. But I have not looked in depth here. Just a quick glance at the NC server code hints into this direction.

I hope you have now most of your questions solved.
Chris