OK, let me chip in. I had to read through the text and found quite some confusion there. I see there are many open questions and I try to answer a few. I like that you find the new documentation useful (I rewrote the REST vs. OCP part recently). If I miss something, sorry for that. Just ask again.
OCS vs (plain) frontend routes
The difference between OCS and plain frontend routes is that OCS encapsulates the response into a structure like this:
{
"meta": {
"status": "ok",
"statuscode": 200
},
"data": { /* Here comes your data */ }
}
Thus it makes no sense to put HTML there. You want the <html> tag to be the top element for the browser to show 
.
So, when you want to transmit data between backend and somewhere with logic attached (be it a browser with JS, a native mobile app, a desktop app, a webapp, or whatever), you can go with OCS. If you need to stick with a predefined API (defined by some other entity like e.g. you want to realize an OAuth scheme), you will probably have to go with plain controller routes.
There was a recent discussion about REST vs. OCS and the docs was even more contradicting than now. In the discussions I had with various people, I found that OCS is generally speaking the way to go as there are some minor differences that make them slightly more convenient as a programmer if you ever think about allowing 3rd party integrations (like mobile apps etc).
Controllers vs Routes
Addressing your main question here, you have to understand what routes and what controllers are:
- Controllers are the classes that encapsulate the data as it is present in PHP as part of your business logic and prepare the data to be transmitted to the client. There are different controllers available with different features on board. Each method of a controller can serve data (generally speaking, could be real data, HTML, or whatever) to the user.
- Responses are a way in the PHP/NC/Symphony world to be returned from a controller method. They contain the actual data to be transmitted but as well meta data like additional headers, cookies to be set, the status code, etc. Think of them as pure data classes holding all data requrired for the core to ship the answer to the user.
- Routes link a controller method with an HTTP endpoint. So you can tell which HTTP request will be served by which method.
Let’s have a look at the various parts.
Routes
The legacy way of defining routes is/was to use the routes.php file. There you had to return an assoc array of arrays [see 1 and 2].
A route defined under routes is representing a plain route to /index.php/apps/<yourappname>/<url>. (This is true, even if your apps are installed in a subfolder custom_apps or similar!) If .htaccess or similar is in place, a prefix of /index.php can be neglected and you see typical routes like /apps/<yourappname>/<url>.
A route defined in ocs is representing an ocs route, that has a different prefix of /ocs/v2.php/<yourappname>/<url>.
There are actually two routes attributes (#[FrontendRoute] and #[ApiRoute]) available. These are just syntactic sugar. Instead of centrally defining the routes in the routes.php, the controller methods can get attributes to them to link the route implicitly with the controller method. The Route class is just a common abstract class if you wanted to define your own routes handling (you do not want that now).
Controllers and controller methods
Generally, a controller can only serve only OCS requests or only non-OCS requests. This is by definition and how the internal search algorithm works. So mixing them in one controller is not possible.
Depending on whether you want to use OCS or not, you must extent either OCSController or Controller. Obviously, is mokes no sense to combine a OCS route with a plain Controller or vice versa.
The PublicShareController and AuthPublicShareController, I have never used. As far as I can guess, these are used by files sharing and similar. But I have to admit, I would have to look out for them individually.
The ApiController is a somewhat extended Controller. It makes only sense for data-only responses. When you want to enable CORS, you need to have an ApiContoller which defines some headers when CORS is activated (preflighted).
Responses
There are some responses that you will, some that you might and some that you will not need.
You will probably need the TemplateResponse. This is the way to return HTML data to the user. By default, the HTML is pregenerated for you and all you have to provide is a stub that is inserted in the middle of the page. Header, footer and the like is handled by the Response object itself while rendering. You can prevent this, but… yeah.
The PublicTemplateResponse is the same as TemplateResponse but renders the public page (aka no head line etc).
Then, you have DataResponse. This is the typical way to answer to OCS requests. OCS has the special case that it can by default return JSON and XML depending on the users requests (and you do not have to do a thing about it). Therefore the data response is a quite common structure that can be marshalled as both JSON or XML.
If you use Controller or ApiController to return data, you might want to use JSONResponse to make clear this is a JSON and the corresponding header is set as well. I thing DataResponse will work here as well.
The others are quite uncommon, I guess. As far as I know, the DownloadResponse allows you to ship a file to the user so that he gets a save as dialog shown from the browser. The FileDisplayResponse is similar but the file is shown inside the browser. Think of a PDF that can be stored on disk or previewed in the browser.
Security and middlewares
Normally, a request from the user comes to the NC core. There the corresponding route is selected. Then all middlewares are run that are registered. These check for attributes (that is BTW how the routes are actually found but ignore this piece of magic for now) and eventually do additional stuff. E.g. by default a user is not allow to access any route. You have to define the attribute #[NoAdminRequired] to allow normal users to access a certain route. There is a middleware [3] that will handle just that.
Similarly, you need to allow public access (#[PublicPage]) or can configure the rate limit protection (#[UserRateLimit]).
Ok, sorry, I have to leave for now. It was a long question and possible multiple ansers are given. I wil come back. If you have more in depth questions, just post. I will try to answer as I can.
Chris