Need to know how to setup Nextcloud to use SSL

You’re right: if you want Nextcloud to play nicely with the reverse proxy, there are some entries to be made in Nextcloud’s config file, and also some recommended tweaks to your reverse proxy config. You can find instructions and an example in the Nextcloud admin docs here.
If you want a more full example, you can see my configs for Nextcloud, the Apache server in my Nextcloud container, and my Apache reverse proxy at the end of this post on the forum.

As far as using SSL straight from the docker image without reverse proxy… it looks like it should just work, so I’d want to see the logs next.

Just to be sure, is that config directory you refer to the same as this one:
./proxy/conf.d:/etc/nginx/conf.d ?
If you’re modifying something else, then you’ll need that file/folder to be listed under ‘volumes’ too, or it will be reset whenever you restart the docker image. The ’ Replacing default proxy settings’ section of that docker help page says to modify /etc/nginx/proxy.conf, but doesn’t tell you this.
If you are modifying something that’s listed under ‘volumes’ and it still resets, I’d ask other users of this image what the deal is, because that seems weird to me.

HTTP 400 Bad Gateway means the proxy can’t reach Nextcloud on the specified port, but HTTP 500 Internal Server Error could be sent by the reverse proxy, by the web server in your Nextcloud container, or by Nextcloud itself. Next step would be to check through the logs for all of the above.