you may look at the paulczar/omgwtfssl docker image. it’s a docker container to generate certificate. additional in the readme you find hints how to use these certificates.
where did you get the error? blank browser page? browser page saying wrong certificate?
the docker gurus use traefik or nginx (or haproxy) for this purpose.
that’s not a docker problem.
an example setup with traefik in front of several docker container you can find here: GitHub - ReinerNippes/nextcloud_on_docker: Run Nextcloud in Docker Container on various Linux Hosts
some ideas about nextcloud cluster you find here: Help needed to setup a NC14 cluster - #5 by Finalls
everything in the diagram you can put in a docker container.