I also run the Docker version but I reverse proxy with another instance of Apache and run certbot there as well. So basically the same, minus nginx. Maybe someone who knows nginx can chime in.
The only header I have to modify on Apache reverse proxy is for HSTS. If you’d be interested in trying it this way instead or maybe just want to compare notes, I wrote a guide for it.