I worked through most of the Collabora integration guide. It is very helpful but I still have not sorted out my issue. I believe I confirmed my suspicion about routing through the proxy. I did a traceroute from both docker containers.
From the collabora container a traceroute to nextcloud has one hop to nextcloud container:
$ traceroute nextcloud.[domain].com
traceroute to nextcloud.[domain].com (172.20.0.6), 30 hops max, 60 byte packets
1 nextcloud-app-1.nextcloud_default (172.20.0.6) 0.053 ms 0.027 ms 0.025 ms
That makes sense. Since it does not go through the proxy it cannot access the https connection.
From the nextcloud container a traceroute to collabora has two hops through the default gateway:
$ traceroute office.[domain].com
traceroute to office.[domain].com (173.224.xxx.53), 30 hops max, 60 byte packets
1 nextcloud (172.20.0.1) 0.084 ms 0.044 ms 0.029 ms
2 ip-173-224-xxx-53.syringanetworks.net (173.224.xxx.53) 0.204 ms 0.156 ms 0.162 ms
I was trying to figure out what the difference is. I had about given up when I noticed I had set the hostname in the docker-compose.yml file for Next Cloud. I removed that and am able to open a document now. Thanks for your help.