Both Nextcloud and Collabora appear to be working correctly when i go to these URLs, but I get the following error message in Nextcloud on the Office settings page.
Your browser has been unable to connect to the Collabora server: http://collabora.***.com
This URL is determined on the Collabora server either from the configured URL or the server_name parameter in coolwsd.xml.
(Note, I have inputted the URL as https, not http as appears in the error message)
I would guess something is wrong with trusted_proxies and/or X-FORWARDED-* http headers. cloudflare magic is known to brake things - try first without fancy stuff like script compression etc…
I needed to change the extra parameters variable in docker from extra_params=--o:ssl.enable=false to extra_params=--o:ssl.enable=false --o:ssl.termination=true, and it fixed the issue.