Update: It appears that Collabora configuration has changed in the latest releases, so @bb77 's solution will probably also fail soon. I realised when I checked the docker logs:
-e ERR: Use of domain variable is not supported. First host/domain who tries to connect to COOL is always allowed.
To allow multiple host and its aliases use something like this and pass it as env variable:
aliasgroup1=https://domain1:443,https://its-alias|its-second-alias:443
aliasgroup2=https://domain2:443,https://its-alias:443
For more info: https://sdk.collaboraonline.com/docs/installation/CODE_Docker_image.html
The linked article confused me a bit, and I did not get it running. Then I discovered that the mentioned configuration apparently will only be available in the upcoming docker release (https://forum.collaboraonline.com/t/fixing-code-multihost-configuration/1001/8)
For the moment one has to change it inside the docker file. I managed to get it running again with the description mentioned here: https://forum.collaboraonline.com/t/21-11-3-4-1-problem-with-multiple-domains/991/3
Thought this may be helpful to others as well.