Step back for a second, think about this from a fresh AIO installation perspective and not salt in URL from posts that really don’t apply to the question.
The typical environment, from what I’m seeing from the forum:
Let’s start with a Linux host with Docker.
The installation is in a DMZ behind NAT.
Let’s use 192.168.1.100 as the Linux host address.
Let’s use 8.8.8.8 and 8.8.4.4 as the host’s DNS servers.
AIO makes a number of assumptions that we’re just supposed to accept. That’s great until things don’t work properly.
AIO creates a nextcloud-aio network on which the nextcloud-aio containers can all “see” each other via name and a unique host ID assigned to them by virtue of Docker’s built in container DNS.
The nextcloud-aio network addressing for all nextcloud-aio containers is DHCP-based. Container names and IDs don’t change, but their IP will.
Let’s use 172.18.1.0/24 as the nextcloud-aio network.
The nextcloud-aio network is separate from the host’s network, the bridged network.
Nextcloud port publishing, Docker:
Depending on the AIO installation the AIO script publishes various ports to the host - going behind a reverse proxy nexctloud-aio-mastercontainer publishes 8080 while apache publishes 11000 and your reverse proxy publish 443. If you are not using a reverse proxy nextcloud-aio-apache publishes 443 to the host, and 8080 and 8443 for nextcloud-aio-mastercontainer.
Ports published to the host by Docker for containers on the nextcloud-aio network are NAT’d against the Host’s IP address of 192.168.1.100.
How the Linux host/Nextcloud installation interacts with the internet:
On the Internet the firewall forwards ports/NATs the hosts IP - 192.168.1.100 - to an external IP - let’s use 130.57.66.5 and “nextcloud.ncaio.com”.
The firewall does not know about the nextcloud-aio network, it is host-only, behind the host NIC, a docker only network.
Outbound traffic to the Internet from the 192.168.1.100 Linux/Proxy host will appear to come from 130.57.66.5 at hostname “nextcloud.ncaio.com”. This is by virtue of NAT at the firewall.
Because of the way Docker is set up, depending on firewall/iptables on the host for the Docker networks, outbound Internet traffic from the Containers on the 172.18.1.0/24 nextcloud-aio network will appear to come from 130.57.66.5 at hostname “nextcloud.ncaio.com”.
The default configuration is to allow outbound internet traffic from the containers via the Linux host IP - only limited by the firewall (allowed or denied).
172.18.1.x port 443 → 192.168.1.100:443 → www.microsoft.com:443 from 130.57.66.5:443
The linux server will resolve “nextcloud.ncaio.com” to 130.57.66.5.
Because of the way Linux and Docker work by default any of the Linux containers will see “nextcloud.ncaio.com” as 130.57.66.5 via the DNS server setting of 8.8.8.8
On NAT:
You can’t “avoid NAT” - it’s how even the NextCloud AIO default installer creates the containers on the nextcloud-aio network - their outbound traffic is NAT’d by default.
Most people with an ISP even if they have static IP will use NAT - unless Nextcloud assumes they’re putting their hosts directly on the Internet with a static IP which is a risk.
I have to ask - does Nextcloud assume people are putting their servers directly on the Internet with a fully routable static IP address? That could explain some of the behavior of the installation, but I would hope that is not the case.
On DNS & the AIO setup: domaincheck
The linux server and the containers in Docker that reside on it will resolve “nextcloud.ncaio.com” to 130.57.66.5.
When Nextcloud-AIO first starts the nextcloud-aio-mastercontainer is spawned. At this point the nextcloud-aio network does not exist.
A notice will appear in the CLI letting the installer know that they need to access their nextcloud-aio-mastercontainer from an INTERNAL IP address at port 8080 - https://192.168.1.100:8080 (this will occur if the install is reverse proxy or not).
Once the installer accesses https://192.168.1.100:8080 the nextcloud-aio network is created and the nextcloud-aio-domaincheck container is spawned.
If you are setting up via reverse proxy and you want it connected to the nextcloud-aio network your first hurdle is waiting for that network to be created, but even if not the nextcloud-aio-domaincheck will publish a port to the host - port 11000 (using the apache-port variable for reverse proxy installs) or - port 443 (for a non-proxied install).
Note: If you are trying to set up a reverse proxy that sits on the nextcloud-aio network you can start the AIO installer script, access “your NC internal IP:8080”, spawn the nextcloud-aio network creation, then open up a second console and kick off your reverse proxy container creation script, upload the config file. imho this is messy and may not always work but if Nextcloud is already using Caddy why not make an AIO installer option that creates the off apache-container reverse proxy with all of the required settings…
The nextcloud-aio-domaincheck container will attempt to validate the domain - and here is the first hurdle.
The nextcloud-aio-domaincheck nextcloud-aio network IP is 172.18.1.3, it’s docker DNS name is nextcloud-aio-domaincheck, it’s publishing port 443 (keeping it simple) to the host at 192.168.1.100 while it’s outbound traffic is appearing to come from 130.57.66.5 which resolves to “nextcloud.ncaio.com”.
The domaincheck isn’t really a domain check, it’s a host check, asking for the FQDN of the host Nextcloud will be operating on.
The NAT logic may forward the traffic to 130.57.66.5 NAT’d behind 192.168.1.100 coming from 172.18.1.3 - but the return traffic typically isn’t from 130.57.66.5 as the firewall knows both the VIP is on the firewall and is represented by the Linux host.
While from a remote PC I can resolve “nextcloud.ncaio.com” and see the published domain key that PC is seeing “nextcloud.ncaio.com” simply by it’s NAT’d IP of 130.57.66.5.
Q: How is the nextcloud-aio-domaincheck container supposed to see “nextcloud.ncaio.com” @ 130.57.66.5 from 192.16.1.100? I’ll see things like NAT hairpin, etc. but while you can’t expect people to be docker experts suddenly they’re supposed to be network experts? BTW a hairpin will work from another network to the virtual IP (NAT’d external address) but may not work with a host on the same network.
If you add “nextcloud.ncaio.com” to the hosts file of the host - nextcloud.ncaio.com 192.168.1.100 - the nextcloud-aio-mastercontainer is smart enough to know that’s not a valid external IP and will show errors in the logs.
You can’t modify the nextcloud-aio-domaincheck container as it’s read only.
In all of these cases the containers, host, can resolve DNS perfectly well - “nextcloud.ncaio.com” = 130.57.66.5
*DNS and --env APACHE_PORT=11000 *
What complicates things even further is trying to do a reverse proxy install with --env APACHE_PORT=11000 \ as part of the installation. Why?
Because when the nextcloud-aio-domaincheck comes up if you have a configuration that uses the container name you’ll have it configured for nextcloud-aio-apache by default - you need to remember to change that as the reverse proxy is installed (during the Nextcloud install because only then is the nextcloud-aio network created, after the nextcloud-aio-domaincheck container is created and running).
If you use 0.0.0.0, 127.0.0.1, etc. for your reverse proxy configuration you still have to address the NAT and the fact that in any case the nextcloud-aio-domaincheck will still publish port 11000 to the host network.
Your reverse proxy will publish 443 to the host at 192.168.1.100 which on the internet is 130.57.66.5. The reverse proxy - with it’s own intermediate network or not - will typically have a leg in the nextcloud-aio network - so in that case the reverse proxy at 172.18.1.4 will attempt to talk to the domain checker container at 172.18.1.3 using Docker DNS OR will attempt to connect on the host network depending on how it’s all set up.
It is inescapable that the nextcloud-aio-domaincheck will attempt to check the DNS key it is publishing at “nextcloud.ncaio.com” on 130.57.66.5 by default.
Q: So how, without continuing to expand the installation to yet another container, are you supposed to configure DNS for a successful domain check process to run. Exactly how. IMHO Nextcloud owes that clear explanation for any installation to ensure success. IMHO they are depending on hairpin NAT/NAT reflection/NAT loopback at the network level.
Yesterday I took down my AIO install for a rebuild from scratch test. No firewall changes. No host changes. AIO and Caddy. Ran all of the same scripts. Domain checking failed for 8hrs, every install until the very last one. No idea why it worked.
And yes, any machine could resolve DNS - internal Docker, externally resolvable. Could every container curl https://nextcloud.ncaio.com - no - but that’s because the 192.168.1.100 NAT’d host cannot access 130.57.66.5 - it is 130.57.66.5.
At note on Domaincheck logs:
There are none.
It’s amazing but I can’t “check the logs” to see why the domain check is failing. Unless I’m missing a logfile somewhere it would be really helpful to know what is happening. Even enabling debug for the install isn’t helpful. The mastercontainer? No useful logs.
Regarding Collabora and NextCloud:
This also goes back to the question of how NextCloud AIO expects domain - really host - resolution to work.
During the installation - any installation - of Nextcloud AIO - there is a Caddy installation on the nextcloud-aio-apache container.
When you go to the Administration… Nextcloud Office… “URL and Port of the Collabora server” what awaits you? The FQDN of the server you entered in the AIO domain validation step - https://nextcloud.ncaio.com
In the case of Collabora the inbound Collabora traffic is sent to https://nextcloud.ncaio.com - 130.57.66.5 on the Internet, 192.168.1.100 on the Linux host, traffic proxied at port 443 and forwarded through the Caddy proxy on nextcloud-aio-apache (172.18.1.5) to nextcloud-aio-collabora (172.18.1.6) via port 9980 on the nextcloud-aio network.
If you are using a reverse proxy 443 is forwarded to 11000 (apache) which is then forwarded to 9980 (collabora)…
In this configuration every system - the Linux host, containers, etc. - can resolve nextcloud.ncaio.com to 130.57.66.5, the containers can all resolve their own host names at 172.18.1.x on the nextcloud-aio network by virtue of Docker.
As a URL and port (implied) for Collabora the default - https://nextcloud.ncaio.com - has never worked in our environment.
Error: “Could not establish connection to the Collabora Online server. Failed to connect to the remote server: cURL error 28: https://nextcloud.ncaio.com/hosting/discovery”
When you put in http://nextcloud-aio-collabora:9980 guess what?
“Collabora Online server is reachable.”
What appears as " URL used by the browser:" and Nextcloud URL used by Collabora: ( Determined from the browser URL)?
The fqdn of the internet address specified in the nextcloud AIO domain validation: https://nextcloud.ncaio.com
The only problem?
Nextcloud office doesn’t work - try to access a document:
Document loading failed
Failed to load Nextcloud Office - please try again later
In the logs?
CheckFileInfo failed for URI https://nextcloud.ncaio.com/index.php/apps/richdocuments/wopi/files…
Invalid URI or access denied to https://nextcloud.ncaio.com/index.php/apps/richdocuments/wopi/files…
Are my WOPI settings correct, yes.
Can I resolve DNS, of course.
Does it appear that - reverse proxy or no - that NC Office functions. No.
In any case nextcloud-aio-apache is proxying nextcloud-aio-collabora traffic on the Apache-based Caddy proxy.
How should DNS be configured - exactly - to make this work?
In the case of a Caddy proxy in front of Apache it’s really 2 Caddy proxies - the one for 443 and the other for the Collabora 9980 traffic sent via 11000.
I’ve been through all of the documented troubleshooting steps for Collabora. None of them have solved the problem. The only solution I’ve found is via Hairpin NAT.
My original question was:
“How is the nextcloud URL resolved at the back end - does it need to, for Apache/Collabora to work?”
The response? A Docker DNS link… when I know docker DNS is working fine - and nextcloud.ncaio.com resolves to 130.57.66.5 but that doesn’t work for Collabora - only HTTP://nextcloud-aio-collabora:9980 “works” to allow me to access WOPI settings but never to use office.
The question remains - how is Collabora supposed to resolve nextcloud.ncaio.com and where must that be configured for a reverse proxied or non-proxied installation. I’ve tried both, neither work.
A: You need hairpin NAT.
As far as from WWE:
“running the system behind reverse proxy results in all internal office requests going trhough the internet which is no ideal from networking”
What does that mean from a technical perspective and where is that documented? The AIO reverse proxy installation never publishes port 9980 on the nextcloud-aio-collabora container nor does the non reverse proxied installation - the Collabora traffic is proxied behind the Caddy proxy on the nextcloud-aio-apache container.
A link to “Docker and Wireguard” isn’t relevant to my question - this is a very simple stock AIO installation - Linux, Docker, AIO, Google DNS.
It really begs the question if we should be installing NextCloud behind a reverse proxy at all but a degree of separation from Apache directly on the Internet isn’t a bad thing, IMHO.
If reverse proxy installs are supported - and the AIO script has variables to support this very thing - should all of the prereqs be clearly documented?
Internal LAN access to NextCloud-AIO:
When WWE says “I’m not sure I misunderstand your statement but in general” what I am saying is that from the Internet the world not on the Linux host see https://nextcloud.ncaio.com as 130.57.66.5.
That traffic is Internet (NAT) → DMZ.
Now add in local LAN users - LAN → DMZ.
Their network is not on the Internet - let’s say it’s 10.0.0.x/24 (192 for DMZ, 172 for nextcloud-aio and 10 for LAN to keep things simple). Let’s say their internal domain is internaldomain.local.
For them nextcloud.ncaio.com resolves to 130.57.66.5 via their internal DNS forwarding traffic to DNS servers authoritative for ncaio.com. Problem is the firewall knows they are internal users trying to access a host on the DMZ.
Solution?
Hairpin NAT.
This allows a host on the 10.0.0.x/24 network to resolve nextcloud.ncaio.com to 130.57.66.5 and access 130.57.66.5 as it it were a client on the internet. This prevents having to split DNS to have nextcloud.ncaio.com pointing to both 192.168.1.100 for internal and 130.57.66.5 for external clients.
Could we resolve 192.168.1.100 to nextcloud.ncaio.com, sure, but again this creates an additional issue depending on how you have DNS set up for your nextcloud installation - and requires additional DNS servers to make this function.
Is this a nextcloud requirement?
If so it should be fully documented to ensure that the system works as intended when installed - if DNS is essential no equivocation on forums with a lot of guess work but clearly documented.
In Closing:
When I see “AFAIK there is nothing special [about AIO]” and a bunch of “but this” after that statement IMHO it’s a sign there are some things that are special. Maybe they are Nextcloud generally or AIO specifically but they are not clearly documented and leave some open questions.
The use of NAT, unless people are putting servers directly on the Internet, is in widespread use. Even AIO uses it by default. It’s largely inescapable. The DNS configuration in AIO assume hairpin NAT - at least from the AIO host to it’s own external IP.
How - from the very start - DNS is resolved to the nextcloud hostname in domain checking is critical to getting an install started. I’m not running NC with “domain validation off” because I lose traffic encryption at the back end and have no idea what the downstream impact is to the functioning of my install.
How MUST DNS be configured? My installation is as vanilla as it can get - network, firewall, NAT, Linux, Docker, AIO. I’ve made no changes to any of the AIO networks (having been told I can’t), DNS resolution is simple - 2 external servers. If I set localhost, etc. to 192.x the “domaincheck” logic flags it as local and refuses to complete.
How do the various aspects of NC communicate - and where does that occur - is critical. If Collabora behind a Caddy proxy in front of, not on, Apache, needs to communicate over the Internet the AIO script doesn’t publish those ports to the host so how is that to work? I’ve not see that in Caddyfiles, etc.
It appears that some of the problems are not as much a DNS issue as a potential network/NAT issue. A container that is trying to access the host’s NAT’d external address NAT’d behind the hosts non-NAT’d address can resolve the external DNS name just fine - the fact that it can’t curl (access) the web page hosted there is an assumption based on networking and the script logic in the mastercontainer that won’t allow the external DNS to resolve to an internal RFC 1918 non-routable address.
IMHO some of the Nextcloud folks have “Expert syndrome”. That’s not a criticism or an insult, it’s just an observation and it happens all the time, in any profession. For any of us in IT for a long time “thinking as a noob” may be a challenge as what is second nature to us us unknown to others.
As a home user looking for an option to stop running out of Google workspace and have my data held for ransom I think NC is a really interesting option. As a business person having a way to contain company data could be very compelling to others to take control of their data - as Nextcloud intended.
Looking at the forum I see patters emerge especially around domain checking - a process that is poorly logged and undocumented - which prevents people from even getting started with their installs. As noted by many the posts mentioned in responses are not always relevant and are generally not a clear solution.
I understand this is a long post, hopefully it answers some of the questions as to “why the heck this idiot is asking these questions” by providing as much context as possible.
I think Nextcloud is a compelling concept, product.
I think this forum could really assist people in having an easier time installing and ultimately using the product and evangelizing it.
I think this forum may also be a way to let Nextcloud know they could do better on documentation and support, which is never a bad thing, and better support the community.
I look forward to hearing from you.