Installing NextCloud AIO behind a reverse proxy (nginx) with a Docker network

Hi,

I have some problem to install NextCloud AIO behind a reverse proxy (nginx) with docker network managed and multiple docker-compose projets.

I’ve followed this documentation : all-in-one/reverse-proxy.md at main · nextcloud/all-in-one · GitHub

  • First my setup without NextCloud:

The docker-compose of my reverse proxy with a certbot for (sub)domain https certificates:

version: '3.9'

services:
  reverseProxy:
    container_name: reverseProxy
    image: nginx:latest
    restart: always
    volumes:
      - ./nginx.conf/:/etc/nginx/conf.d/:ro
      - ./certbot.conf/:/etc/letsencrypt/:ro
      - ./certbot.www/:/var/www/certbot/:ro
    ports:
      - 80:80
      - 443:443
    networks:
      - frontend

  certbot:
    image: certbot/certbot:latest
    container_name: certbot
    volumes:
      - ./certbot.conf/:/etc/letsencrypt/:rw
      - ./certbot.www/:/var/www/certbot/:rw
    depends_on:
      - reverseProxy
    networks:
      - frontend

networks:
  frontend:
    external: true

Beside the reverse proxy I have others docker-compose apps distributed int frontend and backend docker networks. Both network use the internal Docker DNS resolver to resolve IP by container_name

Here is also my Nginx config file:

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    server_name <domain.net> <www.domain.net> ... <sub.domain.net> ;
    server_tokens off;

    location / {
        return 301 https://$host$request_uri;
    }

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

}

server {
    server_name <sub.domain.net>;

    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    <config for internal server proxy passed>
}
.
.
.
  • NextCloud AIO configs:

So now I’m trying to add NextCloud AIO as anoter app to my config.

NextCloud AIO docker-compose:

services:
  nextcloud-aio-mastercontainer:
    image: nextcloud/all-in-one:latest
    init: true
    restart: always
    container_name: nextcloud-aio-mastercontainer # This line is not allowed to be changed as otherwise AIO will not work correctly
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config # This line is not allowed to be changed as otherwise the built-in backup solution will not work
      - /var/run/docker.sock:/var/run/docker.sock:ro # May be changed on macOS, Windows or docker rootless. See the applicable documentation. If adjusting, don't forget to also set 'WATCHTOWER_DOCKER_SOCKET_PATH'!
    ports:
      - 8080:8080
    environment: # Is needed when using any of the options below
      - APACHE_PORT=11000 # Is needed when running behind a web server or reverse proxy (like Apache, Nginx, Cloudflare Tunnel and else). See https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md
      - APACHE_IP_BINDING=0.0.0.0 # Should be set when running behind a web server or reverse proxy (like Apache, Nginx, Cloudflare Tunnel and else) that is running on the same host. See https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md
      - NEXTCLOUD_DATADIR=/mnt/ncdata # Allows to set the host directory for Nextcloud's datadir. ⚠️⚠️⚠️ Warning: do not set or adjust this value after the initial Nextcloud installation is done! See https://github.com/nextcloud/all-in-one#how-to-change-the-default-location-of-nextclouds-datadir
    networks:
      - nextcloud-aio
      - frontend

volumes:
  nextcloud_aio_mastercontainer:
    name: nextcloud_aio_mastercontainer # This line is not allowed to be changed as otherwise the built-in backup solution will not work

networks:
  nextcloud-aio:
    name: nextcloud-aio # This line is not allowed to be changed as otherwise the created network will not be used by the other containers of AIO
    driver: bridge
  frontend:
    external: true

the reverse proxy with NextCloud proxy pass:

map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
}

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    server_name <domain.net> <www.domain.net> ... <sub.domain.net> ;
    server_tokens off;

    location / {
        return 301 https://$host$request_uri;
    }

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

}

server {
...
}

server {
    server_name <nextcloud.domain.net>;
    server_tokens off;

    # SSL Certificate
    ssl_certificate /etc/letsencrypt/live/<nextcloud.domain.net>/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/<nextcloud.domain.net>/privkey.pem;
    # verify chain of trust of OCSP response using Root CA and Intermediate certs
    ssl_trusted_certificate /etc/letsencrypt/live/<nextcloud.domain.net>/chain.pem;

    # HTTPS Common Parameters + Mozilla's best practices config file for HTTPS
    include /etc/letsencrypt/ssl-config.mozilla.org.conf;

    # NextCloud AIO config
    http2 on;                                 # uncomment to enable HTTP/2        - supported on nginx v1.25.1+
    http3 on;                                 # uncomment to enable HTTP/3 / QUIC - supported on nginx v1.25.0+
    quic_retry on;                            # uncomment to enable HTTP/3 / QUIC - supported on nginx v1.25.0+
    add_header Alt-Svc 'h3=":443"; ma=86400'; # uncomment to enable HTTP/3 / QUIC - supported on nginx v1.25.0+
    listen 443 quic reuseport;       # uncomment to enable HTTP/3 / QUIC - supported on nginx v1.25.0+ - please remove "reuseport" if there is already another quic listener on port 443 with enabled reuseport
    listen [::]:443 quic reuseport;  # uncomment to enable HTTP/3 / QUIC - supported on nginx v1.25.0+ - please remove "reuseport" if there is already another quic listener on port 443 with enabled reuseport - keep comment to disable IPv6

    location / {
        proxy_pass http://nextcloud-aio-mastercontainer:11000$request_uri;

        # NextCloud AIO recommanded proxy config
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Scheme $scheme;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Accept-Encoding "";
        proxy_set_header Host $host;
    
        client_body_buffer_size 512k;
        proxy_read_timeout 86400s;
        client_max_body_size 0;

        # Websocket
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
    }
}

SSL Common config (ssl-config.mozilla.org.conf)

# generated 2023-08-25, Mozilla Guideline v5.7, nginx 1.17.7, OpenSSL 1.1.1k, intermediate configuration
# https://ssl-config.mozilla.org/#server=nginx&version=1.17.7&config=intermediate&openssl=1.1.1k&guideline=5.7

listen 443 ssl http2;
listen [::]:443 ssl http2;

ssl_session_timeout 1d;
ssl_session_cache shared:MozSSL:10m;  # about 40000 sessions
ssl_session_tickets off;

# curl https://ssl-config.mozilla.org/ffdhe2048.txt > /path/to/dhparam
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

# intermediate configuration
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305;
ssl_prefer_server_ciphers off;

# HSTS (ngx_http_headers_module is required) (63072000 seconds)
add_header Strict-Transport-Security "max-age=63072000" always;

# OCSP stapling
ssl_stapling on;
ssl_stapling_verify on;

# set Docker internal DNS as DNS resolver
resolver 127.0.0.11 valid=10s;
resolver_timeout 5s;
  • First init

With this config I can connect to the initial setup but I get this error message when submit my domain

Domain does not point to this server or the reverse proxy is not configured correctly. See the mastercontainer logs for more details. (‘sudo docker logs -f nextcloud-aio-mastercontainer’)

And the log result:

xxxx@YYYYY:/Docker$ sudo docker logs -f nextcloud-aio-mastercontainer
Trying to fix docker.sock permissions internally...
Creating docker group internally with id 134
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
 Storage Driver: vfs
Warning: It seems like the storage driver vfs is used. This will lead to problems with disk space and performance and is disrecommended!
..+.+.....+.+++++++++++<lot of . and +>++++++++++......+.+++++
-----
Initial startup of Nextcloud All-in-One complete!
You should be able to open the Nextcloud AIO Interface now on port 8080 of this server!
E.g. https://internal.ip.of.this.server:8080

If your server has port 80 and 8443 open and you point a domain to your server, you can get a valid certificate automatically by opening the Nextcloud AIO Interface via:
https://your-domain-that-points-to-this-server.tld:8443
[Tue Aug 29 22:13:24 2023] PHP 8.2.8 Development Server (http://127.0.0.1:9876) started
{"level":"info","ts":1693347204.0887535,"msg":"using provided configuration","config_file":"/Caddyfile","config_adapter":""}
[Tue Aug 29 22:13:24.090565 2023] [mpm_event:notice] [pid 114:tid 139720481213256] AH00489: Apache/2.4.57 (Unix) OpenSSL/3.1.2 configured -- resuming normal operations
[Tue Aug 29 22:13:24.090950 2023] [core:notice] [pid 114:tid 139720481213256] AH00094: Command line: 'httpd -D FOREGROUND'
[29-Aug-2023 22:13:24] NOTICE: fpm is running, pid 120
[29-Aug-2023 22:13:24] NOTICE: ready to handle connections
</html>nter>nginx</center>y</h1></center>d>nnection attempt to "https://<nextcloud.domain.net>:443" was: <html>
NOTICE: PHP message: Expected was: ad26c03670c48a573e906df480b739b136cca0522ba071ee
NOTICE: PHP message: The error message was: 

Am I missing some point(s) ?

Hi, can you follow https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md#6-how-to-debug-things?

Ok so, I followed the step till point 7, testing the connection between my reverse-proxy and NextCloud AIO. Netcat command could not contact the container on port 11000.

In my config I thought I should set the proxy to pass the requests to the “nextcloud-aio-mastercontainer” but unfortunately it’s not so simple.

I tried first to proxy_pass to the localhost of the host machine by adding

extra_hosts:
    - "host.docker.internal:host-gateway"

and use

proxy_pass http://host.docker.internal:11000$request_uri;

I also had to add a rule to allow port 11000 on the firewall to have a success netcat connection.

But even with this, the domain verification fail on the first init.

If I try to proxy pass to the public IP adresse of my host (and keep port 11000 open in firewall) it works.

proxy_pass http://X.X.X.X:11000$request_uri;

This network configuration seems a little bit weird, why localhost of host trough Docker is not working ? I don’t like the idea to hard writing my public IP address to the reverse proxy config.

And isn’t there a network configuration that would allow traffic to be confined into Docker networks ?

So you added this as extra host in the reverse proxy container?

Did using netcat from the container to host.docker.internal:11000 work then?

Indeed, I added these lines to my reverse proxy container.

Netcat works if I allow port 11000 in the firewall (I’m on Ubuntu server 20.04 LTS so I used ufw)

$ docker exec -it reverseProxy /bin/bash -c "nc -zv host.docker.internal 11000"
Connection to host.docker.internal (172.17.0.1) 11000 port [tcp/*] succeeded!

But the AIO init failed at the domain validation

Domain does not point to this server or the reverse proxy is not configured correctly. See the mastercontainer logs for more details. ('sudo docker logs -f nextcloud-aio-mastercontainer')

Here 's the logs of the master container

</html>nter>nginx</center>y</h1></center>d>nnection attempt to "https://<nextcloud.domaine.net>:443" was: <html>
NOTICE: PHP message: Expected was: b067f3a57d5654b43b93c7af5f5978412d6ef43383bbb577
NOTICE: PHP message: The error message was: 

What do you see now if you open https://yourdomain.com:443 ? Do you see the same string?

I have a 502 Bad Gateway error from the reverse proxy.

Yet I’m using the Docker DNS resolver (127.0.0.11) in the nginx config.

Then still something is incorrectly configured

I tested a little but further and I have strange results.

I tested 3 setups:

  1. Custom Network with Extra_Host

I used a Docker compose custom network (“frontend” like in my first post, the one I use to server other containers).
So I added in the reverse proxy’s docker compose file:

extra_hosts:
  - "host.docker.internal:host-gateway"

And in the nginx.conf I used:

proxy_pass http://host.docker.internal:11000$request_uri;

I launched the reverse proxy and NextCloud-AIO containers. Then tested the connection

$ docker exec -it reverseProxy /bin/bash -c "nc -zv host.docker.internal 11000"
Connection to host.docker.internal (172.17.0.1) 11000 port [tcp/*] succeeded!

It succeeded (I had to open the port 11000 in my FW btw).
But I still get this error message when submit my domain

Domain does not point to this server or the reverse proxy is not configured correctly. See the mastercontainer logs for more details. (‘sudo docker logs -f nextcloud-aio-mastercontainer’)
  1. Custom Network with Public server name

The reverse proxy still use the custom network but i used the public URL of my server instead.
So I didn’t add anything in the reverse proxy’s docker compose file (no extra_hosts: option).
And in nginx.conf I used:

proxy_pass http://<nextcloud.domain.net>:11000$request_uri;

I launched the reverse proxy and NextCloud-AIO containers. Then tested the connection

$ docker exec -it reverseProxy /bin/bash -c "nc -zv <nextcloud.domain.net> 11000"
Connection to <nextcloud.domain.net> (X.X.X.X) 11000 port [tcp/*] succeeded!

It succeeded (I had to open the port 11000 in my FW btw).
But I still get this error message when submit my domain

Domain does not point to this server or the reverse proxy is not configured correctly. See the mastercontainer logs for more details. (‘sudo docker logs -f nextcloud-aio-mastercontainer’)
  1. localhost in host mode

I stop using a custom network and bind the reverse proxy container to the host network by adding this to the revers proxy’s docker compose file:

network_mode: host

In nginx.conf I used:

proxy_pass http://localhost:11000$request_uri;

I launched the reverse proxy and NextCloud-AIO containers. Then tested the connection

$ docker exec -it reverseProxy /bin/bash -c "nc -zv localhost  11000"
Connection to localhost (127.0.0.1) 11000 port [tcp/*] succeeded!

It succeeded (I didn’t had to open any port in the FW).
But I continue to get this error message when submit my domain

Domain does not point to this server or the reverse proxy is not configured correctly. See the mastercontainer logs for more details. (‘sudo docker logs -f nextcloud-aio-mastercontainer’)
  • IP instead of hostname

Now for each test, if I use the respective IP address instead of the hostname:
172.17.0.1 instead of host.docker.internal
X.X.X.X instead of <nextcloud.domain.net>
127.0.0.1 instead of localhost
the domain’s validation succeeded!

  • Conclusion

It seems that when using a hostname, the NextCloud domain’s validation always fails even though the netcat command succeeded in the reverse proxy container!

But when using the respective IP instead of the hostname the validation pass.