[SOLVED] Trying to run Nextcloud-AIO behind a Caddy Reverse Proxy

kromsam · December 9, 2023, 9:28pm

I am at a complete loss.

For a while now, I have been trying to install Nextcloud All in One behind a Reverse Proxy. The guide recommends using- network_mode: "host". But I want to avoid using this. The reason is that I want to portforward the Caddy container, because my OpenMediavault Dashboard is already at ports 443 and 80, and I want to keep it that way. (My router is correctly set up so that 443 and 80 direct to the Caddy Container).

Here is my docker-compose configuration:

services:
  # Nextcloud
  nextcloud-aio-mastercontainer:
    image: nextcloud/all-in-one:latest
    init: true
    restart: always
    container_name: nextcloud-aio-mastercontainer
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config
      - /var/run/docker.sock:/var/run/docker.sock:ro
    ports:
      - 8083:8080
    environment:
      - APACHE_PORT=11000
      - APACHE_IP_BINDING=0.0.0.0
      - SKIP_DOMAIN_VALIDATION=true
    networks:
      - caddy

  caddy:
    image: caddy-docker-proxy-cloudflare-dns
    restart: always
    container_name: caddy
    environment:
    - CADDY_DOCKER_CADDYFILE_PATH=/etc/caddy/Caddyfile
    ports:
      - 82:80
      - 446:443
    networks:
      - caddy
    volumes:
      - ${APPDATA}/Caddy/Caddyfile:/etc/caddy/Caddyfile
      - /var/run/docker.sock:/var/run/docker.sock

networks:
  caddy:
    external: true

volumes:
  nextcloud_aio_mastercontainer:
    name: nextcloud_aio_mastercontainer
  caddy_data: {}

Here is my CaddyFile:

{
  # Email
  email <email>

  # Debug
  # acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
  # debug
}

(cloudflare) {
    tls {
     dns cloudflare <API_TOKEN>
  }
}

https://cloud.domain.tld:443 {
  import cloudflare
  reverse_proxy 172.30.0.1:11000 {
    transport http {
                tls_insecure_skip_verify
        }
  }
}

172.30.0.1 is the result of ip a | grep "scope global" | head -1 | awk '{print $2}' | sed 's|/.*||' which was suggested here: https://github.com/nextcloud/all-in-one/discussions/2364

The error in Caddy with this set up is:

dial tcp 172.30.0.1:11000: i/o timeout

I don’t know what more I can do.

Earlier in my process I had an ‘Page isn’t redirecting properly’ error. Than I added tls_insecure_skip_verify , following the advice here: https://caddy.community/t/the-page-isn-t-redirecting-properly-when-runn/8219

More advice was shared here, but I couldn’t really follow: https://github.com/nextcloud/all-in-one/discussions/2043

szaimen · December 9, 2023, 9:31pm

Hi, can you follow https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md#6-how-to-debug-things?

kromsam · December 9, 2023, 9:43pm

Yes. I did follow all these steps before. The only thing I didn’t do in the previous example is add

extra_hosts: ["host.docker.internal:host-gateway"]

to the caddy-container

and change the following line in the Caddyfile:

reverse_proxy host.docker.internal:11000

With this configuration, the error becomes:

dial tcp 172.17.0.1:11000: i/o timeout

I also tested inside the container:

docker exec -it caddy sh

nc -z localhost 11000; echo $?
1

nc -z host.docker.internal 11000; echo $?

This does not give any response, I just keep waiting. (Which would also connect to the i/o timeout error.

szaimen · December 9, 2023, 9:49pm

After doing so did you already did this? https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md#4-open-the-aio-interface

kromsam · December 9, 2023, 10:11pm

So the Nextcloud container is ‘healthy’ now. But these are the logs:

[09-Dec-2023 23:07:39] NOTICE: fpm is running, pid 683

[09-Dec-2023 23:07:39] NOTICE: ready to handle connections

Activating Collabora config...

Failed to activate any config changes

Server error: `GET https://cloud.domain.tld/hosting/discovery` resulted in a `502 Bad Gateway` response

And all the Caddy logs still look like this:

logger=http.log.error
msg=dial tcp 172.17.0.1:11000: i/o timeout
status=502

Just to be sure all relevant information is out there. My DNS is hosted at Cloudflare, but I have turned off the Proxy.

When I visit cloud.domain.tld, I just see a blank page (with a HTTPS certificate though).

szaimen · December 9, 2023, 10:21pm

What is the value of APACHE_IP_BINDING ?

kromsam · December 9, 2023, 10:22pm

hasn’t been changed, so still 0.0.0.0

kromsam · December 9, 2023, 10:38pm

Just to see what would happen I changed it to 127.0.0.1.

This is the outcome.

Nextcloud logs:

“”"

System config value trusted_proxies => 0 set to string 127.0.0.1

System config value trusted_proxies => 1 set to string ::1

Config value base_endpoint for app notify_push set to https://cloud.domain.tld/push

richdocuments 8.2.3 installed

richdocuments enabled

Config value wopi_url for app richdocuments set to https://cloud.domain.tld/

System config value allow_local_remote_servers set to boolean true

No ipv6-address found for cloud.domain.tld.

Config value wopi_allowlist for app richdocuments set to <public_ip>,127.0.0.1/8,192.168.0.0/16,172.16.0.0/12,10.0.0.0/8,fd00::/8,::1

spreed 17.1.3 installed

spreed enabled

Added cloud.domain.tld:3478.

Added cloud.domain.tld:3478.

There is nothing to delete.

Added signaling server https://cloud.domain.tld/standalone-signaling/.

Config value recording_servers of app spreed deleted

System config value enabledPreviewProviders => 0 set to string OC\Preview\Imaginary

System config value preview_imaginary_url set to string http://nextcloud-aio-imaginary:9000

+ '[' true = true ']'

+ '[' 11000 = 443 ']'

+ '[' 127.0.0.1 = 127.0.0.1 ']'

++ dig nextcloud-aio-apache A +short +search

++ grep '^[0-9.]\+$'

++ sort

++ head -n1

+ IPv4_ADDRESS_APACHE=192.168.96.10

++ dig nextcloud-aio-apache AAAA +short +search

++ grep '^[0-9a-f:]\+$'

++ sort

++ head -n1

+ IPv6_ADDRESS_APACHE=

++ dig nextcloud-aio-mastercontainer A +short +search

++ grep '^[0-9.]\+$'

++ sort

++ head -n1

+ IPv4_ADDRESS_MASTERCONTAINER=192.168.96.2

++ dig nextcloud-aio-mastercontainer AAAA +short +search

++ grep '^[0-9a-f:]\+$'

++ sort

++ head -n1

+ IPv6_ADDRESS_MASTERCONTAINER=

+ sed -i 's|^;listen.allowed_clients|listen.allowed_clients|' /usr/local/etc/php-fpm.d/www.conf

+ sed -i 's|listen.allowed_clients.*|listen.allowed_clients = 127.0.0.1,::1,192.168.96.10,,192.168.96.2,|' /usr/local/etc/php-fpm.d/www.conf

+ sed -i '/^listen.allowed_clients/s/,,/,/g' /usr/local/etc/php-fpm.d/www.conf

+ sed -i '/^listen.allowed_clients/s/,$//' /usr/local/etc/php-fpm.d/www.conf

+ grep listen.allowed_clients /usr/local/etc/php-fpm.d/www.conf

listen.allowed_clients = 127.0.0.1,::1,192.168.96.10,192.168.96.2

+ set +x

[09-Dec-2023 23:33:20] NOTICE: fpm is running, pid 710

[09-Dec-2023 23:33:20] NOTICE: ready to handle connections

Activating Collabora config...

Failed to activate any config changes

Server error: `GET https://cloud.domain.tld/hosting/discovery` resulted in a `502 Bad Gateway` response

Caddy logs haven’t changed.

kromsam · December 11, 2023, 12:38am

On the Caddy forum I also asked for help;

And I also did some findings. I added the caddy container to the nextcloud-aio network. Here are some outputs I got while experimenting:

Caddyfile

reverse_proxy nextcloud-aio-apache:11000

Caddy logs

logger=http.log.error
msg=tls: first record does not look like a TLS handshake
status=502

Caddyfile

reverse_proxy 192.168.96.10:11000

Caddy logs

logger=http.log.error
msg=tls: first record does not look like a TLS handshake
status=502

But might the above two error messages show that I am on to something here?

Caddyfile

reverse_proxy host.docker.internal:11000

Caddy logs

logger=http.log.error
msg=dial tcp 172.17.0.1:11000: i/o timeout
status=502

Caddyfile

reverse_proxy localhost:11000

Caddy logs

logger=http.log.error
msg=dial tcp 127.0.0.1:11000: connect: connection refused
status=502

Really hope someone can help me out here. I really have no idea what is going on. With all these tests I also turned on the Cloudflare DNS proxy on again btw.

kromsam · December 11, 2023, 1:57am

Hooray, I fixed it!

This is my Caddyfile:

{
  # Email
  email <email>

  # Debug
  # acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
  # debug
}

(cloudflare) {
  tls {
    dns cloudflare <api_token>
  }
}

https://cloud.domain.tld:443 {
  import cloudflare
  reverse_proxy nextcloud-aio-apache:11000
}

docker-compose:

services:
  # Nextcloud
  nextcloud-aio-mastercontainer:
    image: nextcloud/all-in-one:latest
    init: true
    restart: always
    container_name: nextcloud-aio-mastercontainer
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config
      - /var/run/docker.sock:/var/run/docker.sock:ro
    ports:
      - 8083:8080
    environment:
      - APACHE_PORT=11000
      - APACHE_IP_BINDING=0.0.0.0

  caddy:
    image: caddy-docker-proxy-cloudflare-dns
    restart: always
    container_name: caddy
    environment:
    - CADDY_DOCKER_CADDYFILE_PATH=/etc/caddy/Caddyfile
    ports:
      - 82:80
      - 446:443
    networks:
      - caddy
    volumes:
      - ${APPDATA}/Caddy/Caddyfile:/etc/caddy/Caddyfile
      - /var/run/docker.sock:/var/run/docker.sock

networks:
  caddy:
    external: true

volumes:
  nextcloud_aio_mastercontainer:
    name: nextcloud_aio_mastercontainer
  caddy_data: {}

yes, I did leave the domaincheck on. because if you change the caddyfile line to reverse_proxy nextcloud-aio-domaincheck:11000 before you open nextcloud-aio for the first time, the domaincheck works. Only make sure to change it back to reverse_proxy nextcloud-aio-apache:11000 before you actually install the nextcloud containers.

LorenDB · December 19, 2023, 11:54pm

@kromsam I’m also trying to get Nextcloud AIO running behind Caddy but for some reason I can’t adapt your solution to my server. My NC docker-compose.yaml is functionally identical to yours with the exception that I have port 8080 bound to 8080 on the host network and I have caddy in a separate docker-compose.yaml (I have a lot of different things running on this server, so I want to keep everything separated for ease of maintenance). Before anyone asks, yes, caddy is on the same Docker network as the Nextcloud container.

The relevant part of my Caddyfile looks like this:

https://nextcloud.my.domain:443 {
       reverse_proxy nextcloud-aio-mastercontainer:11000
}

I’ve verified that port 8080 on the host is accessible to the web. I’ve also done some investigating by using curl to inspect the container:

loren@kerbol$ curl 172.29.0.2:8080
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
Reason: You're speaking plain HTTP to an SSL-enabled server port.<br />
 Instead use the HTTPS scheme to access this URL, please.<br />
</p>
<hr>
<address>Apache/2.4.58 (Unix) Server at localhost Port 8080</address>
</body></html>
loren@kerbol$ curl 172.29.0.2:11000
curl: (7) Failed to connect to 172.29.0.2 port 11000 after 0 ms: Couldn't connect to server
loren@kerbol$

When I attempt to access the base domain (https://nextcloud.my.domain), I get an HTTP 502 response, which I believe is just caddy’s default response when it tries to reverse proxy to something that doesn’t respond. When I access https://nextcloud.my.domain:8080 (I’ve set up port forwarding on port 8080 temporarily), I get a NET::ERR_CERT_AUTHORITY_INVALID error reported from my browser.

Do you (or anyone else) have insights as to what I’ve done wrong?

livethisdream · December 22, 2023, 6:08pm

I spent a long time on getting this up and going for Mac OS X (info here).

Try modifying your Caddyfile:

 https://nextcloud.my.domain:443 {
       reverse_proxy host.docker.internal:11000
}

source

That’s what got me going. Otherwise, you can try the simple caddy image I reference in my repo.

LorenDB · February 23, 2024, 5:19pm

Here’s a (very late) followup:

It turns out I was trying to access port 8080 over WAN; you have to access it over LAN. Once you do so, the AIO container will set up Nextcloud for you, which will start the server behind port 11000. At this point, you can just point your reverse proxy at 11000 and you’re in business.

Basically, don’t try to set up Nextcloud AIO unless you (a) are on the local network or (b) have a VPN connection in to that network.

scottini · April 27, 2024, 2:02pm

Could you please explain further, with your configuration of 2 containers one for Caddy and the other for AIO, what do you mean by “access over LAN” ?

LorenDB · April 28, 2024, 8:02pm

Basically, the AIO configuration page only accepts traffic from your local network; you can’t access it from a remote computer (unless you use a VPN to connect to your local network). That means that you’ll need to access the AIO container from something like 192.168.1.x:8080, while the main Nextcloud instance can be proxied to the internet with Caddy or something else at port 11000.