Bad Gateway (502) Traefik (Proxy) > Nextcloud-AIO > Portainer Setup

Hello,

I want to install at my local home setup the Nextcloud HUB - All In One.

I’m using docker containers over Portainer (Stack / Docker Compose) but actually I’m getting as repsonse a Bad Gateway - 502.

Here you can find some other details to my problem:
[Nextcloud AIO does not support label configuration - Traefik v2 - Traefik Labs Community Forum](https://traefiklabs Forum / my post)

Here a short summary:

I have created a portainer stack / docker compose using the provided template.

...
      - APACHE_PORT=11000 # Is needed when running behind a web server or reverse proxy (like Apache, Nginx, Cloudflare Tunnel and else). See https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md
      - APACHE_IP_BINDING=0.0.0.0 # Should be set when running behind a web server or reverse proxy (like Apache, Nginx, Cloudflare Tunnel and else) that is running on the same host. See https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md
...

Here I’m unsure with the APACHE_IP_BINDING, which address should be set there?!

While the Traefik label config does not work with the AIO Variant, I have switched from the label configuration to the traefik dynamic file configuration.

http:
  routers:
    nextcloud:
      service: nextcloud-service
      entryPoints:
        - websecure
      middlewares:
        - nextcloud-chain
      rule: "Host(`nextcloud.mydomain.de`)"
      tls:
        options: default
    
    ...

  middlewares:
    nextcloud-secure-headers:
      headers:
        hostsProxyHeaders:
          - "X-Forwarded-Host"
        referrerPolicy: "same-origin"

    https-redirect:
      redirectscheme:
        scheme: https

    nextcloud-chain:
      chain:
        middlewares:
          # - ... (e.g. rate limiting middleware)
          - https-redirect
          - nextcloud-secure-headers

  services:
    nextcloud-service:
      loadBalancer:
        servers:
          - url: "http://nextcloud:11000"

...

tls:
  certificates:
    - certFile: /etc/certs/live/mydomain.de/fullchain.pem
      keyFile: /etc/certs/live/mydomain.de/privkey.pem

The external docker networks should work fine, while I can ping from the traefik container the nextcloud-aio container. But a wget will also fail with a “Connection refused”. I hope is it okay, that I don’t duplicate the whole content. Everything is posted here.

Actually I have no other idea to find a solution here to get it running. Hopefully someone can help me here.

Thank you!

Pointing at the nextcloud container is not going to work. Please follow all-in-one/reverse-proxy.md at main · nextcloud/all-in-one · GitHub

Thank you for your response.

Here is list, how to debug things:

  1. Make sure to exactly follow the whole reverse proxy documentation step-for-step from top to bottom!

done

  1. Make sure that you used the docker run command that is described in this reverse proxy documentation. Hint: make sure that you have set the APACHE_PORT via e.g. --env APACHE_PORT=11000 during the docker run command!

here I have used the template from nextcloud-github

and for traefik2.

  1. Make sure to set the APACHE_IP_BINDING variable correctly. If in doubt, set it to --env APACHE_IP_BINDING=0.0.0.0

here I have set the 0.0.0.0. May be here is the problem? Which IP address has to set here?

  1. Make sure that all ports to which your reverse proxy is pointing match the chosen APACHE_PORT.

done. I have set it like the doc to 11000. The dynamic traefik config points to 11000.

  1. Make sure that the reverse proxy is running on the host OS or if running in a container, connected to the host network. If that is not possible (e.g. on Windows or if the reverse proxy is running on a different host), substitute localhost or 127.0.0.1 in the default configurations by the private ip-address of the host that is running the docker daemon. If you are not sure how to retrieve that, you can run: ip a | grep "scope global" | head -1 | awk '{print $2}' | sed 's|/.*||'. If the command returns a public ip-address, use ip a | grep "scope global" | grep docker0 | awk '{print $2}' | sed 's|/.*||' instead (the commands only work on Linux)

Traefik and Nextcloud-aio runs on the same system in two containers controlled via portainer. For what is the command

ip a | grep “scope global” | head -1 | awk ‘{print $2}’ | sed ‘s|/.*||’

good for? Where should the command be executed?

  1. Make sure that the mastercontainer is able to spawn other containers. You can do so by checking that the mastercontainer indeed has access to the Docker socket which might not be positioned in one of the suggested directories like /var/run/docker.sock but in a different directory, based on your OS and the way how you installed Docker. The mastercontainer logs should help figuring this out. You can have a look at them by running sudo docker logs nextcloud-aio-mastercontainer after the container is started the first time.

Here the log output:

docker logs nextcloud-aio-mastercontainer
Trying to fix docker.sock permissions internally...
Adding internal www-data to group root
... ... ...
Initial startup of Nextcloud All-in-One complete!
You should be able to open the Nextcloud AIO Interface now on port 8080 of this server!
E.g. https://internal.ip.of.this.server:8080

If your server has port 80 and 8443 open and you point a domain to your server, you can get a valid certificate automatically by opening the Nextcloud AIO Interface via:
https://your-domain-that-points-to-this-server.tld:8443
[Sun Jul 23 11:48:43.106348 2023] [mpm_event:notice] [pid 102:tid 139741789633352] AH00489: Apache/2.4.57 (Unix) OpenSSL/3.1.1 configured -- resuming normal operations
[Sun Jul 23 11:48:43.106405 2023] [core:notice] [pid 102:tid 139741789633352] AH00094: Command line: 'httpd -D FOREGROUND'
[23-Jul-2023 11:48:43] NOTICE: fpm is running, pid 106
[23-Jul-2023 11:48:43] NOTICE: ready to handle connections
{"level":"info","ts":1690112923.1383607,"msg":"using provided configuration","config_file":"/Caddyfile","config_adapter":""}
  1. Check if after the mastercontainer was started, the reverse proxy if running inside a container, can reach the provided apache port. You can test this by running nc -z localhost 11000; echo $? from inside the reverse proxy container. If the output is 0, everything works. Alternatively you can of course use instead of localhost the ip-address of the host here for the test.

docker exec traefik nc -z localhost 11000; echo $?
1

docker exec traefik nc -z nextcloud-aio-mastercontainer 11000; echo $?
1

The result is NOT 0. It is 1. What does it mean, if 1 will be returned?

Can you use - url: "http://ip.address.of.the.server.that.runs.docker:11000" instead of pointing at nextcloud? The commands mentioned are meant for retreiving that ip-address that should be used here.

I have changed the url to the IP of the host, where docker is running:

services:
nextcloud-service:
loadBalancer:
servers:
- url: “http://192.168.2.28:11000

but it ends by the same result. 502 - Bad Gateway.

Didnyou already open the AIO interface and entered your domain?

Sorry, I’m not sure what you mean. What is the AIO Interface for you? Where should I entered my domain?

Beside nextcloud I have some other container running (openhab, nodered, vaultwarden, …) All that containers are working as expected behind the traefik proxy. So I have initially tried to configure the nextcloud like the others. It is hard to find out, what the problem is.

See all-in-one/reverse-proxy.md at main · nextcloud/all-in-one · GitHub

1 Like

Ah okay, sorry!

I’ve adapted my compose file (nextcloud) so the port 8080 can be accessed from outside and could navigate to jarvis.mydomain.de:8080 to reach the setup page. After a requested update of the master container two new containers were deployed (…-domaincheck + …-watchtower). The next step was to set the domain. Here I have entered nextcloud.mydomain.de and submitted.

Then I have called https://nextcloud.mydomain.de and it ends after 30seconds with a Gateway TImeout 504.

Is it correct to set nextcloud.mydomain.de or is the right address jarvis.mydomain.de?

I suppose this one based on your treafik configuration.

I suppose this one based on your treafik configuration.

Yes, this is correct.

I have deleted all created nextcloud containers, also the volume and called the setup after redeploying again.

I have entered the nextcloud.mydomain.de as domain and then a page was shown to select the optional addons or if I want to install Nextcloud 27 or 26 (the first time that view was not shown).

Currently I hope, that after the installation it will work and can be reached by nextcloud.mydomain.de.

Traefik points at the nextcloud-service to my host jarvis.mydomain.de:11000 (any other location will end with the Bad Gateway 502)

Inside my compose config the two ENV Variables are set to:

  • APACHE_IP_BINDING=0.0.0.0
  • APACHE_PORT=11000

While I’m writing that post a bunch of containers are created (I can monitor this with Portainer). After 5 to 10 minutes the Nextcloud AIO Web UI shows every container is running and gives me an initial password for admin.

The Button “Open your Nextcloud” points to https://nextcloud.mydomain.de/ but ends with a 504 Gateway Timeout, again. First I thought, it would be fine, while I have seen, that a nextcloud-aio-apache container was created. But it isn’t so.

Here the found log to the apache container:

Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
Waiting for Nextcloud to start…
[Sun Jul 23 23:25:30.881408 2023] [mpm_event:notice] [pid 245:tid 140616398375752] AH00489: Apache/2.4.57 (Unix) configured – resuming normal operations
[Sun Jul 23 23:25:30.881519 2023] [core:notice] [pid 245:tid 140616398375752] AH00094: Command line: ‘/usr/local/apache2/bin/httpd -D FOREGROUND’
INF ts=1690147530.887172 msg=using provided configuration config_file=/tmp/Caddyfile config_adapter=

and the state of the container has switched to “unhealthy” (this will be shown, if I switch to the container tab in Portainer).

I’m at a loss. Any help is welcome.

Maybe this video helps you to figure things out? Installing Traefik as a Reverse Proxy for Nextcloud - YouTube

Thank you for the youtube link. I have compared the traefik settings with my settings. The main parts are the same. The little differences I have merged to my config.

Actually I have the same result: Bad Gateway 504. In Portainer I can see, that all - except the nextcloud-aio-apache container are running with status “healthy”. The apache container is running, but in state unhealthy!

Here some input of that container:

8dd8b4e5e3663dcd7a06bcb4c13547bf7ef4883290d39ea937d825a4239b9429
AppArmorProfile
Args [ /usr/bin/supervisord, -c, /supervisord.conf ]
Config { AttachStderr: false, AttachStdin: false, AttachStdout: false, Cmd: /usr/bin/supervisord,-c,/supervisord.conf, Domainname: , Entrypoint: /start.sh, Env: NC_DOMAIN=nextcloud.mydomain.de,NEXTCLOUD_HOST=nextcloud-aio-nextcloud,COLLABORA_HOST=nextcloud-aio-collabora,TALK_HOST=nextcloud-aio-talk,APACHE_PORT=11000,ONLYOFFICE_HOST=nextcloud-aio-onlyoffice,TZ=Europe/Berlin,APACHE_MAX_SIZE=10737418240,APACHE_MAX_TIME=3600,NOTIFY_PUSH_HOST=nextcloud-aio-notify-push,PATH=/usr/local/apache2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,HTTPD_PREFIX=/usr/local/apache2,HTTPD_VERSION=2.4.57,HTTPD_SHA256=dbccb84aee95e095edfbb81e5eb926ccd24e6ada55dcd83caecb262e5cf94d2a,HTTPD_PATCHES=rewrite-windows-testchar-h.patch 1d5620574fa03b483262dc5b9a66a6906553389952ab5d3070a02f887cc20193, ExposedPorts: [object Object], Healthcheck: [object Object], Hostname: 8dd8b4e5e366, Image: nextcloud/aio-apache:latest, Labels: [object Object], OnBuild: null, OpenStdin: false, StdinOnce: false, StopSignal: SIGWINCH, Tty: false, User: www-data, Volumes: [object Object], WorkingDir: /usr/local/apache2 }
Created 2023-07-25T19:14:25.275833039Z
Driver overlay2
ExecIDs [ 2180cbce46712729abd75986d0326fc5f71635889911c72817131563bd21535b ]
GraphDriver { Data: [object Object], Name: overlay2 }
HostConfig { AutoRemove: false, Binds: nextcloud_aio_nextcloud:/var/www/html:ro,nextcloud_aio_apache:/mnt/data:rw, BlkioDeviceReadBps: null, BlkioDeviceReadIOps: null, BlkioDeviceWriteBps: null, BlkioDeviceWriteIOps: null, BlkioWeight: 0, BlkioWeightDevice: null, CapAdd: null, CapDrop: null, Cgroup: , CgroupParent: , CgroupnsMode: host, ConsoleSize: 0,0, ContainerIDFile: , CpuCount: 0, CpuPercent: 0, CpuPeriod: 0, CpuQuota: 0, CpuRealtimePeriod: 0, CpuRealtimeRuntime: 0, CpuShares: 0, CpusetCpus: , CpusetMems: , DeviceCgroupRules: null, DeviceRequests: null, Devices: null, Dns: null, DnsOptions: null, DnsSearch: null, ExtraHosts: null, GroupAdd: null, IOMaximumBandwidth: 0, IOMaximumIOps: 0, IpcMode: private, Isolation: , KernelMemory: 0, KernelMemoryTCP: 0, Links: null, LogConfig: [object Object], MaskedPaths: /proc/asound,/proc/acpi,/proc/kcore,/proc/keys,/proc/latency_stats,/proc/timer_list,/proc/timer_stats,/proc/sched_debug,/proc/scsi,/sys/firmware, Memory: 0, MemoryReservation: 0, MemorySwap: 0, MemorySwappiness: null, NanoCpus: 0, NetworkMode: nextcloud-aio, OomKillDisable: false, OomScoreAdj: 0, PidMode: , PidsLimit: null, PortBindings: [object Object], Privileged: false, PublishAllPorts: false, ReadonlyPaths: /proc/bus,/proc/fs,/proc/irq,/proc/sys,/proc/sysrq-trigger, ReadonlyRootfs: true, RestartPolicy: [object Object], Runtime: runc, SecurityOpt: null, ShmSize: 67108864, Tmpfs: [object Object], UTSMode: , Ulimits: [object Object], UsernsMode: , VolumeDriver: , VolumesFrom: null }
HostnamePath /share/CACHEDEV1_DATA/Container/container-station-data/lib/docker/containers/8dd8b4e5e3663dcd7a06bcb4c13547bf7ef4883290d39ea937d825a4239b9429/hostname
HostsPath /share/CACHEDEV1_DATA/Container/container-station-data/lib/docker/containers/8dd8b4e5e3663dcd7a06bcb4c13547bf7ef4883290d39ea937d825a4239b9429/hosts
Id 8dd8b4e5e3663dcd7a06bcb4c13547bf7ef4883290d39ea937d825a4239b9429
Image sha256:e0c0bddff69b4e3a0491aeed6d823457a28eddeb987a8ac6c653488aa7d5f022
LogPath /share/CACHEDEV1_DATA/Container/container-station-data/lib/docker/containers/8dd8b4e5e3663dcd7a06bcb4c13547bf7ef4883290d39ea937d825a4239b9429/8dd8b4e5e3663dcd7a06bcb4c13547bf7ef4883290d39ea937d825a4239b9429-json.log
MountLabel
Mounts [ [object Object], [object Object] ]
Name /nextcloud-aio-apache
NetworkSettings { Bridge: , EndpointID: , Gateway: , GlobalIPv6Address: , GlobalIPv6PrefixLen: 0, HairpinMode: false, IPAddress: , IPPrefixLen: 0, IPv6Gateway: , LinkLocalIPv6Address: , LinkLocalIPv6PrefixLen: 0, MacAddress: , Networks: [object Object], Ports: [object Object], SandboxID: 536502ae342734013a2373ffbb286fc5bf37267f542c91c39a375a639dff5a59, SandboxKey: /var/run/docker/netns/536502ae3427, SecondaryIPAddresses: null, SecondaryIPv6Addresses: null }
Path /start.sh
Platform linux
ProcessLabel
ResolvConfPath /share/CACHEDEV1_DATA/Container/container-station-data/lib/docker/containers/8dd8b4e5e3663dcd7a06bcb4c13547bf7ef4883290d39ea937d825a4239b9429/resolv.conf
RestartCount 0
State
Dead false
Error
ExitCode 0
FinishedAt 0001-01-01T00:00:00Z
Health
FailingStreak 25
Log
0 { End: 2023-07-25T21:36:03.172116145+02:00, ExitCode: -1, Output: Health check exceeded timeout (30s), Start: 2023-07-25T21:35:33.17096096+02:00 }
1 { End: 2023-07-25T21:37:03.304800202+02:00, ExitCode: -1, Output: Health check exceeded timeout (30s), Start: 2023-07-25T21:36:33.30398436+02:00 }
2 { End: 2023-07-25T21:38:03.393592204+02:00, ExitCode: -1, Output: Health check exceeded timeout (30s), Start: 2023-07-25T21:37:33.392678884+02:00 }
3 { End: 2023-07-25T21:39:03.481471599+02:00, ExitCode: -1, Output: Health check exceeded timeout (30s), Start: 2023-07-25T21:38:33.480703497+02:00 }
4
End 2023-07-25T21:40:03.583147658+02:00
ExitCode -1
Output Health check exceeded timeout (30s)
Start 2023-07-25T21:39:33.582079981+02:00
Status unhealthy
OOMKilled false
Paused false
Pid 29827
Restarting false
Running true
StartedAt 2023-07-25T19:14:30.546759795Z
Status running

Inside the container I have tried to find some logs with further information, what the problem is. But under /var/log/… I didn’t find something.

Where can I find some more details? The main reason should be the situation, that the apache-container does not start successfully and ends in that unhealthy state.

It’s a pitty. The reason to choose the AIO variant is to have an easy update procedure and get the best predefined settings for the database and so on, instead of doing this manually.

I mean AIO can handle the tls proxying itself which could be used instead of adding a reverse proxy manually which would simplify the installation by a lot. Also choosing caddy as reverse proxy instead of treafik would have been much easier imho.

I don’t use nextlcoud only in my home-system. There are several services/container, that I’m using. And therefore I have installed traefik, that handles the only usage of https (ssl). Here I’m using a certbot, so I can create over that via dns challenge a valid let’s encrypt cert over my own domain without open a port outside, that will be provided for my traefik proxy.

So I won’t give up that working setup.

Actually I decide to skip the AIO variant und use the “classic” way to provide Nextcloud HUB directly. Can I simply add features like:

  • Backup solution (based on BorgBackup)
  • Imaginary (for previews of heic, heif, illustrator, pdf, svg, tiff and webp) (very interessting)
  • Fulltextsearch

?

I have seen, that AIO is using a PostgresDB. What is the best configuration to get the best performance or are the default settings sufficient? If some documention exists for it, it would be nice, if someone can share it.

I stumped on this post while I was having similar issue. first was 502 Bad Gateway and then I was stuck on 504 Gateway. This post helped me analyse/debug my own setup. My Traefik was running on ‘Socket Proxy’ Network and Nexcloud had its own network (network-aio) , this was reason for 502! I added the netowrk-aio to traefik (not sure if this is good idea) but the 502 error went away and I was able to ping nextcloud containers from traefik. Next was 504 which was troubling for some time now. My Load Balancer config was pointing to http://192.168.x.x:11000 and when I tried one of the command mentioned above ‘docker exec traefik nc -z 192.168.x.x 11000; echo $?’ it returned 1, I tried with the apache container and it returned 0 ‘docker exec traefik nc -z nextcloud-aio-apache 11000; echo $?’ . I updated this in my load balancer and refreshed the login page of nextcloud and I can now see the login Page. Thank you all!

never using this god awful proxy garbage ever again in my life

You mean Traefik?