Fresh NC-AIO Install: Default Deploy Daemon is not accessible

Hello dear Nextcloud community,
I just installed a fresh NC AIO Instance and was able to set it up behind a nginx reverse proxy. I can access everything and got my letsencrypt working after a bit of trial and error. Sadly when the mastercontainer starts the nextcloud/aio-docker-socket-proxy container the port 2375 is not opened.

alpine-docker-host-01:~# docker ps
CONTAINER ID   IMAGE                                      COMMAND                  CREATED          STATUS                      PORTS                                                                                  NAMES
8a5ca2f81619   nextcloud/aio-apache:latest                "/start.sh /usr/bin/…"   15 minutes ago   Up 15 minutes (unhealthy)   80/tcp, 0.0.0.0:11000->11000/tcp                                                       nextcloud-aio-apache
ffda9c2faa63   nextcloud/aio-notify-push:latest           "/start.sh"              15 minutes ago   Up 15 minutes (healthy)                                                                                            nextcloud-aio-notify-push
4da44b418861   nextcloud/aio-nextcloud:latest             "/start.sh /usr/bin/…"   15 minutes ago   Up 15 minutes (healthy)     9000/tcp                                                                               nextcloud-aio-nextcloud
222ee483eb36   nextcloud/aio-docker-socket-proxy:latest   "/start.sh"              15 minutes ago   Up 15 minutes (healthy)                                                                                            nextcloud-aio-docker-socket-proxy
f61a5e6f46f4   nextcloud/aio-imaginary:latest             "imaginary -return-s…"   15 minutes ago   Up 15 minutes (healthy)                                                                                            nextcloud-aio-imaginary
d7d14998457a   nextcloud/aio-fulltextsearch:latest        "/bin/tini -- /usr/l…"   15 minutes ago   Up 15 minutes (healthy)     9200/tcp, 9300/tcp                                                                     nextcloud-aio-fulltextsearch
0d4e4adfc4eb   nextcloud/aio-redis:latest                 "/start.sh"              15 minutes ago   Up 15 minutes (healthy)     6379/tcp                                                                               nextcloud-aio-redis
94e6aca3c4bf   nextcloud/aio-postgresql:latest            "/start.sh"              15 minutes ago   Up 15 minutes (healthy)     5432/tcp                                                                               nextcloud-aio-database
dd1e78149ecf   nextcloud/all-in-one:latest                "/start.sh"              11 hours ago     Up 11 hours (healthy)       80/tcp, 8443/tcp, 9000/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp                  nextcloud-aio-mastercontainer
alpine-docker-host-01:~# docker logs nextcloud-aio-docker-socket-proxy
Waiting for Nextcloud to start...
    # ACL to deny if there are any binds
    acl binds_present req.body -m reg -i "\"HostConfig\"\s*:.*\"Binds\"\s*:"
    # ACL to restrict the type of Mounts to volume
    acl type_not_volume req.body -m reg -i "\"Mounts\":\s*\[[^\]]*(\"Type\":\s*\"(?!volume\b)\w+\"[^\]]*)+\]"
    http-request deny if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/create } nc_app_container_name !one_mount_volume binds_present type_not_volume METH_POST

    # ACL to restrict container creation, that it has HostConfig.Privileged not set
    acl no_privileged_flag req.body -m reg -i "\"HostConfig\":\s?{[^}]*\"Privileged\""
    # ACL to allow mount volume with strict pattern for name: nc_app_[a-zA-Z0-9_.-]+_data
    acl nc_app_volume_data_only req.body -m reg -i "\"Mounts\":\s?\[\s?{[^}]*\"Source\":\s?\"nc_app_[a-zA-Z0-9_.-]+_data\""
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/create } nc_app_container_name !no_privileged_flag nc_app_volume_data_only METH_POST
    # end of container create

    # volume create: POST volumes/create
    # restrict name
    acl nc_app_volume_data req.body -m reg -i "\"Name\":\s?\"nc_app_[a-zA-Z0-9_.-]+_data\""
    # do not allow to use "device" word e.g., "--opt device=:/path/to/dir"
    acl volume_no_device req.body -m reg -i "\"device\""
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/volumes/create } nc_app_volume_data !volume_no_device METH_POST
    # volume rm: DELETE volumes/%s
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/volumes/nc_app_[a-zA-Z0-9_.-]+_data } METH_DELETE
    # image pull: POST images/create?fromImage=%s
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/images/create } METH_POST
    http-request deny
    default_backend dockerbackend

backend dockerbackend
    mode http
    server dockersocket /var/run/docker.sock'
+ dig nextcloud-aio-nextcloud AAAA +short +search
+ grep '^[0-9a-f:]\+$'
+ sort
+ head -n1
+ IPv6_ADDRESS_NC=
+ '[' -n  ]
+ sed 's# || { src NC_IPV6_PLACEHOLDER }##g' /tmp/haproxy.cfg
+ HAPROXYFILE='# Inspiration: https://github.com/Tecnativa/docker-socket-proxy/blob/master/haproxy.cfg

global
    maxconn 10

defaults
    timeout connect 10s
    timeout client 10s
    timeout server 10s

frontend http
    mode http
    bind :::2375 v4v6
    http-request deny unless { src 127.0.0.1 } || { src ::1 } || { src 172.29.0.8 }
    # docker system _ping
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/_ping } METH_GET
    # container inspect: GET containers/%s/json
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/json } METH_GET
    # container start/stop: POST containers/%s/start containers/%s/stop
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/((start)|(stop)) } METH_POST
    # container rm: DELETE containers/%s
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+ } METH_DELETE


    # container create: POST containers/create?name=%s
    # ACL to restrict container name to nc_app_[a-zA-Z0-9_.-]+
    acl nc_app_container_name url_param(name) -m reg -i "^nc_app_[a-zA-Z0-9_.-]+"

    # ACL to restrict the number of Mounts to 1
    acl one_mount_volume req.body -m reg -i "\"Mounts\"\s*:\s*\[\s*(?:(?!\"Mounts\"\s*:\s*\[)[^}]*)}[^}]*\]"
    # ACL to deny if there are any binds
    acl binds_present req.body -m reg -i "\"HostConfig\"\s*:.*\"Binds\"\s*:"
    # ACL to restrict the type of Mounts to volume
    acl type_not_volume req.body -m reg -i "\"Mounts\":\s*\[[^\]]*(\"Type\":\s*\"(?!volume\b)\w+\"[^\]]*)+\]"
    http-request deny if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/create } nc_app_container_name !one_mount_volume binds_present type_not_volume METH_POST

    # ACL to restrict container creation, that it has HostConfig.Privileged not set
    acl no_privileged_flag req.body -m reg -i "\"HostConfig\":\s?{[^}]*\"Privileged\""
    # ACL to allow mount volume with strict pattern for name: nc_app_[a-zA-Z0-9_.-]+_data
    acl nc_app_volume_data_only req.body -m reg -i "\"Mounts\":\s?\[\s?{[^}]*\"Source\":\s?\"nc_app_[a-zA-Z0-9_.-]+_data\""
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/create } nc_app_container_name !no_privileged_flag nc_app_volume_data_only METH_POST
    # end of container create

    # volume create: POST volumes/create
    # restrict name
    acl nc_app_volume_data req.body -m reg -i "\"Name\":\s?\"nc_app_[a-zA-Z0-9_.-]+_data\""
    # do not allow to use "device" word e.g., "--opt device=:/path/to/dir"
    acl volume_no_device req.body -m reg -i "\"device\""
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/volumes/create } nc_app_volume_data !volume_no_device METH_POST
    # volume rm: DELETE volumes/%s
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/volumes/nc_app_[a-zA-Z0-9_.-]+_data } METH_DELETE
    # image pull: POST images/create?fromImage=%s
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/images/create } METH_POST
    http-request deny
    default_backend dockerbackend

backend dockerbackend
    mode http
    server dockersocket /var/run/docker.sock'
+ echo '# Inspiration: https://github.com/Tecnativa/docker-socket-proxy/blob/master/haproxy.cfg

global
    maxconn 10

defaults
    timeout connect 10s
    timeout client 10s
    timeout server 10s

frontend http
    mode http
    bind :::2375 v4v6
    http-request deny unless { src 127.0.0.1 } || { src ::1 } || { src 172.29.0.8 }
    # docker system _ping
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/_ping } METH_GET
    # container inspect: GET containers/%s/json
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/json } METH_GET
    # container start/stop: POST containers/%s/start containers/%s/stop
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+/((start)|(stop)) } METH_POST
    # container rm: DELETE containers/%s
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/nc_app_[a-zA-Z0-9_.-]+ } METH_DELETE


    # container create: POST containers/create?name=%s
    # ACL to restrict container name to nc_app_[a-zA-Z0-9_.-]+
    acl nc_app_container_name url_param(name) -m reg -i "^nc_app_[a-zA-Z0-9_.-]+"

    # ACL to restrict the number of Mounts to 1
    acl one_mount_volume req.body -m reg -i "\"Mounts\"\s*:\s*\[\s*(?:(?!\"Mounts\"\s*:\s*\[)[^}]*)}[^}]*\]"
    # ACL to deny if there are any binds
    acl binds_present req.body -m reg -i "\"HostConfig\"\s*:.*\"Binds\"\s*:"
    # ACL to restrict the type of Mounts to volume
    acl type_not_volume req.body -m reg -i "\"Mounts\":\s*\[[^\]]*(\"Type\":\s*\"(?!volume\b)\w+\"[^\]]*)+\]"
    http-request deny if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/create } nc_app_container_name !one_mount_volume binds_present type_not_volume METH_POST

    # ACL to restrict container creation, that it has HostConfig.Privileged not set
    acl no_privileged_flag req.body -m reg -i "\"HostConfig\":\s?{[^}]*\"Privileged\""
    # ACL to allow mount volume with strict pattern for name: nc_app_[a-zA-Z0-9_.-]+_data
    acl nc_app_volume_data_only req.body -m reg -i "\"Mounts\":\s?\[\s?{[^}]*\"Source\":\s?\"nc_app_[a-zA-Z0-9_.-]+_data\""
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/containers/create } nc_app_container_name !no_privileged_flag nc_app_volume_data_only METH_POST
    # end of container create

    # volume create: POST volumes/create
    # restrict name
    acl nc_app_volume_data req.body -m reg -i "\"Name\":\s?\"nc_app_[a-zA-Z0-9_.-]+_data\""
    # do not allow to use "device" word e.g., "--opt device=:/path/to/dir"
    acl volume_no_device req.body -m reg -i "\"device\""
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/volumes/create } nc_app_volume_data !volume_no_device METH_POST
    # volume rm: DELETE volumes/%s
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/volumes/nc_app_[a-zA-Z0-9_.-]+_data } METH_DELETE
    # image pull: POST images/create?fromImage=%s
    http-request allow if { path,url_dec -m reg -i ^(/v[\d\.]+)?/images/create } METH_POST
    http-request deny
    default_backend dockerbackend

backend dockerbackend
    mode http
    server dockersocket /var/run/docker.sock'
+ set +x

The app API admin backend section obviously is not happy about not reaching the container. It makes sense that it cant be reached since “docker ps” shows that the port ist not opened. What I dont get is why the port isnt open nor how I could fix it.
Usually I would check the docker compose yaml or the run command but everything is started from the AIO Mastercontainer so I dont have that.
Also the log at least to me looks like it should be opening the port.
The other ports open as expected.

My Mastercontainer compose: TLS Certs work, the check just acme check just didnt work for some reason and I skipped it.

version: "3.3"
# before changing anything check for options, pitfalls and best practices at https://github.com/nextcloud/all-in-one/blob/main/compose.yaml
services:
  nextcloud-aio-mastercontainer:
    image: nextcloud/all-in-one:latest
    init: true
    restart: always
    container_name: nextcloud-aio-mastercontainer
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config
      - /var/run/docker.sock:/var/run/docker.sock:ro
    ports:
      - 8080:8080
    environment:
      - APACHE_PORT=11000
      - APACHE_IP_BINDING=0.0.0.0
      - SKIP_DOMAIN_VALIDATION=true
volumes:
  nextcloud_aio_mastercontainer:
    name: nextcloud_aio_mastercontainer
networks: {}

As seen in the docker ps above the aio-apache container also has a status of unhealthy but I dont know why. The webserver serves everthing and this is the only thing in its log:

alpine-docker-host-01:~# docker logs nextcloud-aio-apache
Waiting for Nextcloud to start...
Waiting for Nextcloud to start...
Connection to nextcloud-aio-nextcloud (172.29.0.8) 9000 port [tcp/*] succeeded!
[Sat Feb 17 11:38:24.122854 2024] [mpm_event:notice] [pid 39:tid 139847629208328] AH00489: Apache/2.4.58 (Unix) configured -- resuming normal operations
[Sat Feb 17 11:38:24.123327 2024] [core:notice] [pid 39:tid 139847629208328] AH00094: Command line: '/usr/local/apache2/bin/httpd -D FOREGROUND'
{"level":"info","ts":1708169904.1277683,"msg":"using provided configuration","config_file":"/tmp/Caddyfile","config_adapter":""}

Any advice for how to debug this is appreciated.

Solved:
I needed a dns server so the containers could resolve the domain locally.
Besides the solved link (box below), you might want to check out my last comment in this thread regarding pihole issues.

Cc @bigcat88 any idea?

“mastercontainer starts the nextcloud/aio-docker-socket-proxy container the port 2375 is not” - how this could happen?
Can you double check if port opened or not please.

@Err can you follow https://github.com/nextcloud/all-in-one/discussions/1358?

Still no open port

alpine-docker-host-01:~# docker ps
CONTAINER ID   IMAGE                                      COMMAND                  CREATED        STATUS                   PORTS                                                                                  NAMES
8a5ca2f81619   nextcloud/aio-apache:latest                "/start.sh /usr/bin/…"   2 hours ago    Up 2 hours (unhealthy)   80/tcp, 0.0.0.0:11000->11000/tcp                                                       nextcloud-aio-apache
ffda9c2faa63   nextcloud/aio-notify-push:latest           "/start.sh"              2 hours ago    Up 2 hours (healthy)                                                                                            nextcloud-aio-notify-push
4da44b418861   nextcloud/aio-nextcloud:latest             "/start.sh /usr/bin/…"   2 hours ago    Up 2 hours (healthy)     9000/tcp                                                                               nextcloud-aio-nextcloud
222ee483eb36   nextcloud/aio-docker-socket-proxy:latest   "/start.sh"              2 hours ago    Up 2 hours (healthy)                                                                                            nextcloud-aio-docker-socket-proxy
f61a5e6f46f4   nextcloud/aio-imaginary:latest             "imaginary -return-s…"   2 hours ago    Up 2 hours (healthy)                                                                                            nextcloud-aio-imaginary
d7d14998457a   nextcloud/aio-fulltextsearch:latest        "/bin/tini -- /usr/l…"   2 hours ago    Up 2 hours (healthy)     9200/tcp, 9300/tcp                                                                     nextcloud-aio-fulltextsearch
0d4e4adfc4eb   nextcloud/aio-redis:latest                 "/start.sh"              2 hours ago    Up 2 hours (healthy)     6379/tcp                                                                               nextcloud-aio-redis
94e6aca3c4bf   nextcloud/aio-postgresql:latest            "/start.sh"              2 hours ago    Up 2 hours (healthy)     5432/tcp                                                                               nextcloud-aio-database
dd1e78149ecf   nextcloud/all-in-one:latest                "/start.sh"              13 hours ago   Up 13 hours (healthy)    80/tcp, 8443/tcp, 9000/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp                  nextcloud-aio-mastercontainer

No new log entries in the container (socket-proxy)

Websocket support is enabled on the reverse proxy.

My docker host (alpine linux vm) has

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

in its /etc/hosts file, there is nothing else in there. If I understand correctly I should remove that and restart the mastercontainer. Yes?

Edit: On second thought what can go wrong, I did it and sadly nothing changed.

Does collabora and talk work for you?

I do not need them so I never checked.

Can you check?

Sure, so I installed both. Talk seems to be working without a problem.
But Colabora has some problems.

In the admin section the Nextcloud office settings give me the following error:

Could not establish connection to the Collabora Online server.
Failed to connect to the remote server: cURL error 7: Failed to connect to nextcloud.mydomain.com port 443 after 3 ms: Couldn't connect to server (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://nextcloud.mydomain.com/custom_apps/richdocumentscode/proxy.php?req=/hosting/discovery

aswell as this warning

You have not configured the allow-list for WOPI requests. Without this setting users may download restricted files via WOPI requests to the Nextcloud server. Click here for more info

If I open the url mentioned in the error I get on 443 a response.
Typical white page with like a uncustomized error code page. Aswell as this text:

Socket proxy error

Error: no_glibc

I assume glibc is a library that is missing? How did I manage to get the container into a state where its missing a lib? :thinking:

I see. Can you follow https://github.com/nextcloud/all-in-one/discussions/1358 then?

Output of the curl discovery command:
I obfuscated my ipv6 since in ipv6 its the same address internal and external. I replaced my it with MY_IPV6.

8456dc18a606:/var/www/html# curl -vvv https://$NC_DOMAIN:443/hosting/discovery
exit
* Host $NC_DOMAIN:443 was resolved.
* IPv6: MY_IPV6
* IPv4: (none)
*   Trying [MY_IPV6]:443...
* Immediate connect fail for MY_IPV6: Address not available
* Failed to connect to $NC_DOMAIN port 443 after 2 ms: Couldn't connect to server
* Closing connection
curl: (7) Failed to connect to $NC_DOMAIN port 443 after 2 ms: Couldn't connect to server
exit

I don’t know where I can find the talk and collabora logs. At first I thought it would be containers, but docker ps doesnt show me any collabora or talk container.

Also I am not so sure anymore that Talk actually works as I said. I can click the Talk Icon on the top when logged in and text myself. I did not check anything else.

I thought so.

So the container cannot connect to itself cia the domain.

See GitHub - nextcloud/all-in-one: The official Nextcloud installation method. Provides easy deployment and maintenance with most features included in this one Nextcloud instance.

Okay, so for whatever reason the containers cant resolve my domain.
So the fix is to add a local DNS Server to the mix that can resolve the domain for the containers so they can communicate with each other.

I added a pihole docker to my setup and added the needed record to it.
Now the containers need to use that DNS server. I am not a docker expert so I am confused a bit here. Should I add the server as the default dns route to the docker host? or the container? or both?

You need to edit daemon.json. see Configuring DNS | dockerlabs

1 Like

So at first my aio mastercontainer didnt start after the change it could not resolve nextcloud.com, the pihole showed the requests as resolved and manual requests to the pihole via command line also worked. The etc/resolv.conf inside of the mastercontainer pointed to 127.0.0.11 tho which was refusing connections.

If anyone ever finds this and has the same problem I had to also add the ip of the host to the compose file of my pihole.

See citizentwelve’s answer in

This is probably just a workaround and I assume it has something to do with the docker networks, maybe I should have added the pihole to the same network as the docker containers. Anyway this works and the AIO docker socket proxy is reachable now (it still has no port mapping in the docker ps).

Collabora still complains about:

Failed to connect to the remote server: Could not find urlsrc for application/vnd.openxmlformats-officedocument.wordprocessingml.document in WOPI discovery response

but my inital problem is solved and since I do not need collabora nor talk and just wanted to get the external apps working I am content with this.

Thanks a lot syaimen for your patience and help. It is very much appreciated!