AIO: Apache container is unhealthy and access to Nextcloud fail

Since yesterday 08/22/2024 10am my Nextcloud AIO hosted at Strato is not working any more.

Environment: Nextcloud AIO v9.4.1

The apache container is going in the status unhealthy and no connection to the could is working any more.

My AIO is running behind an Apache as reverse proxy. Until now I never had any problems.

Apache container log

2024-08-23T06:06:47.703245749Z Waiting for Nextcloud to start...
2024-08-23T06:06:52.705972455Z Waiting for Nextcloud to start...
2024-08-23T06:06:57.708674927Z Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!
2024-08-23T06:07:02.047275151Z [Fri Aug 23 08:07:02.046751 2024] [mpm_event:notice] [pid 74:tid 74] AH00489: Apache/2.4.62 (Unix) configured -- resuming normal operations
2024-08-23T06:07:02.047327550Z [Fri Aug 23 08:07:02.046809 2024] [core:notice] [pid 74:tid 74] AH00094: Command line: '/usr/local/apache2/bin/httpd -D FOREGROUND'
2024-08-23T06:07:02.134412801Z {"level":"info","ts":1724393222.1340866,"msg":"using config from file","file":"/tmp/Caddyfile"}
2024-08-23T06:07:02.136904966Z {"level":"info","ts":1724393222.1367307,"msg":"adapted config to JSON","adapter":"caddyfile"}
My reverse proxy configuration

My reverse proxy configuration

<IfModule mod_ssl.c>

<IfDefine IgnoreBlockComment>
<VirtualHost *:80>
        ServerName cloud.waswaasisch.de
        ServerAlias cloud

        RewriteEngine On
        RewriteCond %{HTTPS} off
        RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
        RewriteCond %{SERVER_NAME} =cloud.waswaasisch.de
        RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
</IfDefine>

<VirtualHost *:443>
        ServerAdmin webmaster@waswaasisch.de
        ServerName  cloud.waswaasisch.de
        ServerAlias cloud

        # Reverse proxy
        RewriteEngine On
        ProxyPreserveHost On
        RequestHeader set X-Real-IP %{REMOTE_ADDR}s
        AllowEncodedSlashes NoDecode

#        RewriteCond %{HTTP:Upgrade} websocket [NC]
#        RewriteCond %{HTTP:Connection} upgrade [NC]
#        RewriteRule ^/(.*) "ws://localhost:11000/$1" [P,L]

        RewriteCond %{HTTP:Upgrade} websocket [NC]
        RewriteCond %{HTTP:Connection} upgrade [NC]
        RewriteCond %{THE_REQUEST} "^[a-zA-Z]+ /(.*) HTTP/\d+(\.\d+)?$"
        RewriteRule .? "ws://localhost:11000/%1" [P,L,UnsafeAllow3F]

#       ProxyRequests off

        # Solves slow upload speeds caused by http2
        H2WindowSize 5242880

        ProxyPass / http://localhost:11000/ nocanon
        ProxyPassReverse / http://localhost:11000/

        # Enable h2, h2c and http1.1
        Protocols h2 h2c http/1.1

        Header set X-Frame-Options "SAMEORIGIN"
        ErrorLog ${APACHE_LOG_DIR}/ssl_cloud_error.log
        LogLevel warn
        CustomLog ${APACHE_LOG_DIR}/ssl_cloud_access.log combined

        SSLEngine on
        SSLCertificateFile    /etc/ssl/letsencrypt/signed.crt
        SSLCertificateKeyFile /etc/ssl/letsencrypt/domain.key
        SSLCertificateChainFile /etc/ssl/letsencrypt/fullchain.pem

        SSLProtocol all -SSLv2 -SSLv3 -TLSv1
        SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256
        SSLHonorCipherOrder on
        SSLCompression off


        # Disable HTTP TRACE method.
        TraceEnable off
        <Files ".ht*">
            Require all denied
        </Files>
</VirtualHost>
</IfModule>

I ran

sudo docker exec -it nextcloud-aio-apache bash -x /healthcheck.sh
+ nc -z nextcloud-aio-nextcloud 9000
Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!
+ nc -z 127.0.0.1 8000
Connection to 127.0.0.1 8000 port [tcp/*] succeeded!
+ nc -z 127.0.0.1 11000
Connection to 127.0.0.1 11000 port [tcp/*] succeeded!
+ nc -z cloud.waswaasisch.de 443
+ echo 'Could not reach cloud.waswaasisch.de on port 443.'
Could not reach cloud.waswaasisch.de on port 443.
+ exit 1
Did also inspect of the apache contianer
sudo docker inspect nextcloud-aio-apache
[
    {
        "Id": "d30cc582ef7acd1c1d2627d1844b8db9f06a7b6153d76b7c9270cbfe59883dc4",
        "Created": "2024-08-23T06:05:47.340578428Z",
        "Path": "/start.sh",
        "Args": [
            "/usr/bin/supervisord",
            "-c",
            "/supervisord.conf"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 919371,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2024-08-23T06:05:51.141694995Z",
            "FinishedAt": "0001-01-01T00:00:00Z",
            "Health": {
                "Status": "unhealthy",
                "FailingStreak": 29,
                "Log": [
                    {
                        "Start": "2024-08-23T08:30:47.920249171+02:00",
                        "End": "2024-08-23T08:31:19.967435357+02:00",
                        "ExitCode": -1,
                        "Output": "Health check exceeded timeout (30s): Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 8000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 11000 port [tcp/*] succeeded!\n"
                    },
                    {
                        "Start": "2024-08-23T08:31:49.967949104+02:00",
                        "End": "2024-08-23T08:32:20.013632867+02:00",
                        "ExitCode": -1,
                        "Output": "Health check exceeded timeout (30s): Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 8000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 11000 port [tcp/*] succeeded!\n"
                    },
                    {
                        "Start": "2024-08-23T08:32:50.014679216+02:00",
                        "End": "2024-08-23T08:33:20.058428826+02:00",
                        "ExitCode": -1,
                        "Output": "Health check exceeded timeout (30s): Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 8000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 11000 port [tcp/*] succeeded!\n"
                    },
                    {
                        "Start": "2024-08-23T08:33:50.059196864+02:00",
                        "End": "2024-08-23T08:34:21.493482902+02:00",
                        "ExitCode": -1,
                        "Output": "Health check exceeded timeout (30s): Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 8000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 11000 port [tcp/*] succeeded!\n"
                    },
                    {
                        "Start": "2024-08-23T08:34:51.494489119+02:00",
                        "End": "2024-08-23T08:35:21.541909171+02:00",
                        "ExitCode": -1,
                        "Output": "Health check exceeded timeout (30s): Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 8000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 11000 port [tcp/*] succeeded!\n"
                    }
                ]
            }
        },
        "Image": "sha256:dc12e9895b977d6b49405810c2db2aca5a5d6ae00c0eac96d55462be7a414fd8",
        "ResolvConfPath": "/var/lib/docker/containers/d30cc582ef7acd1c1d2627d1844b8db9f06a7b6153d76b7c9270cbfe59883dc4/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/d30cc582ef7acd1c1d2627d1844b8db9f06a7b6153d76b7c9270cbfe59883dc4/hostname",
        "HostsPath": "/var/lib/docker/containers/d30cc582ef7acd1c1d2627d1844b8db9f06a7b6153d76b7c9270cbfe59883dc4/hosts",
        "LogPath": "/var/lib/docker/containers/d30cc582ef7acd1c1d2627d1844b8db9f06a7b6153d76b7c9270cbfe59883dc4/d30cc582ef7acd1c1d2627d1844b8db9f06a7b6153d76b7c9270cbfe59883dc4-json.log",
        "Name": "/nextcloud-aio-apache",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": [
            "723c0c73e45e2d7e615ae9b9bbc2985d35c83f064e49b7b6b467cbe3eb46ced2"
        ],
        "HostConfig": {
            "Binds": [
                "nextcloud_aio_nextcloud:/var/www/html:ro",
                "nextcloud_aio_apache:/mnt/data:rw"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "nextcloud-aio",
            "PortBindings": {
                "11000/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "11000"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "unless-stopped",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "ConsoleSize": [
                0,
                0
            ],
            "CapAdd": null,
            "CapDrop": [
                "NET_RAW"
            ],
            "CgroupnsMode": "host",
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": true,
            "SecurityOpt": [
                "label:disable"
            ],
            "Tmpfs": {
                "/home/www-data": "",
                "/tmp": "",
                "/usr/local/apache2/logs": "",
                "/var/log/supervisord": "",
                "/var/run/supervisord": ""
            },
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware",
                "/sys/devices/virtual/powercap"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ],
            "Init": true
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/8a536119ccda38a801f04f0cb6c0b5d114bcbbd751fc16d2c700e8f67c0cc397-init/diff:/var/lib/docker/overlay2/fbe59fb530d1f7f87be3b05f8401d77715f9f2f2d7681d89636893c93db63cf2/diff:/var/lib/docker/overlay2/839f3f55c36ea33e214c6f095cfe2d9ca3cdc1a10377dc557af30f12f57f47a4/diff:/var/lib/docker/overlay2/051cdd5d6858cfd847c60306a1d2dd3db4286e0609096fabbba412eeb147bc26/diff:/var/lib/docker/overlay2/15e14032103c98f5644316c17ad743c3cb16afd84d28f3c8adb8ba4f34505f79/diff:/var/lib/docker/overlay2/2b8b5d2051998de7d597476a59ed859aa1e0bdcfb6247836fefd0fdac4a18f07/diff:/var/lib/docker/overlay2/bb7da6b42923fffc62de3d30ba76be42b185cadfa41634c5db4930b25135de4f/diff:/var/lib/docker/overlay2/8f87f68fab466dcd970aabb3577eac3a03a46636801fab0948dbd3e540a113b4/diff:/var/lib/docker/overlay2/09b4f994a22c1d4f872bfd5b3566bb517dd696fe7e1e0c2238b2cc5741f9f132/diff:/var/lib/docker/overlay2/133b1a0f2f9f0a7d2ac25d0f6337a7f2709d33498e8e25cb87c1991b9ed43e8a/diff:/var/lib/docker/overlay2/41c30c0303be7520433f38a623a28fd9fd09f2efab59acb94dd0af5055ec21c4/diff:/var/lib/docker/overlay2/e11fca2d18e0ec14f307ae6eb7469d71c5be3deedc5788c0df5d663885e329bf/diff:/var/lib/docker/overlay2/6ec0995deb211357978dcc93a9f7e93c18efd3328fdeca6a8159e4415619380d/diff:/var/lib/docker/overlay2/b19e461b849439af405dd762bd45633006eadabe8229f0187e1a787d4931fb98/diff:/var/lib/docker/overlay2/b970399fee2388b7630c89e11ae772b01f278a1ee1ba8964ceef2cacba54fe95/diff",
                "MergedDir": "/var/lib/docker/overlay2/8a536119ccda38a801f04f0cb6c0b5d114bcbbd751fc16d2c700e8f67c0cc397/merged",
                "UpperDir": "/var/lib/docker/overlay2/8a536119ccda38a801f04f0cb6c0b5d114bcbbd751fc16d2c700e8f67c0cc397/diff",
                "WorkDir": "/var/lib/docker/overlay2/8a536119ccda38a801f04f0cb6c0b5d114bcbbd751fc16d2c700e8f67c0cc397/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "volume",
                "Name": "nextcloud_aio_nextcloud",
                "Source": "/var/lib/docker/volumes/nextcloud_aio_nextcloud/_data",
                "Destination": "/var/www/html",
                "Driver": "local",
                "Mode": "ro",
                "RW": false,
                "Propagation": ""
            },
            {
                "Type": "volume",
                "Name": "nextcloud_aio_apache",
                "Source": "/var/lib/docker/volumes/nextcloud_aio_apache/_data",
                "Destination": "/mnt/data",
                "Driver": "local",
                "Mode": "rw",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "d30cc582ef7a",
            "Domainname": "",
            "User": "www-data",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "11000/tcp": {},
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "NC_DOMAIN=cloud.waswaasisch.de",
                "NEXTCLOUD_HOST=nextcloud-aio-nextcloud",
                "APACHE_HOST=nextcloud-aio-apache",
                "COLLABORA_HOST=nextcloud-aio-collabora",
                "TALK_HOST=nextcloud-aio-talk",
                "APACHE_PORT=11000",
                "ONLYOFFICE_HOST=nextcloud-aio-onlyoffice",
                "TZ=Europe/Berlin",
                "APACHE_MAX_SIZE=10737418240",
                "APACHE_MAX_TIME=3600",
                "NOTIFY_PUSH_HOST=nextcloud-aio-notify-push",
                "PATH=/usr/local/apache2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "HTTPD_PREFIX=/usr/local/apache2",
                "HTTPD_VERSION=2.4.62",
                "HTTPD_SHA256=674188e7bf44ced82da8db522da946849e22080d73d16c93f7f4df89e25729ec",
                "HTTPD_PATCHES="
            ],
            "Cmd": [
                "/usr/bin/supervisord",
                "-c",
                "/supervisord.conf"
            ],
            "Healthcheck": {
                "Test": [
                    "CMD-SHELL",
                    "/healthcheck.sh"
                ]
            },
            "Image": "nextcloud/aio-apache:latest",
            "Volumes": {
                "/mnt/data": {}
            },
            "WorkingDir": "/usr/local/apache2",
            "Entrypoint": [
                "/start.sh"
            ],
            "OnBuild": null,
            "Labels": {
                "com.centurylinklabs.watchtower.enable": "false"
            },
            "StopSignal": "SIGWINCH"
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "56ed7026989321d8152039d996e158e54f3078fa791a7547c76d6d35d7c2c089",
            "SandboxKey": "/var/run/docker/netns/56ed70269893",
            "Ports": {
                "11000/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "11000"
                    },
                    {
                        "HostIp": "::",
                        "HostPort": "11000"
                    }
                ],
                "80/tcp": null
            },
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "nextcloud-aio": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "MacAddress": "02:42:ac:12:00:08",
                    "DriverOpts": null,
                    "NetworkID": "d8bf03b48097ae0a7af91c32c17411c3faa22adec9d98bba479edd5023e71395",
                    "EndpointID": "bf6a25931ec2c1118a0cb9d58204a54b8fa88347c73bf044353a1bc3c56bc9d4",
                    "Gateway": "172.18.0.1",
                    "IPAddress": "172.18.0.8",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "DNSNames": [
                        "nextcloud-aio-apache",
                        "d30cc582ef7a"
                    ]
                }
            }
        }
    }
]

and

curl http://localhost:11000/login
curl: (56) Recv failure: Connection reset by peer

I restarted the whole server. Checked the DNS setting in the Strato dashboard. Nothing seems to have changed.
All other services behind the Proxy Apache are working properly.

I contacted Strato if on their and is something wrong with their DNS configuration.

But did not get any answer from them until now.

Anything that I can check or do?

You will not believe it.
I have crowdsec running on my machine.

After I stopped the crowdsec-firewall-bouncer the Apache starts properly and the cloud can be accessed.

Now I need to find out what happened and why only the cloud was affected and not the rest of the server itself.

I hoped to fix the problem by whitelisting the Strato DNS server and localhost.

sudo nano /etc/crowdsec/parsers/s02-enrich/mywhitelist.yaml

And filled in this

name: my/whitelists
description: "Whitelist events from private ipv4 addresses and Strato DNS"
whitelist:
  reason: "private ipv4/ipv6 ip/ranges"
  ip:
    - "127.0.0.1"
    - "::1"
    - "81.169.163.106"
    - "85.214.7.22"
    - "81.169.148.34"
  cidr:
    - "192.168.0.0/16"
    - "10.0.0.0/8"
    - "172.16.0.0/12"

After that I started crowdsec and the firewall bouncer again.

And then unfortunately the apache docker container goes again in unhealthy status and the cloud is not accessible any more.

Does any one know what the apache container does that may be blocked by crowdsec?

once you know crowdsec is the root cause I would analyze the problem from this side - there must be some log telling you the reason why the access is blocked.

I have found the problem.

For some reason the IP 172.18.0.8 is on the Crowdsec Blocklist Firehol greensnow.co list.
I removed all blocklists to never get in such a problem again.

The question I still have, is it possible to change the IP range for the nextcloud-aio network?

That’s not a solution imho, because 172.18.0.8 shouldn’t be on any public blocklists in the first place, as it’s a private address within the RFC1918 space. I mean, what if the IP range you change it to suddenly ends up on their blocklist too? I would contact them and ask them why they think it’s a good idea to add RFC1918 addresses to their blocklists… Contact - GreenSnow

1 Like

Yes, you can set the ip-range by recreating the nextcloud-aio network: https://github.com/nextcloud/all-in-one/discussions/4989

During the docker network create command, simply specify the ip-range.

Thank you @szaimen and @bb77.

I contacted GreenSnow. Let see what happens.
And I will change my IP range so that I can use again the GreenSnow list.

Thank you all for your help.