Since yesterday 08/22/2024 10am my Nextcloud AIO hosted at Strato is not working any more.
Environment: Nextcloud AIO v9.4.1
The apache container is going in the status unhealthy and no connection to the could is working any more.
My AIO is running behind an Apache as reverse proxy. Until now I never had any problems.
Apache container log
2024-08-23T06:06:47.703245749Z Waiting for Nextcloud to start...
2024-08-23T06:06:52.705972455Z Waiting for Nextcloud to start...
2024-08-23T06:06:57.708674927Z Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!
2024-08-23T06:07:02.047275151Z [Fri Aug 23 08:07:02.046751 2024] [mpm_event:notice] [pid 74:tid 74] AH00489: Apache/2.4.62 (Unix) configured -- resuming normal operations
2024-08-23T06:07:02.047327550Z [Fri Aug 23 08:07:02.046809 2024] [core:notice] [pid 74:tid 74] AH00094: Command line: '/usr/local/apache2/bin/httpd -D FOREGROUND'
2024-08-23T06:07:02.134412801Z {"level":"info","ts":1724393222.1340866,"msg":"using config from file","file":"/tmp/Caddyfile"}
2024-08-23T06:07:02.136904966Z {"level":"info","ts":1724393222.1367307,"msg":"adapted config to JSON","adapter":"caddyfile"}
My reverse proxy configuration
My reverse proxy configuration
<IfModule mod_ssl.c>
<IfDefine IgnoreBlockComment>
<VirtualHost *:80>
ServerName cloud.waswaasisch.de
ServerAlias cloud
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
RewriteCond %{SERVER_NAME} =cloud.waswaasisch.de
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
</IfDefine>
<VirtualHost *:443>
ServerAdmin webmaster@waswaasisch.de
ServerName cloud.waswaasisch.de
ServerAlias cloud
# Reverse proxy
RewriteEngine On
ProxyPreserveHost On
RequestHeader set X-Real-IP %{REMOTE_ADDR}s
AllowEncodedSlashes NoDecode
# RewriteCond %{HTTP:Upgrade} websocket [NC]
# RewriteCond %{HTTP:Connection} upgrade [NC]
# RewriteRule ^/(.*) "ws://localhost:11000/$1" [P,L]
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteCond %{HTTP:Connection} upgrade [NC]
RewriteCond %{THE_REQUEST} "^[a-zA-Z]+ /(.*) HTTP/\d+(\.\d+)?$"
RewriteRule .? "ws://localhost:11000/%1" [P,L,UnsafeAllow3F]
# ProxyRequests off
# Solves slow upload speeds caused by http2
H2WindowSize 5242880
ProxyPass / http://localhost:11000/ nocanon
ProxyPassReverse / http://localhost:11000/
# Enable h2, h2c and http1.1
Protocols h2 h2c http/1.1
Header set X-Frame-Options "SAMEORIGIN"
ErrorLog ${APACHE_LOG_DIR}/ssl_cloud_error.log
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/ssl_cloud_access.log combined
SSLEngine on
SSLCertificateFile /etc/ssl/letsencrypt/signed.crt
SSLCertificateKeyFile /etc/ssl/letsencrypt/domain.key
SSLCertificateChainFile /etc/ssl/letsencrypt/fullchain.pem
SSLProtocol all -SSLv2 -SSLv3 -TLSv1
SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256
SSLHonorCipherOrder on
SSLCompression off
# Disable HTTP TRACE method.
TraceEnable off
<Files ".ht*">
Require all denied
</Files>
</VirtualHost>
</IfModule>
I ran
sudo docker exec -it nextcloud-aio-apache bash -x /healthcheck.sh
+ nc -z nextcloud-aio-nextcloud 9000
Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!
+ nc -z 127.0.0.1 8000
Connection to 127.0.0.1 8000 port [tcp/*] succeeded!
+ nc -z 127.0.0.1 11000
Connection to 127.0.0.1 11000 port [tcp/*] succeeded!
+ nc -z cloud.waswaasisch.de 443
+ echo 'Could not reach cloud.waswaasisch.de on port 443.'
Could not reach cloud.waswaasisch.de on port 443.
+ exit 1
Did also inspect of the apache contianer
sudo docker inspect nextcloud-aio-apache
[
{
"Id": "d30cc582ef7acd1c1d2627d1844b8db9f06a7b6153d76b7c9270cbfe59883dc4",
"Created": "2024-08-23T06:05:47.340578428Z",
"Path": "/start.sh",
"Args": [
"/usr/bin/supervisord",
"-c",
"/supervisord.conf"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 919371,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-08-23T06:05:51.141694995Z",
"FinishedAt": "0001-01-01T00:00:00Z",
"Health": {
"Status": "unhealthy",
"FailingStreak": 29,
"Log": [
{
"Start": "2024-08-23T08:30:47.920249171+02:00",
"End": "2024-08-23T08:31:19.967435357+02:00",
"ExitCode": -1,
"Output": "Health check exceeded timeout (30s): Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 8000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 11000 port [tcp/*] succeeded!\n"
},
{
"Start": "2024-08-23T08:31:49.967949104+02:00",
"End": "2024-08-23T08:32:20.013632867+02:00",
"ExitCode": -1,
"Output": "Health check exceeded timeout (30s): Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 8000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 11000 port [tcp/*] succeeded!\n"
},
{
"Start": "2024-08-23T08:32:50.014679216+02:00",
"End": "2024-08-23T08:33:20.058428826+02:00",
"ExitCode": -1,
"Output": "Health check exceeded timeout (30s): Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 8000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 11000 port [tcp/*] succeeded!\n"
},
{
"Start": "2024-08-23T08:33:50.059196864+02:00",
"End": "2024-08-23T08:34:21.493482902+02:00",
"ExitCode": -1,
"Output": "Health check exceeded timeout (30s): Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 8000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 11000 port [tcp/*] succeeded!\n"
},
{
"Start": "2024-08-23T08:34:51.494489119+02:00",
"End": "2024-08-23T08:35:21.541909171+02:00",
"ExitCode": -1,
"Output": "Health check exceeded timeout (30s): Connection to nextcloud-aio-nextcloud (172.18.0.6) 9000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 8000 port [tcp/*] succeeded!\nConnection to 127.0.0.1 11000 port [tcp/*] succeeded!\n"
}
]
}
},
"Image": "sha256:dc12e9895b977d6b49405810c2db2aca5a5d6ae00c0eac96d55462be7a414fd8",
"ResolvConfPath": "/var/lib/docker/containers/d30cc582ef7acd1c1d2627d1844b8db9f06a7b6153d76b7c9270cbfe59883dc4/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/d30cc582ef7acd1c1d2627d1844b8db9f06a7b6153d76b7c9270cbfe59883dc4/hostname",
"HostsPath": "/var/lib/docker/containers/d30cc582ef7acd1c1d2627d1844b8db9f06a7b6153d76b7c9270cbfe59883dc4/hosts",
"LogPath": "/var/lib/docker/containers/d30cc582ef7acd1c1d2627d1844b8db9f06a7b6153d76b7c9270cbfe59883dc4/d30cc582ef7acd1c1d2627d1844b8db9f06a7b6153d76b7c9270cbfe59883dc4-json.log",
"Name": "/nextcloud-aio-apache",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": [
"723c0c73e45e2d7e615ae9b9bbc2985d35c83f064e49b7b6b467cbe3eb46ced2"
],
"HostConfig": {
"Binds": [
"nextcloud_aio_nextcloud:/var/www/html:ro",
"nextcloud_aio_apache:/mnt/data:rw"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "nextcloud-aio",
"PortBindings": {
"11000/tcp": [
{
"HostIp": "",
"HostPort": "11000"
}
]
},
"RestartPolicy": {
"Name": "unless-stopped",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": [
"NET_RAW"
],
"CgroupnsMode": "host",
"Dns": null,
"DnsOptions": null,
"DnsSearch": null,
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": true,
"SecurityOpt": [
"label:disable"
],
"Tmpfs": {
"/home/www-data": "",
"/tmp": "",
"/usr/local/apache2/logs": "",
"/var/log/supervisord": "",
"/var/run/supervisord": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": null,
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": [
"/proc/asound",
"/proc/acpi",
"/proc/kcore",
"/proc/keys",
"/proc/latency_stats",
"/proc/timer_list",
"/proc/timer_stats",
"/proc/sched_debug",
"/proc/scsi",
"/sys/firmware",
"/sys/devices/virtual/powercap"
],
"ReadonlyPaths": [
"/proc/bus",
"/proc/fs",
"/proc/irq",
"/proc/sys",
"/proc/sysrq-trigger"
],
"Init": true
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/8a536119ccda38a801f04f0cb6c0b5d114bcbbd751fc16d2c700e8f67c0cc397-init/diff:/var/lib/docker/overlay2/fbe59fb530d1f7f87be3b05f8401d77715f9f2f2d7681d89636893c93db63cf2/diff:/var/lib/docker/overlay2/839f3f55c36ea33e214c6f095cfe2d9ca3cdc1a10377dc557af30f12f57f47a4/diff:/var/lib/docker/overlay2/051cdd5d6858cfd847c60306a1d2dd3db4286e0609096fabbba412eeb147bc26/diff:/var/lib/docker/overlay2/15e14032103c98f5644316c17ad743c3cb16afd84d28f3c8adb8ba4f34505f79/diff:/var/lib/docker/overlay2/2b8b5d2051998de7d597476a59ed859aa1e0bdcfb6247836fefd0fdac4a18f07/diff:/var/lib/docker/overlay2/bb7da6b42923fffc62de3d30ba76be42b185cadfa41634c5db4930b25135de4f/diff:/var/lib/docker/overlay2/8f87f68fab466dcd970aabb3577eac3a03a46636801fab0948dbd3e540a113b4/diff:/var/lib/docker/overlay2/09b4f994a22c1d4f872bfd5b3566bb517dd696fe7e1e0c2238b2cc5741f9f132/diff:/var/lib/docker/overlay2/133b1a0f2f9f0a7d2ac25d0f6337a7f2709d33498e8e25cb87c1991b9ed43e8a/diff:/var/lib/docker/overlay2/41c30c0303be7520433f38a623a28fd9fd09f2efab59acb94dd0af5055ec21c4/diff:/var/lib/docker/overlay2/e11fca2d18e0ec14f307ae6eb7469d71c5be3deedc5788c0df5d663885e329bf/diff:/var/lib/docker/overlay2/6ec0995deb211357978dcc93a9f7e93c18efd3328fdeca6a8159e4415619380d/diff:/var/lib/docker/overlay2/b19e461b849439af405dd762bd45633006eadabe8229f0187e1a787d4931fb98/diff:/var/lib/docker/overlay2/b970399fee2388b7630c89e11ae772b01f278a1ee1ba8964ceef2cacba54fe95/diff",
"MergedDir": "/var/lib/docker/overlay2/8a536119ccda38a801f04f0cb6c0b5d114bcbbd751fc16d2c700e8f67c0cc397/merged",
"UpperDir": "/var/lib/docker/overlay2/8a536119ccda38a801f04f0cb6c0b5d114bcbbd751fc16d2c700e8f67c0cc397/diff",
"WorkDir": "/var/lib/docker/overlay2/8a536119ccda38a801f04f0cb6c0b5d114bcbbd751fc16d2c700e8f67c0cc397/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "volume",
"Name": "nextcloud_aio_nextcloud",
"Source": "/var/lib/docker/volumes/nextcloud_aio_nextcloud/_data",
"Destination": "/var/www/html",
"Driver": "local",
"Mode": "ro",
"RW": false,
"Propagation": ""
},
{
"Type": "volume",
"Name": "nextcloud_aio_apache",
"Source": "/var/lib/docker/volumes/nextcloud_aio_apache/_data",
"Destination": "/mnt/data",
"Driver": "local",
"Mode": "rw",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "d30cc582ef7a",
"Domainname": "",
"User": "www-data",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"11000/tcp": {},
"80/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"NC_DOMAIN=cloud.waswaasisch.de",
"NEXTCLOUD_HOST=nextcloud-aio-nextcloud",
"APACHE_HOST=nextcloud-aio-apache",
"COLLABORA_HOST=nextcloud-aio-collabora",
"TALK_HOST=nextcloud-aio-talk",
"APACHE_PORT=11000",
"ONLYOFFICE_HOST=nextcloud-aio-onlyoffice",
"TZ=Europe/Berlin",
"APACHE_MAX_SIZE=10737418240",
"APACHE_MAX_TIME=3600",
"NOTIFY_PUSH_HOST=nextcloud-aio-notify-push",
"PATH=/usr/local/apache2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"HTTPD_PREFIX=/usr/local/apache2",
"HTTPD_VERSION=2.4.62",
"HTTPD_SHA256=674188e7bf44ced82da8db522da946849e22080d73d16c93f7f4df89e25729ec",
"HTTPD_PATCHES="
],
"Cmd": [
"/usr/bin/supervisord",
"-c",
"/supervisord.conf"
],
"Healthcheck": {
"Test": [
"CMD-SHELL",
"/healthcheck.sh"
]
},
"Image": "nextcloud/aio-apache:latest",
"Volumes": {
"/mnt/data": {}
},
"WorkingDir": "/usr/local/apache2",
"Entrypoint": [
"/start.sh"
],
"OnBuild": null,
"Labels": {
"com.centurylinklabs.watchtower.enable": "false"
},
"StopSignal": "SIGWINCH"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "56ed7026989321d8152039d996e158e54f3078fa791a7547c76d6d35d7c2c089",
"SandboxKey": "/var/run/docker/netns/56ed70269893",
"Ports": {
"11000/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "11000"
},
{
"HostIp": "::",
"HostPort": "11000"
}
],
"80/tcp": null
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"nextcloud-aio": {
"IPAMConfig": null,
"Links": null,
"Aliases": null,
"MacAddress": "02:42:ac:12:00:08",
"DriverOpts": null,
"NetworkID": "d8bf03b48097ae0a7af91c32c17411c3faa22adec9d98bba479edd5023e71395",
"EndpointID": "bf6a25931ec2c1118a0cb9d58204a54b8fa88347c73bf044353a1bc3c56bc9d4",
"Gateway": "172.18.0.1",
"IPAddress": "172.18.0.8",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"nextcloud-aio-apache",
"d30cc582ef7a"
]
}
}
}
}
]
and
curl http://localhost:11000/login
curl: (56) Recv failure: Connection reset by peer
I restarted the whole server. Checked the DNS setting in the Strato dashboard. Nothing seems to have changed.
All other services behind the Proxy Apache are working properly.
I contacted Strato if on their and is something wrong with their DNS configuration.
But did not get any answer from them until now.
Anything that I can check or do?