503 Service Temporarily Unavailable after update (fpm nginx proxy)

Burner · April 12, 2021, 4:32pm

Hi,
I’m using the example docker/.examples/docker-compose/with-nginx-proxy/mariadb/fpm/docker-compose.yml from GitHub - nextcloud/docker: ⛴ Docker image of Nextcloud as a basis for my setup. I upgraded yesterday as usual and ended up in getting a 503 from my server.

For me it seems the nginx-proxy doesn’t really work but I don’t know how to fix this. I’m especially sceptical about this warning/error:

proxy_nginx              | nginx: [emerg] no servers are inside upstream in /etc/nginx/conf.d/default.conf:68

Below the log of the docker startup, can someone help?
Thanks!

docker-compose up      
Creating network "docker_default" with the default driver
Creating network "docker_proxy-tier" with the default driver
Creating proxy_nginx    ... done
Creating redis       ... done
Creating mariadb     ... done
Creating nextcloud      ... done
Creating cron_nextcloud ... done
Creating letsencrypt    ... done
Creating nginx          ... done
Attaching to mariadb, redis, proxy_nginx, cron_nextcloud, nextcloud, letsencrypt, nginx
cron_nextcloud           | crond: crond (busybox 1.32.1) started, log level 0
cron_nextcloud           | crond: user:www-data entry:*/5 * * * * php -f /var/www/html/cron.php
cron_nextcloud           | 100001000010000100001000010000100001000010000100001000010000
cron_nextcloud           | 111111111111111111111111
cron_nextcloud           | 11111111111111111111111111111111
cron_nextcloud           | 111111111111
cron_nextcloud           | 1111111
mariadb                  | 2021-04-12 11:12:28+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.9+maria~focal started.
mariadb                  | 2021-04-12 11:12:28+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
mariadb                  | 2021-04-12 11:12:28+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.9+maria~focal started.
mariadb                  | 2021-04-12 11:12:29 0 [Note] mysqld (mysqld 10.5.9-MariaDB-1:10.5.9+maria~focal) starting as process 1 ...
mariadb                  | 2021-04-12 11:12:29 0 [Warning] You need to use --log-bin to make --binlog-format work.
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: Uses event mutexes
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: Number of pools: 1
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
mariadb                  | 2021-04-12 11:12:29 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: Using Linux native AIO
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: Completed initialization of buffer pool
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: 128 rollback segments are active.
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: 10.5.9 started; log sequence number 63579510598; transaction id 55481974
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
mariadb                  | 2021-04-12 11:12:29 0 [Note] Plugin 'FEEDBACK' is disabled.
mariadb                  | 2021-04-12 11:12:29 0 [Note] Server socket created on IP: '::'.
mariadb                  | 2021-04-12 11:12:29 0 [Note] InnoDB: Buffer pool(s) load completed at 210412 11:12:29
mariadb                  | 2021-04-12 11:12:29 0 [Warning] 'proxies_priv' entry '@% root@482a9b02f4e9' ignored in --skip-name-resolve mode.
mariadb                  | 2021-04-12 11:12:29 0 [Note] Reading of all Master_info entries succeeded
mariadb                  | 2021-04-12 11:12:29 0 [Note] Added new Master_info '' to hash table
mariadb                  | 2021-04-12 11:12:29 0 [Note] mysqld: ready for connections.
mariadb                  | Version: '10.5.9-MariaDB-1:10.5.9+maria~focal'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
letsencrypt              | Info: running acme-companion version v2.1.0-1-gd56d1aa
letsencrypt              | Warning: '/etc/acme.sh' does not appear to be a mounted volume.
letsencrypt              | Info: Custom Diffie-Hellman group found, generation skipped.
nextcloud                | [12-Apr-2021 11:12:29] NOTICE: fpm is running, pid 1
nextcloud                | [12-Apr-2021 11:12:29] NOTICE: ready to handle connections
letsencrypt              | Reloading nginx proxy (a77...)...
proxy_nginx              | WARNING: /etc/nginx/dhparam/dhparam.pem was not found. A pre-generated dhparam.pem will be used for now while a new one
proxy_nginx              | is being generated in the background.  Once the new dhparam.pem is in place, nginx will be reloaded.
proxy_nginx              | forego     | starting dockergen.1 on port 5000
proxy_nginx              | Generating DSA parameters, 4096 bit long prime
proxy_nginx              | forego     | starting nginx.1 on port 5100
proxy_nginx              | dockergen.1 | 2021/04/12 11:12:29 Generated '/etc/nginx/conf.d/default.conf' from 3 containers
proxy_nginx              | dockergen.1 | 2021/04/12 11:12:29 Watching docker events
proxy_nginx              | dockergen.1 | 2021/04/12 11:12:29 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'
proxy_nginx              | dockergen.1 | 2021/04/12 11:12:29 Received event start for container a4c7945f6fec
proxy_nginx              | dockergen.1 | 2021/04/12 11:12:29 Received event start for container 38e5fad2e302
proxy_nginx              | dockergen.1 | 2021/04/12 11:12:30 Received event start for container 2f48c3fed9f6
redis                    | 1:C 12 Apr 2021 11:12:29.087 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis                    | 1:C 12 Apr 2021 11:12:29.087 # Redis version=6.2.1, bits=64, commit=00000000, modified=0, pid=1, just started
redis                    | 1:C 12 Apr 2021 11:12:29.087 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis                    | 1:M 12 Apr 2021 11:12:29.088 * monotonic clock: POSIX clock_gettime
redis                    | 1:M 12 Apr 2021 11:12:29.088 * Running mode=standalone, port=6379.
redis                    | 1:M 12 Apr 2021 11:12:29.088 # Server initialized
redis                    | 1:M 12 Apr 2021 11:12:29.088 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis                    | 1:M 12 Apr 2021 11:12:29.088 * Ready to accept connections
nginx                    | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx                    | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx                    | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
nginx                    | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
proxy_nginx              | dockergen.1 | 2021/04/12 11:12:30 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'
proxy_nginx              | dockergen.1 | 2021/04/12 11:12:30 Received event start for container d5ce61d13e5a
proxy_nginx              | dockergen.1 | 2021/04/12 11:12:30 Generated '/etc/nginx/conf.d/default.conf' from 7 containers
proxy_nginx              | dockergen.1 | 2021/04/12 11:12:30 Error running notify command: nginx -s reload, exit status 1
nginx                    | 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
nginx                    | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
proxy_nginx              | dockergen.1 | 2021/04/12 11:12:30 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'
nginx                    | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
nginx                    | /docker-entrypoint.sh: Configuration complete; ready for start up
proxy_nginx              | dockergen.1 | 2021/04/12 11:12:30 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'
letsencrypt              | 2021/04/12 11:12:30 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification ''
letsencrypt              | 2021/04/12 11:12:30 Generated '/app/letsencrypt_service_data' from 7 containers
letsencrypt              | 2021/04/12 11:12:30 Running '/app/signal_le_service'
letsencrypt              | 2021/04/12 11:12:30 Watching docker events
letsencrypt              | 2021/04/12 11:12:30 Contents of /app/letsencrypt_service_data did not change. Skipping notification '/app/signal_le_service'
letsencrypt              | [Mon Apr 12 11:12:31 UTC 2021] Create account key ok.
letsencrypt              | [Mon Apr 12 11:12:31 UTC 2021] Registering account: https://acme-v02.api.letsencrypt.org/directory
letsencrypt              | [Mon Apr 12 11:12:33 UTC 2021] Registered
letsencrypt              | [Mon Apr 12 11:12:33 UTC 2021] ACCOUNT_THUMBPRINT='O...'
letsencrypt              | Reloading nginx proxy (a77f753aafa7aa385dbae456811d5e943c4eeb09b4f3b29968f80f370ddec02c)...
letsencrypt              | 2021/04/12 11:12:33 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification ''
letsencrypt              | Creating/renewal MY_DOMAIN certificates... (MY_DOMAIN)
letsencrypt              | [Mon Apr 12 11:12:34 UTC 2021] Using CA: https://acme-v02.api.letsencrypt.org/directory
letsencrypt              | [Mon Apr 12 11:12:34 UTC 2021] Creating domain key
letsencrypt              | [Mon Apr 12 11:12:35 UTC 2021] The domain key is here: /etc/acme.sh/fullacount_and_path.key
letsencrypt              | [Mon Apr 12 11:12:35 UTC 2021] Single domain='MY_DOMAIN'
letsencrypt              | [Mon Apr 12 11:12:35 UTC 2021] Getting domain auth token for each domain
proxy_nginx              | 2021/04/12 11:12:35 [emerg] 98#98: no servers are inside upstream in /etc/nginx/conf.d/default.conf:68
proxy_nginx              | nginx: [emerg] no servers are inside upstream in /etc/nginx/conf.d/default.conf:68
proxy_nginx              | dhparam generation complete, reloading nginx
letsencrypt              | [Mon Apr 12 11:12:37 UTC 2021] Getting webroot for domain='MY_DOMAIN'
letsencrypt              | [Mon Apr 12 11:12:37 UTC 2021] Verifying: MY_DOMAIN
proxy_nginx              | nginx.1    | MY_DOMAIN 52.28.236.88 - - [12/Apr/2021:11:12:39 +0000] "GET /.well-known/acme-challenge/T2N-... HTTP/1.1" 503 197 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
proxy_nginx              | nginx.1    | MY_DOMAIN 34.211.6.84 - - [12/Apr/2021:11:12:39 +0000] "GET /.well-known/acme-challenge/T2N-... HTTP/1.1" 503 197 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
proxy_nginx              | nginx.1    | MY_DOMAIN 3.22.70.135 - - [12/Apr/2021:11:12:39 +0000] "GET /.well-known/acme-challenge/T2N-... HTTP/1.1" 503 197 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
proxy_nginx              | nginx.1    | MY_DOMAIN 64.78.149.164 - - [12/Apr/2021:11:12:39 +0000] "GET  @/.well-known/acme-challenge/T2N-... HTTP/1.1" 503 197 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
letsencrypt              | [Mon Apr 12 11:12:41 UTC 2021] MY_DOMAIN:Verify error:Invalid response from http://MY_DOMAIN/.well-known/acme-challenge/T2N-... [MY_IP]: 
letsencrypt              | [Mon Apr 12 11:12:41 UTC 2021] Please check log file for more details: /dev/null
letsencrypt              | Sleep for 3600s
cron_nextcloud           | crond: user:www-data entry:*/5 * * * * php -f /var/www/html/cron.php
cron_nextcloud           | 100001000010000100001000010000100001000010000100001000010000
cron_nextcloud           | 111111111111111111111111
cron_nextcloud           | 11111111111111111111111111111111
cron_nextcloud           | 111111111111
cron_nextcloud           | 1111111

Reiner_Nippes · April 12, 2021, 5:49pm

did you try to restart docker sudo systemctl restart docker or did you check the output from sudo docker ps?

Burner · April 12, 2021, 6:27pm

Yes I did both. The restart of the docker service or even the whole machine does not change the behaviour.

Output of docker-compose ps

docker-compose ps
     Name                   Command                 State       Ports  
-----------------------------------------------------------------------
cron_nextcloud   /cron.sh                         Up           9000/tcp
letsencrypt      /bin/bash /app/entrypoint. ...   Up                   
mariadb          docker-entrypoint.sh --tra ...   Up           3306/tcp
nextcloud        /entrypoint.sh php-fpm           Up           9000/tcp
nginx            /docker-entrypoint.sh ngin ...   Up           80/tcp  
proxy_nginx      /app/docker-entrypoint.sh  ...   Restarting           
redis            docker-entrypoint.sh redis ...   Up           6379/tcp

The always restarting nginx-proxy was my hint regarding the “no servers are inside upstream …” warning. But I don’t know what do change.

edit: Now I’m getting a connection refused error. Makes sense since there is no service bound to the ports 80 or 443 since the nginx-proxy is not running. Still don’t know how to fix it.

Reiner_Nippes · April 12, 2021, 6:31pm

you won’t get a certificate for MY_DOMAIN. or?

Burner · April 12, 2021, 6:56pm

I exchanges my actual domain with MY_DOMAIN if you mean this.

But that s also a thing if the proxy actually starts up it doesnt work properly. The ACME challenges don’t seem to be answered properly.

Reiner_Nippes · April 13, 2021, 8:15am

did the proxy work before? did you ever got an cert from letsencrypt?

Burner · April 13, 2021, 12:24pm

Yes the proxy was running and I got the certs from letsencrypt. This was fine for around 2 years and the last change was the update.

The setup was also running directly before the update. Other servers (like openvpn; not using the proxy and the letsencrypt cert system) running via docker on the same machine are fine.

Burner · April 18, 2021, 3:07pm

Described my problem and the solution here:

github.com/nextcloud/docker

503 Service Temporarily Unavailable after update (fpm nginx proxy)

opened 11:37AM - 12 Apr 21 UTC

Mailblocker

Hi, I'm using the example docker/.examples/docker-compose/with-nginx-proxy/mari…adb/fpm/docker-compose.yml as a basis for my setup. I upgraded yesterday as usual and ended up in getting a 503 from my server. For me it seems the nginx-proxy doesn't really work but I don't know how to fix this. I'm especially sceptical about this warning/error: ``` proxy_nginx | nginx: [emerg] no servers are inside upstream in /etc/nginx/conf.d/default.conf:68 ``` Below the log of the docker startup, can someone help? Thanks! ``` docker-compose up Creating network "docker_default" with the default driver Creating network "docker_proxy-tier" with the default driver Creating proxy_nginx ... done Creating redis ... done Creating mariadb ... done Creating nextcloud ... done Creating cron_nextcloud ... done Creating letsencrypt ... done Creating nginx ... done Attaching to mariadb, redis, proxy_nginx, cron_nextcloud, nextcloud, letsencrypt, nginx cron_nextcloud | crond: crond (busybox 1.32.1) started, log level 0 cron_nextcloud | crond: user:www-data entry:*/5 * * * * php -f /var/www/html/cron.php cron_nextcloud | 100001000010000100001000010000100001000010000100001000010000 cron_nextcloud | 111111111111111111111111 cron_nextcloud | 11111111111111111111111111111111 cron_nextcloud | 111111111111 cron_nextcloud | 1111111 mariadb | 2021-04-12 11:12:28+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.9+maria~focal started. mariadb | 2021-04-12 11:12:28+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql' mariadb | 2021-04-12 11:12:28+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.9+maria~focal started. mariadb | 2021-04-12 11:12:29 0 [Note] mysqld (mysqld 10.5.9-MariaDB-1:10.5.9+maria~focal) starting as process 1 ... mariadb | 2021-04-12 11:12:29 0 [Warning] You need to use --log-bin to make --binlog-format work. mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: Uses event mutexes mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: Number of pools: 1 mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions mariadb | 2021-04-12 11:12:29 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts) mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: Using Linux native AIO mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728 mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: Completed initialization of buffer pool mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: 128 rollback segments are active. mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: Creating shared tablespace for temporary tables mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: 10.5.9 started; log sequence number 63579510598; transaction id 55481974 mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool mariadb | 2021-04-12 11:12:29 0 [Note] Plugin 'FEEDBACK' is disabled. mariadb | 2021-04-12 11:12:29 0 [Note] Server socket created on IP: '::'. mariadb | 2021-04-12 11:12:29 0 [Note] InnoDB: Buffer pool(s) load completed at 210412 11:12:29 mariadb | 2021-04-12 11:12:29 0 [Warning] 'proxies_priv' entry '@% root@482a9b02f4e9' ignored in --skip-name-resolve mode. mariadb | 2021-04-12 11:12:29 0 [Note] Reading of all Master_info entries succeeded mariadb | 2021-04-12 11:12:29 0 [Note] Added new Master_info '' to hash table mariadb | 2021-04-12 11:12:29 0 [Note] mysqld: ready for connections. mariadb | Version: '10.5.9-MariaDB-1:10.5.9+maria~focal' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution letsencrypt | Info: running acme-companion version v2.1.0-1-gd56d1aa letsencrypt | Warning: '/etc/acme.sh' does not appear to be a mounted volume. letsencrypt | Info: Custom Diffie-Hellman group found, generation skipped. nextcloud | [12-Apr-2021 11:12:29] NOTICE: fpm is running, pid 1 nextcloud | [12-Apr-2021 11:12:29] NOTICE: ready to handle connections letsencrypt | Reloading nginx proxy (a77...)... proxy_nginx | WARNING: /etc/nginx/dhparam/dhparam.pem was not found. A pre-generated dhparam.pem will be used for now while a new one proxy_nginx | is being generated in the background. Once the new dhparam.pem is in place, nginx will be reloaded. proxy_nginx | forego | starting dockergen.1 on port 5000 proxy_nginx | Generating DSA parameters, 4096 bit long prime proxy_nginx | forego | starting nginx.1 on port 5100 proxy_nginx | dockergen.1 | 2021/04/12 11:12:29 Generated '/etc/nginx/conf.d/default.conf' from 3 containers proxy_nginx | dockergen.1 | 2021/04/12 11:12:29 Watching docker events proxy_nginx | dockergen.1 | 2021/04/12 11:12:29 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload' proxy_nginx | dockergen.1 | 2021/04/12 11:12:29 Received event start for container a4c7945f6fec proxy_nginx | dockergen.1 | 2021/04/12 11:12:29 Received event start for container 38e5fad2e302 proxy_nginx | dockergen.1 | 2021/04/12 11:12:30 Received event start for container 2f48c3fed9f6 redis | 1:C 12 Apr 2021 11:12:29.087 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo redis | 1:C 12 Apr 2021 11:12:29.087 # Redis version=6.2.1, bits=64, commit=00000000, modified=0, pid=1, just started redis | 1:C 12 Apr 2021 11:12:29.087 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf redis | 1:M 12 Apr 2021 11:12:29.088 * monotonic clock: POSIX clock_gettime redis | 1:M 12 Apr 2021 11:12:29.088 * Running mode=standalone, port=6379. redis | 1:M 12 Apr 2021 11:12:29.088 # Server initialized redis | 1:M 12 Apr 2021 11:12:29.088 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. redis | 1:M 12 Apr 2021 11:12:29.088 * Ready to accept connections nginx | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration nginx | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ nginx | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh nginx | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf proxy_nginx | dockergen.1 | 2021/04/12 11:12:30 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload' proxy_nginx | dockergen.1 | 2021/04/12 11:12:30 Received event start for container d5ce61d13e5a proxy_nginx | dockergen.1 | 2021/04/12 11:12:30 Generated '/etc/nginx/conf.d/default.conf' from 7 containers proxy_nginx | dockergen.1 | 2021/04/12 11:12:30 Error running notify command: nginx -s reload, exit status 1 nginx | 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf nginx | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh proxy_nginx | dockergen.1 | 2021/04/12 11:12:30 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload' nginx | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh nginx | /docker-entrypoint.sh: Configuration complete; ready for start up proxy_nginx | dockergen.1 | 2021/04/12 11:12:30 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload' letsencrypt | 2021/04/12 11:12:30 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification '' letsencrypt | 2021/04/12 11:12:30 Generated '/app/letsencrypt_service_data' from 7 containers letsencrypt | 2021/04/12 11:12:30 Running '/app/signal_le_service' letsencrypt | 2021/04/12 11:12:30 Watching docker events letsencrypt | 2021/04/12 11:12:30 Contents of /app/letsencrypt_service_data did not change. Skipping notification '/app/signal_le_service' letsencrypt | [Mon Apr 12 11:12:31 UTC 2021] Create account key ok. letsencrypt | [Mon Apr 12 11:12:31 UTC 2021] Registering account: https://acme-v02.api.letsencrypt.org/directory letsencrypt | [Mon Apr 12 11:12:33 UTC 2021] Registered letsencrypt | [Mon Apr 12 11:12:33 UTC 2021] ACCOUNT_THUMBPRINT='O...' letsencrypt | Reloading nginx proxy (a77f753aafa7aa385dbae456811d5e943c4eeb09b4f3b29968f80f370ddec02c)... letsencrypt | 2021/04/12 11:12:33 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification '' letsencrypt | Creating/renewal MY_DOMAIN certificates... (MY_DOMAIN) letsencrypt | [Mon Apr 12 11:12:34 UTC 2021] Using CA: https://acme-v02.api.letsencrypt.org/directory letsencrypt | [Mon Apr 12 11:12:34 UTC 2021] Creating domain key letsencrypt | [Mon Apr 12 11:12:35 UTC 2021] The domain key is here: /etc/acme.sh/fullacount_and_path.key letsencrypt | [Mon Apr 12 11:12:35 UTC 2021] Single domain='MY_DOMAIN' letsencrypt | [Mon Apr 12 11:12:35 UTC 2021] Getting domain auth token for each domain proxy_nginx | 2021/04/12 11:12:35 [emerg] 98#98: no servers are inside upstream in /etc/nginx/conf.d/default.conf:68 proxy_nginx | nginx: [emerg] no servers are inside upstream in /etc/nginx/conf.d/default.conf:68 proxy_nginx | dhparam generation complete, reloading nginx letsencrypt | [Mon Apr 12 11:12:37 UTC 2021] Getting webroot for domain='MY_DOMAIN' letsencrypt | [Mon Apr 12 11:12:37 UTC 2021] Verifying: MY_DOMAIN proxy_nginx | nginx.1 | MY_DOMAIN 52.28.236.88 - - [12/Apr/2021:11:12:39 +0000] "GET /.well-known/acme-challenge/T2N-... HTTP/1.1" 503 197 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" proxy_nginx | nginx.1 | MY_DOMAIN 34.211.6.84 - - [12/Apr/2021:11:12:39 +0000] "GET /.well-known/acme-challenge/T2N-... HTTP/1.1" 503 197 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" proxy_nginx | nginx.1 | MY_DOMAIN 3.22.70.135 - - [12/Apr/2021:11:12:39 +0000] "GET /.well-known/acme-challenge/T2N-... HTTP/1.1" 503 197 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" proxy_nginx | nginx.1 | MY_DOMAIN 64.78.149.164 - - [12/Apr/2021:11:12:39 +0000] "GET @/.well-known/acme-challenge/T2N-... HTTP/1.1" 503 197 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" letsencrypt | [Mon Apr 12 11:12:41 UTC 2021] MY_DOMAIN:Verify error:Invalid response from http://MY_DOMAIN/.well-known/acme-challenge/T2N-... [MY_IP]: letsencrypt | [Mon Apr 12 11:12:41 UTC 2021] Please check log file for more details: /dev/null letsencrypt | Sleep for 3600s cron_nextcloud | crond: user:www-data entry:*/5 * * * * php -f /var/www/html/cron.php cron_nextcloud | 100001000010000100001000010000100001000010000100001000010000 cron_nextcloud | 111111111111111111111111 cron_nextcloud | 11111111111111111111111111111111 cron_nextcloud | 111111111111 cron_nextcloud | 1111111 ```