I’m running Nextcloud on Ubuntu 16.04 Server with Nginx and php7.0-fpm. Everything works well (including CalDAV and CardDAV sync) except for an issue I have renewing a Certbot HTTPS certificate. Does anyone have a tested and working Nginx server block and/or configuration tips on how to get this working?
To test renewing Certbot HTTPS certificates, I run the command sudo ~/certbot-auto renew --dry-run --agree-tos which returns the error “The client lacks sufficient authorization”. This issue is unique to my Nextcloud site, and I have several other (static HTML) sites hosted on the same machine and renewing their certificates works perfectly.
The Nextcloud site’s document root is at /var/www/EXAMPLE.COM and my Nextcloud data directory is at /var/nextcloud-data. Nginx runs as the www-data user and /var/www/EXAMPLE.COM is also owned by www-data, so I’m confused by the authorization error.
There is nothing in Nginx’s error log after running the Certbot script, and I don’t understand how to troubleshoot this further. Any ideas please?
Here’s the full output of the command with errors:
-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/EXAMPLE.COM.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for EXAMPLE.COM
http-01 challenge for www.EXAMPLE.COM
Waiting for verification...
Cleaning up challenges
Attempting to renew cert from /etc/letsencrypt/renewal/EXAMPLE.COM.conf produced an unexpected error: Failed authorization procedure. EXAMPLE.COM (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://EXAMPLE.COM/.well-known/acme-challenge/cjEH-KS0ZzxSJoZ6lpGYP2q1ch9j08QstTNp4I4tEj0: "<!DOCTYPE html>
<html class="ng-csp" data-placeholder-focus="false" lang="en" >
<head data-requesttoken="Bk9GOVgTXAsDLX5rLjlcIR", www.EXAMPLE.COM (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.EXAMPLE.COM/.well-known/acme-challenge/8_vMjX6dzYF2TQ_JSQY-subI8nD4R5omuGMmxDf8kDs: "<!DOCTYPE html>
<html class="ng-csp" data-placeholder-focus="false" lang="en" >
<head data-requesttoken="CzsdBSwhElZhRlxxREo7M0". Skipping.
The following certs could not be renewed:
/etc/letsencrypt/live/EXAMPLE.COM/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates above have not been saved.)
1 renew failure(s), 0 parse failure(s)
IMPORTANT NOTES:
- The following errors were reported by the server:
Domain: EXAMPLE.COM
Type: unauthorized
Detail: Invalid response from
http://EXAMPLE.COM/.well-known/acme-challenge/cjEH-KS0ZzxSJoZ6lpGYP2q1ch9j08QstTNp4I4tEj0:
"<!DOCTYPE html>
<html class="ng-csp" data-placeholder-focus="false" lang="en" >
<head data-requesttoken="Bk9GOVgTXAsDLX5rLjlcIR"
Domain: www.EXAMPLE.COM
Type: unauthorized
Detail: Invalid response from
http://www.EXAMPLE.COM/.well-known/acme-challenge/8_vMjX6dzYF2TQ_JSQY-subI8nD4R5omuGMmxDf8kDs:
"<!DOCTYPE html>
<html class="ng-csp" data-placeholder-focus="false" lang="en" >
<head data-requesttoken="CzsdBSwhElZhRlxxREo7M0"
The script creates the folder and I verified that this does happen as expected. I can delete the folder, run the Certbot script, and the folder is re-created (but the same error persists). I have also tried placing the .well-known folder outside of the directory Nextcloud is in and linking to it with a Nginx location block and this also does not work.
All troubleshooting attempts to date have returned the same error message starting with <html class="ng-csp" data-placeholder-focus="false" lang="en" >… which is not a Ngink error, I believe this response is originating from Nextcloud. There must be loads of people using Nextcloud with Nginx and Certbot by now and I’d love to learn how to make this work.
I think I tried that before (with default charset and allow directives) but just tried again and I have a different error Attempting to renew cert from /etc/letsencrypt/renewal/EXAMPLE.COM.conf produced an unexpected error: At least one of the required ports is already taken.. Skipping.
If I check with netstat I see Nginx is the only thing listening on ports 80 & 443, as expected. I have multiple domains on this machine and all the others test and renew with no errors.
I don’t know if this will help but here’s the full Nginx server configuration for the Nextcloud domain:
upstream php-handler {
server unix:/run/php/php7.0-fpm.sock;
}
server {
# The basics
listen 80;
server_name EXAMPLE.COM www.EXAMPLE.COM;
# Redirect HTTP > HTTPS
# HSTS preload requires HTTP > HTTPS redirect before redirect to www subdomain
return 301 https://$server_name$request_uri;
# Set cache & expires headers along with 301 in case we ever change it
expires 24h;
add_header Cache-Control "no-store, no-cache, must-revalidate";
}
server {
# The basics
listen 443 ssl http2;
server_name EXAMPLE.COM;
# Includes
include /etc/nginx/conf.d/https.conf;
# HTTPS
ssl_certificate /etc/letsencrypt/live/EXAMPLE.COM/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/EXAMPLE.COM/privkey.pem;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload";
# OCSP trusted Intermediate
ssl_trusted_certificate /etc/nginx/ssl/ocsp-lets-encrypt-x3-cross-signed.pem;
# Redirect no-www to www
return 301 https://www.EXAMPLE.COM$request_uri;
# Set cache & expires headers along with 301 in case we ever change it
expires 24h;
add_header Cache-Control "no-store, no-cache, must-revalidate";
}
server {
# The basics
server_name www.EXAMPLE.COM;
root /var/www/EXAMPLE.COM;
index index.html;
# HTTPS
listen 443 ssl http2;
ssl_certificate /etc/letsencrypt/live/EXAMPLE.COM/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/EXAMPLE.COM/privkey.pem;
# Includes
include /etc/nginx/conf.d/https.conf;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload";
# OCSP trusted Intermediate
ssl_trusted_certificate /etc/nginx/ssl/ocsp-lets-encrypt-x3-cross-signed.pem;
# HPKP (Certbot, then Gandi for a 'plan b')
add_header Public-Key-Pins 'pin-sha256="YLh1dUR9y6Kja30RrAn7JKnbQG/uEtLMkBgFF2Fuihg="; pin-sha256="WGJkyYjx1QMdMe0UqlyOKXtydPDVrk7sl2fV+nNm1r4="; max-age=2592000; report-uri="https://tom.report-uri.io/r/default/hpkp/enforce"';
# Security
server_tokens off;
add_header X-Content-Type-Options nosniff;
add_header X-Frame-Options "SAMEORIGIN";
add_header X-XSS-Protection "1; mode=block";
add_header X-Robots-Tag none;
add_header X-Download-Options noopen;
add_header X-Permitted-Cross-Domain-Policies none;
# Block the sending of all referer headers
add_header referrer-policy "no-referrer" always;
# Disallow accessing dotfiles (`/.well-known/` is allowed)
location ~* /\.(?!well-known\/) {
deny all;
}
# Prevent clients from accessing to backup/config/source files
location ~* (?:\.(?:build|tests|config|lib|3rdparty|templates|data|bak|sql|fla|psd|ini|log|sh|inc|swp|dist)|~)$ {
deny all;
}
location = /robots.txt {
allow all;
log_not_found off;
access_log off;
}
location = /.well-known/carddav {
return 301 $scheme://$host/remote.php/dav;
}
location = /.well-known/caldav {
return 301 $scheme://$host/remote.php/dav;
}
location /.well-known/acme-challenge { }
# Max upload size
client_max_body_size 1G;
fastcgi_buffers 64 4K;
gzip off;
error_page 403 /core/templates/403.php;
error_page 404 /core/templates/404.php;
location / {
rewrite ^ /index.php$uri;
}
location ~^/(?:index|remote|public|cron|core/ajax/update|status|ocs/v[12]|updater/.+|ocs-provider/.+|core/templates/40[34])\.php(?:$|/) {
include fastcgi_params;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
fastcgi_param HTTPS on;
#Avoid sending the security headers twice
fastcgi_param modHeadersAvailable true;
fastcgi_param front_controller_active true;
fastcgi_pass php-handler;
fastcgi_intercept_errors on;
fastcgi_request_buffering off;
}
location ~ ^/(?:updater|ocs-provider)(?:$|/) {
try_files $uri/ =404;
index index.php;
}
location ~* \.(?:css|js)$ {
try_files $uri /index.php$uri$is_args$args;
add_header Cache-Control "public, max-age=7200";
add_header X-Content-Type-Options nosniff;
add_header X-Frame-Options "SAMEORIGIN";
add_header X-XSS-Protection "1; mode=block";
add_header X-Robots-Tag none;
add_header X-Download-Options noopen;
add_header X-Permitted-Cross-Domain-Policies none;
# Optional: Don't log access to assets
access_log off;
}
location ~* \.(?:svg|gif|png|html|ttf|woff|ico|jpg|jpeg)$ {
try_files $uri /index.php$uri$is_args$args;
access_log off;
}
}
(also, thanks for the troubleshooting help so far)
I’m not sure if the order of listen and server_name in the vhost configuration matters. You can also try with curl to see what exactly is happening. You use a few redirects, not sure if that matters for the process.
A quick and dirty solution would be, to create a new vhost-file only for the verification process. Once it is done, you put in place your normal Nextcloud config file.
Thanks much, I tried that and it still didn’t work, however it did remind me to try using Certbot’s --webroot option and that did work. I wish I had thought to try that earlier!
Here’s the line in my root crontab that is now tested and working for automated renewal of all certs on the machine: @daily /home/tom/certbot-auto/certbot-auto renew --webroot -w /var/www/certbot/ --quiet --agree-tos
I’m using the full path to the latest certbot-auto executable (/home/tom/certbot-auto/certbot-auto) so either change the path for your machine or replace the command with what the executable is named on your machine to adapt my example. I’ll now revert to my original Nginx server block for Nextcloud, as it no longer has any effect on the Certbot renewal process.
line 202, in _poll_challenges
raise errors.FailedChallenges(all_failed_achalls)
FailedChallenges: Failed authorization procedure. my-website.domain.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://my-website.domain.com/.well-known/acme-challenge/Z0OOPk9LNsnk6JuVzxVA8_zFq7lqbczJBvO-r2kDznE: "<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>"
Please see the logfiles in /var/log/letsencrypt for more details.
IMPORTANT NOTES:
- The following errors were reported by the server:
Domain: my-website.domain.com
Type: unauthorized
Detail: Invalid response from
http:/my-website.domain.com/.well-known/acme-challenge/Z0OOPk9LNsnk6JuVzxVA8_zFq7lqbczJBvO-r2kDznE:
"<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>"
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.
(ElasticBeanstalk::ExternalInvocationError)