NC 23.0.3 not accessible over HTTPS

pim · April 2, 2022, 6:50am

Nextcloud version: 23.0.3 (Installed through manual archive)
Operating system and version: Rocky Linux 8.5 x86_64 (CentOS/RHEL based distro)
Apache version: 2.4.37
PHP version: 7.4.28

The issue you are facing

My manual installation of Nextcloud assigned to a custom domain is not accessible through HTTPS.

The output of curl -I http://cloud.mydomain.com/:

HTTP/1.1 302 Found
Date: Sat, 02 Apr 2022 06:44:27 GMT
Server: Apache/2.4.37 (rocky) OpenSSL/1.1.1k
X-Powered-By: PHP/7.4.28
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Content-Security-Policy: default-src 'self'; script-src 'self' 'nonce-XXXX='; style-src 'self' 'unsafe-inline'; frame-src *; img-src * data: blob:; font-src 'self' data:; media-src *; connect-src *; object-src 'none'; base-uri 'self';
Set-Cookie: oc_sessionPassphrase=XXXX; path=/; HttpOnly; SameSite=Lax
Set-Cookie: XXXX; path=/; HttpOnly; SameSite=Lax
Set-Cookie: nc_sameSiteCookielax=true; path=/; httponly;expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax
Set-Cookie: nc_sameSiteCookiestrict=true; path=/; httponly;expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict
Referrer-Policy: no-referrer
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Robots-Tag: none
X-XSS-Protection: 1; mode=block
Location: http://cloud.mydomain.com/login
Content-Type: text/html; charset=UTF-8

Output of curl -I https://cloud.mydomain.com/:

curl: (7) Failed to connect to cloud.mydomain.com port 443 after 11 ms: No route to host

Is this the first time you’ve seen this error?: Y

Steps to replicate it

Archive install of 23.0.3 following the CentOS Example Installation verbatim, using redis, apache and SELinux.
Following the hardening and security guide to enable HTTPS on apache and nextcloud.
Setup DDNS from Namecheap using ddclient.
Attempt to access Nextcloud over https://cloud.mydomain.com.

Config and log outputs

The output of config.php:

<?php
$CONFIG = array (
  'instanceid' => 'ocgud5r92gf2',
  'passwordsalt' => 'XXXX',
  'secret' => 'XXXX',
  'trusted_domains' =>
  array (
	  0 => 'localhost',
	  1 => '192.168.0.208',
	  2 => 'cloud.mydomain.com',
  ),
  'datadirectory' => '/mnt/nextcloud/data',
  'dbtype' => 'mysql',
  'version' => '23.0.3.2',
  'overwrite.cli.url' => 'https://cloud.mydomain.com',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'XXXX',
  'dbpassword' => 'XXXX',
  'installed' => true,
  'htaccess.RewriteBase' => '/',
  'music.lastfm_api_key' => 'XXXX',
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'redis' =>
  array (
    'host' => 'localhost',
    'port' => 6379,
  ),
  'maintenance' => false,
);

The output of Apache log in /var/log/httpd/ (upon fresh restart of httpd.service):

[Sat Apr 02 08:16:18.046242 2022] [mpm_event:notice] [pid 12219:tid 140105173596480] AH00492: caught SIGWINCH, shutting down gracefully
[Sat Apr 02 08:16:28.113673 2022] [core:notice] [pid 12548:tid 139743056038208] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Sat Apr 02 08:16:28.114422 2022] [suexec:notice] [pid 12548:tid 139743056038208] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sat Apr 02 08:16:28.124279 2022] [so:warn] [pid 12548:tid 139743056038208] AH01574: module ssl_module is already loaded, skipping
[Sat Apr 02 08:16:28.127059 2022] [lbmethod_heartbeat:notice] [pid 12548:tid 139743056038208] AH02282: No slotmem from mod_heartmonitor
[Sat Apr 02 08:16:28.129395 2022] [mpm_event:notice] [pid 12548:tid 139743056038208] AH00489: Apache/2.4.37 (rocky) OpenSSL/1.1.1k configured -- resuming normal operations
[Sat Apr 02 08:16:28.129417 2022] [core:notice] [pid 12548:tid 139743056038208] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'

Apache configuration in nextcloud.conf:

<VirtualHost *:80>
  DocumentRoot /var/www/html/nextcloud/
  ServerName cloud.mydomain.com
#   Redirect permanent / https://cloud.mydomain.com/

  <Directory /var/www/html/nextcloud/>
    Require all granted
    AllowOverride All
    Options FollowSymLinks MultiViews

    <IfModule mod_dav.c>
      Dav off
    </IfModule>

  </Directory>
</VirtualHost>

<VirtualHost *:443>
  ServerName cloud.mydomain.com
    <IfModule mod_headers.c>
      Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains"
    </IfModule>
 </VirtualHost>

Additional troubleshooting information

The Nextcloud instance is fully accessible and functional over LAN.
By disabling the HTTPS redirect, I’ve been able to confirm that my Nextcloud instance is reachable from my domain over unecrypted HTTP. Therefore, the DDNS service seems to be set up properly. I presume neither http nor https would work for cloud.mydomain.com if it were setup incorrectly.
For testing purposes I enabled a universal SELinux httpd rule for the entire /var/www/html/nextcloud/ folder to no avail.
I’ve refreshed the Let’s Encrypt certificate and even had an entirely new one created for cloud.mydomain.com, but it did not fix the issue.

bb77 · April 2, 2022, 7:58am

Hi @pim

If you redirect everything to another VirtualHost, you also need to move the configuration to the corresponding VirtualHost… Try it like this:

<VirtualHost *:80>
  DocumentRoot /var/www/html/nextcloud/
  ServerName cloud.mydomain.com
  Redirect permanent / https://cloud.mydomain.com/
</VirtualHost>

<VirtualHost *:443>
  ServerName cloud.mydomain.com
  DocumentRoot /var/www/html/nextcloud/
  
  <Directory /var/www/html/nextcloud/>
    Require all granted
    AllowOverride All
    Options FollowSymLinks MultiViews

    <IfModule mod_dav.c>
      Dav off
    </IfModule>

  </Directory>
  
 <IfModule mod_headers.c>
      Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains"
 </IfModule>
 </VirtualHost>

bb77 · April 2, 2022, 8:19am

Addition:

Your VirtualHost is also missing the entire SSL config. How exactly did you get the certificates?

pim · April 2, 2022, 8:39am

@bb77 Thank you for the swift reply.

I have updated my apache configuration as per your suggestion. In my opinion, it was not clear from the documentation that it was supposed to set up in this way as it just suggested to add the VirtualHost block for port 443 as in my original nextcloud.conf. Anyways, I fixed it now and the result is below.

I requested the certificate using certbot --apache for only the subdomain cloud.mydomain.com.

I just ran certbot --apache again and selected “Attempt to reinstall this existing certificate” and the SSL settings now seem to have been added to nextcloud.conf, it now looks like this:

<VirtualHost *:80>
  DocumentRoot /var/www/html/nextcloud/
  ServerName cloud.mydomain.com
  Redirect permanent / https://cloud.mydomain.com/
  RewriteEngine on
  RewriteCond %{SERVER_NAME} =cloud.mydomain.com
  RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

<VirtualHost *:443>
  ServerName cloud.mydomain.com
  DocumentRoot /var/www/html/nextcloud/

  <Directory /var/www/html/nextcloud/>
    Require all granted
    AllowOverride All
    Options FollowSymLinks MultiViews

    <IfModule mod_dav.c>
      Dav off
    </IfModule>

  </Directory>

 <IfModule mod_headers.c>
      Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains"
 </IfModule>
  SSLCertificateFile /etc/letsencrypt/live/cloud.mydomain.com/fullchain.pem
  SSLCertificateKeyFile /etc/letsencrypt/live/cloud.mydomain.com/privkey.pem
  Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>

I just ran sudo systemctl restart httpd.service and accessed https://cloud.mydomain.com/ through a browser which does not store any cookies. However, there is still no response from the server over HTTPS.

bb77 · April 2, 2022, 8:44am

Try removing the following line from the first VirtualHost…

Redirect permanent / https://cloud.mydomain.com/

…and restart the httpd service.

This line is not needed anymore because Cerbot added it’s own rewrite rule…

pim · April 2, 2022, 9:03am

Oops, yeah it seems like that. I removed the redundant line, but this did not resolve the issue.

I forgot to mention in the original post, but before you wonder, yes I also triple checked firewall and port forwarding rules, those are not the issue.

Is there any relevant logs for apache that might be useful? I have the following logfiles in /var/log/httpd: access_log, error_log, ssl_access_log, ssl_error_log, ssl_request_log.

ssl_access_log and ssl_request_log are empty (to be expected, I guess). ssl_error_log contains the following warning:

[Sat Apr 02 11:00:50.432118 2022] [ssl:warn] [pid 18683:tid 140475901507904] AH01909: xxxx:0 server certificate does NOT include an ID which matches the server name

bb77 · April 2, 2022, 9:57am

Not a 100% sure but it could have something to do with the hostname of your server. Make sure the contents of your /etc/hosts file look something like this:

127.0.0.1	localhost
127.0.1.1	cloud.yourdomain.com	cloud
192.168.0.10	cloud.yourdomain.net	cloud

Also change the contents of /etc/hostname to…

cloud

You could also try to add the ServerName directive to the global httpd.conf file. But I don’t think that’s necessary.

After you made these changes restart the httpd service and execute the following…

sudo apachectl configtest

and / or

sudo systemctl status httpd

…to check for any errors in the configuration.

If you still get errors you could also try to start over:

Remove all nextcloud.conf or nextcloud-le-ssl.conf files in /etc/httpd/conf.d/
Remove already issued certificates:

rm -rf /etc/letsencrypt/live/${DOMAIN}
rm -rf /etc/letsencrypt/renewal/${DOMAIN}.conf
rm -rf /etc/letsencrypt/archive/${DOMAIN}

Recreate the nextcloud.conf file with the following content…

<VirtualHost *:80>
  DocumentRoot /var/www/html/nextcloud/
  ServerName  cloud.yourdomain.com

  <Directory /var/www/html/nextcloud/>
    Require all granted
    AllowOverride All
    Options FollowSymLinks MultiViews

    <IfModule mod_dav.c>
      Dav off
    </IfModule>

  </Directory>
</VirtualHost>

run certbot again with the following command:

certbot --apache --rsa-key-size 4096 --agree-tos --no-eff-email --redirect

# Enter email address (used for urgent renewal and security notices) (Enter 'c' to cancel):

Enter a valid Email address!

# Which names would you like to activate HTTPS for?
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# 1: cloud.yourdomain.com
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Select the appropriate numbers separated by commas and/or spaces, or leave input
# blank to select all options shown (Enter 'c' to cancel):

Press ENTER!

Certbot should then automatically create a new VirtualHost for HTTPS. Most likely it will put the config for the new VirtualHost to an additional config file, which wil be named nextcloud-le-ssl.conf, to which you then can add the Header always set Strict-Transport-Security directive.

If such a file already exists, remove it before you start certbot!

pim · April 2, 2022, 12:02pm

Thank you for the very detailed reply.

I’m sorry to say that the issue did turn out to be the firewall… a firewall I wasn’t even aware was running. Upon my installation of Nextcloud, I set up iptables as it should be, allowing the necessary ports. Now it turns out there was another package, firewalld, and upon reboot, the firewalld daemon launched, another firewall program which comes preinstalled on Rocky Linux. I was unaware of this programme even being installed, as I had done a minimal install. Disabling firewalld fixed the issue.

I’m very sorry for any inconvenience, but I must say I’ve learned a lot from your tips and now have a proper and clean configuration setup. Thanks!

Have a good weekend.

bb77 · April 2, 2022, 12:07pm

No worries, glad to hear that everything is up and running now.

thx, and same to you