What exactly does "Checking your server..." do? (Server cannot reach itself via FQDN.)

The Basics

  • Nextcloud Server version (e.g., 29.x.x):
    • 32.0.5
  • Operating system and version (e.g., Ubuntu 24.04):
    • cat /etc/debian_version gives: 12.11
  • Web server and version (e.g, Apache 2.4.25):
    • apache2 -v gives: Apache/2.4.62 (Debian)
  • PHP version (e.g, 8.3):
    • php -v gives: PHP 8.2.28

Log entries

Nextcloud

The downloaded log from Administration settings->Logging screen is HUGE. Essentially there are several

cURL error Could not resolve host:

and then followed by some

cURL error Failed to connect

Web server

Debian with apache2, Nextcloud installed natively. (No docker etc)

Server in a DMZ, behind a firewall (IPfire with the NC local IP in the hosts file); NAT configured to forward HTTP and HTTPS to the server in the DMZ. WebAccess, WebDAV, Bookmark sync, Desktop clients in various PCs, etc. all working just fine from the Internet and LAN (Green network in IPfire terms). No known problems from the user-side.

Nextcloud

The output of sudo -u www-data php occ config:list system :

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "www.myserver.com",
            "192.168.1.1"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "32.0.5.0",
        "overwrite.cli.url": "https:\/\/www.myserver.com",
        "overwriteprotocol": "https",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "maintenance_window_start": 1,
        "default_phone_region": "ch",
        "maintenance": false,
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpauth": true,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "theme": "",
        "auth.webauthn.enabled": false,
        "loglevel": 0,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "updater.release.channel": "stable",
        "app_install_overwrite": [],
        "mail_smtpsecure": "ssl"
    }
}

Summary of the issue:

Not sure whether it appeared earlier, but I noticed it during the update 31.0.5 > 31.0.13 > 32.0.5.

In the admin interface > Administrative Settings > Overview. When the “Checking your server …" is done I get several warnings/errors that the “Web server can connect to itself”.

So I tried to investigate what is wrong. Somehow it seems that the server cannot always reach itself, so I setup a watch command on the servers CMD-line to see whats going on:

watch “curl -I https://www.server.com/nextcloud/index.php/heartbeat”

Now, after I open the Overview (which triggers the server “to check itself", the above curl command - reproducibly - suddenly gives a “Failed to connect" (basically a 404).

However, after a while (~10minutes ?) the curl command again works fine (I get code 200 and the header).

So, then it looks again all good. The server can reach itself using its FQDN, and one may assume that the “selfcheck" should also work just fine. BUT, as soon as I open the Overview again, which triggers the selfcheck again, the curl command suddenly gives a “Failed to connect" (= 404) again. Which then obviously leads to the self-check giving several errors/warnings due to being unable to reach itself via its FQDN.

So. First I though there’s something wrong with my firewall, or the NAT setup. But observing the situation, I get the feeling that the Nextcloud might be doing something odd. Maybe flooding the firewall, or some odd request? Because: How come, that the curl -I command is just fine (=the server can reach itself via its FQDN giving code 200), until the selfcheck is triggered???

Somehow the selfcheck leads to a situation where the server suddenly cannot reach itself anymore via its FQDN. And subsequently the self-check generates quite some output…

Hence, before I start to use wireshark an reverse engineer the whole procedure, I wonder if someone has in-depth-knowledge of the “self-check" and can tell me what exactly is happening there. I am pretty lost here, and I am afraid that the firewall-support says “that’s a nextcloud problem”, while the nextcould side may say “that’s a problem with your firewall”.

Hence, I try to understand both sides better to find out what it the actual root cause.

Any help would be greatly appreciated. If further infos are required, just ask (ideally let me know where or how to obtain).

Thanks in advance!

Hm, if it is the webserver of Nextcloud and you get a 404 error, you should see that in the access and/or error logs. If there is nothing, there is a chance that it is a proxy (if you are using one), normally if a firewall blocks a connection or the host is not reachable, you either run into a timeout or get a connection failed but not a 404 error.

There is a growing number of setup checks, I don’t know if there is a full list. Mostly it is configuration, e.g. if some parameters are set (maintenance window, caching servers), if file permissions of the code or the data folder are too vast, these “internal” test are probably not the issue, more the ones where it check e.g. if the data folder is directly accessible without authentication.

If you resolve the FQDN on your server, do you get the public IP or the local network ip? If on the command line you do a curl -v https://nextcloud.example.com/status.php ?

Hi tflidd, thanks for thinking about my problem.

tbh, it is not really a 404, my bad, it is a failed to connect:

curl: (7) Failed to connect to ``www.myserver.com`` port 443 after 29 ms: Couldn’t connect to server

I am not using a proxy, the NC (served by Apache2) is just sitting in a DMZ behind a firewall (IPfire), HTTP and HTTPS forwarded via NAT rules to the server.

And nslookup www.myserver.com does give back my public IP (= RED interface of the firewall), this always works. But as far as I understand, the problem “could be related" to something called NAT reflection, because when I run the curl command from the servers cmd-line, it needs to resolve the FQDN not as public IP, but it needs to get its local IP within the DMZ (i.e. 192.168.1.1). To address this, I added this local IP of the server to the hosts file of the firewall, mapping the FQDN to this local IP in the DMZ. This is apparently one method for ‘NAT reflection’, and it does apparently enable the server to run curl commands on itself using the FQDN. So far so good.

What is weird is the fact that this is "not stable” somehow.

This morning the remote shell on the server was still open and the command

watch “curl -I https://www.server.com/nextcloud/index.php/heartbeat”

properly returned the header. Great! So, I went back into the NC admin interface, loaded the Overview, and tadda: No errors, no warnings. Just two info notes. So all seems to be fine in fact. However, right after the Overview was loaded, the curl command again gave a Failed to connect as given above. And then, if I reload the Overview again, it takes a long time and then suddenly I get again several warnings (i.e. JavaScript source map support, JavaScript modules support, OCS provider resolving) referring to a “make sure that your Web server can connect to itself”. Matching the failed curl command. Until…. I think it’s something around 15-30mins… then the curl command works again, and then the first load of the Overview works again just fine, but somehow causing the curl command to fail again.

It’s really odd.