Nextcloud site inaccessable from LAN using domain but SSH works using domain from LAN

ℹ️ Support
1

Nextcloud version: 25.0.2
Operating system and version: Debian 11
Apache or nginx version: Apache/2.4.54
PHP version: 7.4

The issue you are facing:
SSH works from within the instance’s LAN using its published domain name.
Nextcloud site/client never resolves (times out) from within the instance’s LAN using its domain name.
Nextcloud is accessible using the server’s LAN ip.
Everything works as expected from outside the LAN.

This is a new issue and I’m at a loss to figure out what updated or changed. This issue is affecting another, older nextcloud instance as well. What these two instances have in common is that they both use the same model router. However, all ports are correctly forwarded (this never changed from when it did work) but, because SSH can be logged into using the domain name, it doesn’t seem like this is a router issue.

A workaround is to use a vpn or its local ip address to load the nextcloud page or client but this is unnecessary for SSH. I’m at a complete loss. config.php was never changed but perhaps an apache update did this?

Is this the first time you’ve seen this error? (Y/N): yes, though it’s more of a mystery than an error

No errors are generated in any log, no logs even change. The browser and client just time out, as if they are simply ignored. The apache logs don’t see the attempted access and no error is generated anywhere.

My config.php:

<?php
$CONFIG = array (
.
.
.
  'trusted_domains' => 
  array (
    0 => '192.168.1.*',
    1 => '------.ddns.net',
  ),
  'datadirectory' => '/mnt/nc_a/nc_data',
  'dbtype' => 'mysql',
  'version' => '25.0.2.3',
  'overwrite.cli.url' => 'http://---------.ddns.net',
  'htaccess.RewriteBase' => '/',
  'dbname' => 'nc_------',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => '---',
  'dbpassword' => '-------',
  'installed' => true,
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'filelocking.enabled' => true,
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => 'localhost',
    'port' => 6379,
  ),
  'maintenance' => false,
  'default_phone_region' => 'US',
  'mail_from_address' => '---------',
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'mail_domain' => 'gmail.com',
  'mail_smtpsecure' => 'ssl',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'smtp.gmail.com',
  'mail_smtpport' => '465',
  'mail_smtpname' => '---------',
  'mail_smtppassword' => '------------------',
);
1 Like
2

Are you running local DNS?

It’s possible the issue could be with the web server configuration. It might help to post that as well and explain more about your setup.

3

Hi,

I faced an issue that is similar to yours.
It was a firmware update from my IPS that changed something in the routing logic but only when it comes from the LAN and it did not affect the WAN connections.

I can suggest:

1.Verify if there is an HTTP or HTTPS port open for your router on the port facing WAN and try to disable it.
2. Just in case: are you using HTTPS and not HTTP ? (In case there is no port redirection).

Workaround:
Use split DNS. Use your internal DNS to resolve your public DN to your local nextcloud private IP.

4

Not running a local DNS. The server is a virtual host, current site-enabled:

<VirtualHost *:80>
    RewriteEngine On
    RewriteRule ^(.*)$ https://%{HTTP_HOST}$1 [R=301,L]
RewriteCond %{SERVER_NAME} =———.ddns.net
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

<VirtualHost *:443>

         ServerName ———.ddns.net
         ServerAdmin admin@———.ddns.net
         DocumentRoot /var/www/nextcloud

         <Directory /var/www/nextcloud/>
                Require all granted
                AllowOverride All
                Options FollowSymlinks MultiViews

                <IfModule mod_dav.c>
                        Dav off
                </IfModule>

         Satisfy Any
         
         </Directory>


    # enable HTTP/2, if available
    Protocols h2 http/1.1

    # HTTP Strict Transport Security (mod_headers is required) (63072000 seconds)
    Header always set Strict-Transport-Security "max-age=63072000"


SSLCertificateFile /etc/letsencrypt/live/———.ddns.net/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/———.ddns.net/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>

## modern configuration
SSLProtocol             all -SSLv3 -TLSv1 -TLSv1.1 -TLSv1.2
SSLHonorCipherOrder     off
SSLSessionTickets       off

SSLUseStapling On
SSLStaplingCache "shmcb:logs/ssl_stapling(32768)"

Current enabled apache2 modules:

Loaded Modules:
 core_module (static)
 so_module (static)
 watchdog_module (static)
 http_module (static)
 log_config_module (static)
 logio_module (static)
 version_module (static)
 unixd_module (static)
 access_compat_module (shared)
 alias_module (shared)
 auth_basic_module (shared)
 authn_core_module (shared)
 authn_file_module (shared)
 authz_core_module (shared)
 authz_host_module (shared)
 authz_unixgroup_module (shared)
 authz_user_module (shared)
 autoindex_module (shared)
 deflate_module (shared)
 dir_module (shared)
 env_module (shared)
 filter_module (shared)
 headers_module (shared)
 http2_module (shared)
 mime_module (shared)
 mpm_event_module (shared)
 negotiation_module (shared)
 proxy_module (shared)
 proxy_fcgi_module (shared)
 reqtimeout_module (shared)
 rewrite_module (shared)
 setenvif_module (shared)
 socache_shmcb_module (shared)
 ssl_module (shared)
 status_module (shared)

systemctl status apache2 output:

● apache2.service - The Apache HTTP Server
     Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2022-12-24 02:05:44 AKST; 8h ago
       Docs: https://httpd.apache.org/docs/2.4/
    Process: 20010 ExecStart=/usr/sbin/apachectl start (code=exited, status=0/SUCCESS)
   Main PID: 20024 (apache2)
      Tasks: 105 (limit: 4224)
        CPU: 2min 19.734s
     CGroup: /system.slice/apache2.service
             ├─20024 /usr/sbin/apache2 -k start
             ├─20027 /usr/sbin/apache2 -k start
             └─20028 /usr/sbin/apache2 -k start

Dec 24 02:05:43 ———.ddns.net systemd[1]: Starting The Apache HTTP Server...
Dec 24 02:05:44 ———.ddns.net systemd[1]: Started The Apache HTTP Server.

Please let me know if any other details of this config would be helpful to look at.

5

I know that there was a firmware update pushed to the router at some point. This has been where I was primarily focused until I realized that SSH had no issues. However, it just occurred to me that sshd is listening on a non-standard port, and that port is the one the router accepts and forwards traffic from.

I will try your suggestions. To your #2, both HTTP and HTTPS are listening for traffic and the HTTP is redirected to HTTPS at the server level.

DNS is not my forte (a lot of this isn’t) but I may go that route!

6

Split-horizon DNS is the “real” way to fix some problems like this. In essence what you do is override the FQDN so it points to the LAN IP when the client is also on the LAN.

Without doing that, you have something called hairpin routing or NAT reflection where your router has to NAT and reroute the traffic back in as it’s leaving. This isn’t a good practice from a networking perspective, and many routers either don’t allow it or have problems doing it.

I probably talk about it once a week on this forum for the past several years.

I don’t know for sure that’s your issue, but it’s worth investigating anyway. Try adding the FQDN to your hosts file with the LAN IP and see if that fixes it. Then you know.

7

I did try adding the FQDN to the /etc/hosts, with the LAN ip before (probably after reading one of your posts) but there was no change in behavior. I tried again just now to be sure but the page still just times out.

I had focused hairpin routing but realized that SSH was having no issues. Wouldn’t it too be affected? This must be a DNS issue though, right? This is why I’m so confused.

I was thinking about throwing a pi-hole on the network which I imagine would be the ultimate workaround.

8

The only thing DNS is doing in this case is resolving the name to the WAN IP. If the IP connection is good (since SSH works) then what else could make a difference? The web server’s vhost config is one thing. A web server can be configured to respond differently (or not at all) based on the IP. I don’t see anything like that in what you posted though.

Can you run tcpdump on your server and see if the network packets are getting to it when you try HTTPS from your LAN? That would rule out any issue with the router.

9

I’m trying to add some details to @KarlF12 great response :
After the DNS resolve your DN to your public IP (which is already validated by your ssh), then the next protocol kicks in : HTTP(S) or SSH to simplify.
The DNS response works the same way for both.

My guess is : your problem is more attached to hairpin routing and local traffic (or only local traffic depending on your router software) that will thing the HTTP(S) is for him.

As I said, I already saw this once on a client LAN.
The http (no S) from his lan (without split DNS) was intercepted by the internal router management interface but the HTTPS was reflecting to his public server since the router management interface was only receiving HTTP (no S).
So my guess his : since your router thinks it’s for is own interface since the HTTPS comes from inside.
But requests that arrive by public interface are redirected to NAT since there is no management interface on that interface.

That’s why I suggested split DNS or removing the management interface if possible.

PS : Sorry I modified this message a lot of time…

10

I need to ponder this a bit. Thank you!

11

Two tcpdump snippets the first of which is from the LAN as the page tries to load. But I think most of the packets are the SSH connection. The second one I’m using a VPN proxy.
6544 is the SSH listen port.

Inside LAN:

14:04:29.594322 IP ————.ddns.net:443.6544 > 142-26-237-24.gci.net.43172: Flags [P.], seq 2564133000:2564133116, ack 794350105, win 501, options [nop,nop,TS val 2844168572 ecr 3913578953], length 116
14:04:29.594454 IP ————.ddns.net:443.6544 > 142-26-237-24.gci.net.43172: Flags [P.], seq 116:232, ack 1, win 501, options [nop,nop,TS val 2844168573 ecr 3913578953], length 116
14:04:29.596844 IP ————.ddns.net:443.33753 > dns.google.domain: 63079+ PTR? 142.26.237.24.in-addr.arpa. (44)
14:04:29.599066 IP 142-26-237-24.gci.net.43172 > ————.ddns.net:443.6544: Flags [.], ack 116, win 928, options [nop,nop,TS val 3913579648 ecr 2844168572], length 0
14:04:29.599073 IP 142-26-237-24.gci.net.43172 > ————.ddns.net:443.6544: Flags [.], ack 232, win 928, options [nop,nop,TS val 3913579648 ecr 2844168573], length 0
14:04:29.723384 IP dns.google.domain > ————.ddns.net:443.33753: 63079 1/0/0 PTR 142-26-237-24.gci.net. (79)
14:04:29.723911 IP ————.ddns.net:443.6544 > 142-26-237-24.gci.net.43172: Flags [P.], seq 232:476, ack 1, win 501, options [nop,nop,TS val 2844168702 ecr 3913579648], length 244
14:04:29.724012 IP ————.ddns.net:443.6544 > 142-26-237-24.gci.net.43172: Flags [P.], seq 476:704, ack 1, win 501, options [nop,nop,TS val 2844168702 ecr 3913579648], length 228
14:04:29.724122 IP ————.ddns.net:443.45487 > dns.google.domain: 17265+ PTR? 8.8.8.8.in-addr.arpa. (38)
14:04:29.756349 IP ————.ddns.net:443.6544 > 142-26-237-24.gci.net.43172: Flags [P.], seq 704:852, ack 1, win 501, options [nop,nop,TS val 2844168734 ecr 3913579775], length 148
14:04:29.756459 IP ————.ddns.net:443.6544 > 142-26-237-24.gci.net.43172: Flags [P.], seq 852:1064, ack 1, win 501, options [nop,nop,TS val 2844168735 ecr 3913579775], length 212
14:04:29.756561 IP ————.ddns.net:443.6544 > 142-26-237-24.gci.net.43172: Flags [P.], seq 1064:1276, ack 1, win 501, options [nop,nop,TS val 2844168735 ecr 3913579775], length 212
14:04:29.756663 IP ————.ddns.net:443.6544 > 142-26-237-24.gci.net.43172: Flags [P.], seq 1276:1424, ack 1, win 501, options [nop,nop,TS val 2844168735 ecr 3913579775], length 148
14:04:29.756770 IP ————.ddns.net:443.6544 > 142-26-237-24.gci.net.43172: Flags [P.], seq 1424:1652, ack 1, win 501, options [nop,nop,TS val 2844168735 ecr 3913579775], length 228
14:04:29.756873 IP ————.ddns.net:443.6544 > 142-26-237-24.gci.net.43172: Flags [P.], seq 1652:1880, ack 1, win 501, options [nop,nop,TS val 2844168735 ecr 3913579775], length 228
14:04:29.756975 IP ————.ddns.net:443.6544 > 142-26-237-24.gci.net.43172: Flags [P.], seq 1880:2028, ack 1, win 501, options [nop,nop,TS val 2844168735 ecr 3913579775], length 148

Through a VPN:

13:59:31.931454 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23795112:23795340, ack 14545, win 501, options [nop,nop,TS val 3060547419 ecr 2705549576], length 228
13:59:31.931512 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23795340:23795760, ack 14545, win 501, options [nop,nop,TS val 3060547419 ecr 2705549576], length 420
13:59:31.931557 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23795760:23795988, ack 14545, win 501, options [nop,nop,TS val 3060547419 ecr 2705549576], length 228
13:59:31.931631 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23795988:23796216, ack 14545, win 501, options [nop,nop,TS val 3060547419 ecr 2705549576], length 228
13:59:31.931676 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23796216:23796444, ack 14545, win 501, options [nop,nop,TS val 3060547419 ecr 2705549576], length 228
13:59:31.931750 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23796444:23796672, ack 14545, win 501, options [nop,nop,TS val 3060547419 ecr 2705549576], length 228
13:59:31.931797 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23796672:23796900, ack 14545, win 501, options [nop,nop,TS val 3060547420 ecr 2705549576], length 228
13:59:31.931872 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23796900:23797128, ack 14545, win 501, options [nop,nop,TS val 3060547420 ecr 2705549576], length 228
13:59:31.931916 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23797128:23797356, ack 14545, win 501, options [nop,nop,TS val 3060547420 ecr 2705549576], length 228
13:59:31.931990 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23797356:23797584, ack 14545, win 501, options [nop,nop,TS val 3060547420 ecr 2705549576], length 228
13:59:31.932035 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23797584:23797812, ack 14545, win 501, options [nop,nop,TS val 3060547420 ecr 2705549576], length 228
13:59:31.932108 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23797812:23798040, ack 14545, win 501, options [nop,nop,TS val 3060547420 ecr 2705549576], length 228
13:59:31.932153 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23798040:23798268, ack 14545, win 501, options [nop,nop,TS val 3060547420 ecr 2705549576], length 228
13:59:31.932227 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23798268:23798496, ack 14545, win 501, options [nop,nop,TS val 3060547420 ecr 2705549576], length 228
13:59:31.932273 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23798496:23798724, ack 14545, win 501, options [nop,nop,TS val 3060547420 ecr 2705549576], length 228
13:59:31.932313 IP 193.29.61.182.33012 > ————.ddns.net:443.6544: Flags [P.], seq 14545:14581, ack 23515288, win 6092, options [nop,nop,TS val 2705549578 ecr 3060547357], length 36
13:59:31.932355 IP ————.ddns.net:443.6544 > 193.29.61.182.33012: Flags [P.], seq 23798724:23798952, ack 14581, win 501, options [nop,nop,TS val 3060547420 ecr 2705549578], length 228
12

I don’t see any LAN traffic on the LAN capture. Looks to me like it’s all Nextcloud traffic from the internet. You should use a filter to just see traffic from that one client. For example tcpdump -f 'host 192.168.1.100 and tcp port 443'

It’s possible that hairpin routing may cause it to show your own WAN IP as the client. It depends on the router’s behavior.

13

I did run a filtered tcpdump as you recommend and, using the domain name, nothing happens. Just the typical time out.

When I use the hots’s LAN IP to access the vhost there is traffic:

12:19:27.106246 IP 192.168.1.171.39620 > ————.ddns.net:443.https: Flags [P.], seq 1878:1917, ack 13988, win 501, options [nop,nop,TS val 2621136892 ecr 3698310690], length 39
12:19:27.106292 IP ————.ddns.net:443.https > 192.168.1.171.39620: Flags [.], ack 1917, win 501, options [nop,nop,TS val 3698310693 ecr 2621136892], length 0

Running tcpdump for the SSH port, using the domain name, does show my WAN IP as the SSH client.

14

As well, running traceroute using the router’s diagnostics page, seems to have no issues with hairpinning (I know that’s not ideal) which makes sense, since SSH works fine:

traceroute to ——————.ddns.net (24.xxx.xx.142), 30 hops max, 38 byte packets
1 142-xx-xxx-24.gci.net (24.xxx.xx.142) 0.022 ms 0.023 ms 0.100 ms
=============Complete==============

To @ligal point above, if I’m reading it correctly, I can’t find how this router might be trying to capture HTTPS traffic for itself rather than forwarding it to my nextcloud host. Of course this only fails with hairpinning HTTPS traffic, not in any other case.

15

So when you use the DNS name with HTTPS, the connection never reaches the server. So the problem is upstream from the server somewhere. I think it has to be the router.

If the router uses port 443 for its own web interface, it could be in some way that’s messing up the NAT reflection.

Just to verify, can you forward some other port that isn’t used anywhere (for example 8443) to port 443 on your server and see if you can get to it?

16

Well, thank you. Unfortunately that does it. So what’s happening is, because the request is coming from inside the LAN, the router tries to redirect traffic to itself (this is what ligal had reasoned, I believe); but from outside it just forwards packets as it should. Makes sense, for this issue.

So, short of finding a router setting which may not exist, I’ll either need to always tack on a port number or just use the same workarounds I’ve already figured out?

In any case, thank you very much, and @ligal , for helping me to understand what’s going on!

17

A lot of routers have an option to change the port of their admin interface.

18

Yeah. This one did at one point as well. A firmware push a few months ago not only seems to have nulled Remote Management, and port granularity, but also the functionality of the DDNS client which is now just broken.

Oh well.

Thanks much for your help. I’ve learned a lot.

1 Like