This server has no working Internet connection, DNS_CNAME resolving with local domain name appended

In Security & setup warnings

This server has no working Internet connection: Multiple endpoints could not be reached. This means that some of the features like mounting external storage, notifications about updates or installation of third-party apps will not work. Accessing files remotely and sending of notification emails might not work, either. Establish a connection from this server to the Internet to enjoy all features.

Nextcloud version (eg, 20.0.5): 22.1.1 I am using the Nextcloud docker Version
Docker Host Operating system and version (eg, Ubuntu 20.04): Debian 10
PHP version (eg, 7.4): 8.0.10

The issue you are facing:

My Nextcloud docker install cannot resolve outside dns since the introduction of the dns pinning

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

  1. Update to NC 22.* from NC 21
  2. Attempt to open App store or go to Security & setup warnings
  3. No internet access available all urls resolve local addresses as shown in log OC\Http\Client\LocalAddressChecker->ThrowIfLocalIp("10.1.1.2")

Testing the ability for php to resolve www.edri.org it appears to resolve fine

root@e69a995b306c:/var/www/html# php -r "var_dump(dns_get_record('www.edri.org', DNS_A | DNS_AAAA));"
array(1) {
  [0]=>
  array(5) {
    ["host"]=>
    string(12) "www.edri.org"
    ["class"]=>
    string(2) "IN"
    ["ttl"]=>
    int(6779)
    ["type"]=>
    string(1) "A"
    ["ip"]=>
    string(12) "45.66.33.123"
  }
}

Commenting out $stack->push($this->dnsPinMiddleware->addDnsPinning()); in lib/private/Http/Client/ClientService.php as per this comment it works around the issue, however this obviously reverts on upgrades.

Nextcloud log:

[internet_connection_check] Error: Cannot connect to: www.edri.org

GET /settings/ajax/checksetup
from **redacted** by dugite-code at 2021-09-01T10:07:52+08:00

Raw log

{"reqId":"uFcytnDuDnh82MNieAKb","level":3,"time":"2021-09-01T10:07:52+08:00","remoteAddr":"**redacted**","user":"james.knight","app":"internet_connection_check","method":"GET","url":"/settings/ajax/checksetup","message":"Cannot connect to: www.edri.org","userAgent":"Mozilla/5.0 (Windows NT 10.0; rv:91.0) Gecko/20100101 Firefox/91.0","version":"22.1.1.2","exception":{"Exception":"OCP\\Http\\Client\\LocalServerException","Message":"Host violates local access rules","Code":0,"Trace":[{"file":"/var/www/html/lib/private/Http/Client/DnsPinMiddleware.php","line":136,"function":"ThrowIfLocalIp","class":"OC\\Http\\Client\\LocalAddressChecker","type":"->","args":["10.1.1.2"]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php","line":35,"function":"OC\\Http\\Client\\{closure}","class":"OC\\Http\\Client\\DnsPinMiddleware","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Middleware.php","line":31,"function":"__invoke","class":"GuzzleHttp\\PrepareBodyMiddleware","type":"->","args":["*** sensitive parameter replaced ***","*** sensitive parameter replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/RedirectMiddleware.php","line":71,"function":"GuzzleHttp\\{closure}","class":"GuzzleHttp\\Middleware","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Middleware.php","line":63,"function":"__invoke","class":"GuzzleHttp\\RedirectMiddleware","type":"->","args":["*** sensitive parameter replaced ***","*** sensitive parameter replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/HandlerStack.php","line":75,"function":"GuzzleHttp\\{closure}","class":"GuzzleHttp\\Middleware","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Client.php","line":331,"function":"__invoke","class":"GuzzleHttp\\HandlerStack","type":"->","args":["*** sensitive parameter replaced ***","*** sensitive parameter replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Client.php","line":168,"function":"transfer","class":"GuzzleHttp\\Client","type":"->","args":["*** sensitive parameter replaced ***","*** sensitive parameter replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Client.php","line":187,"function":"requestAsync","class":"GuzzleHttp\\Client","type":"->","args":["get",{"__class__":"GuzzleHttp\\Psr7\\Uri"},{"verify":"/var/www/html/resources/config/ca-bundle.crt","timeout":30,"allow_redirects":{"on_redirect":{"__class__":"Closure"}},"nextcloud":{"allow_local_address":false},"synchronous":true,"handler":{"__class__":"GuzzleHttp\\HandlerStack"},"http_errors":true,"decode_content":true,"cookies":false,"idn_conversion":false,"_conditional":{"User-Agent":"GuzzleHttp/7"}}]},{"file":"/var/www/html/lib/private/Http/Client/Client.php","line":223,"function":"request","class":"GuzzleHttp\\Client","type":"->","args":["get","http://www.edri.org/",{"verify":"/var/www/html/resources/config/ca-bundle.crt","timeout":30,"allow_redirects":{"on_redirect":{"__class__":"Closure"}},"nextcloud":{"allow_local_address":false},"headers":{"User-Agent":"Nextcloud Server Crawler","Accept-Encoding":"gzip"},"synchronous":true}]},{"file":"/var/www/html/apps/settings/lib/Controller/CheckSetupController.php","line":179,"function":"get","class":"OC\\Http\\Client\\Client","type":"->","args":["http://www.edri.org/"]},{"file":"/var/www/html/apps/settings/lib/Controller/CheckSetupController.php","line":162,"function":"isSiteReachable","class":"OCA\\Settings\\Controller\\CheckSetupController","type":"->","args":["www.edri.org"]},{"file":"/var/www/html/apps/settings/lib/Controller/CheckSetupController.php","line":742,"function":"hasInternetConnectivityProblems","class":"OCA\\Settings\\Controller\\CheckSetupController","type":"->","args":[]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":217,"function":"check","class":"OCA\\Settings\\Controller\\CheckSetupController","type":"->","args":[]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":126,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Settings\\Controller\\CheckSetupController"},"check"]},{"file":"/var/www/html/lib/private/AppFramework/App.php","line":156,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Settings\\Controller\\CheckSetupController"},"check"]},{"file":"/var/www/html/lib/private/Route/Router.php","line":301,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["OCA\\Settings\\Controller\\CheckSetupController","check",{"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},{"_route":"settings.CheckSetup.check"}]},{"file":"/var/www/html/lib/base.php","line":1000,"function":"match","class":"OC\\Route\\Router","type":"->","args":["/settings/ajax/checksetup"]},{"file":"/var/www/html/index.php","line":36,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"/var/www/html/lib/private/Http/Client/LocalAddressChecker.php","Line":42,"CustomMessage":"Cannot connect to: www.edri.org"},"id":"612ee2d55a8dd"}

docker-compose.yaml

version: '2'

services:
  mdb:
    image: mariadb
    command: --log-bin=mysqld-bin --transaction-isolation=READ-COMMITTED --binlog-format=ROW --skip-innodb-read-only-compressed
    restart: always
    volumes:
      - mdb:/var/lib/mysql
      - ./conf.d:/etc/mysql/conf.d
    env_file:
      - db.env
    networks:
      - nextcloud_net

  app:
    image: nextcloud:apache
    restart: always
    ports:
      - 127.0.0.1:8066:80
    volumes:
      - nextcloud:/var/www/html
      - /opt/nextcloud/php.ini:/usr/local/etc/php/conf.d/zzz-custom.ini
    depends_on:
      - mdb
      - redis
    dns:
      - 10.1.1.1
      - 127.0.0.1
    networks:
      - nextcloud_net

  notify_push:
    container_name: notify_push
    image: nextcloud:apache
    restart: always
    networks:
      nextcloud_net:
    ports:
      - 127.0.0.1:7867:7867
    environment:
      - PORT=7867
      - NEXTCLOUD_URL=https://example.tld/
    volumes:
       - nextcloud:/var/www/html:ro
    entrypoint: /var/www/html/custom_apps/notify_push/bin/x86_64/notify_push /var/www/html/config/config.php
    depends_on:
      - mdb
      - app
      - redis

  redis:
    image: redis:alpine
    restart: always
    networks:
      - nextcloud_net

volumes:
  mdb:
  nextcloud:

networks:
  nextcloud_net:

config.php:

<?php
$CONFIG = array (
  'loglevel' => 0,
  'logtimezone' => 'Australia/Perth',
  'instanceid' => '**redacted**',
  'passwordsalt' => '**redacted**',
  'secret' => '**redacted**',
  'default_phone_region' => 'AU',
  'trusted_domains' =>
  array (
    0 => 'example.tld',
    1 => '127.0.0.1',
  ),
  'trusted_proxies' =>
  array (
    1 => '10.0.0.0/8',
  ),
  'overwrite.cli.url' => 'https://example.tld',
  'overwritehost' => 'example.tld',
  'overwriteprotocol' => 'https',
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'mysql',
  'dbhost' => 'mdb:3306',
  'dbname' => 'nextcloud',
  'dbuser' => 'nextcloud',
  'dbpassword' => '**redacted**',
  'dbtableprefix' => 'oc_',
  'version' => '22.1.1.2',
  'installed' => true,
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' =>
  array (
    'host' => 'redis',
    'port' => '6379',
  ),
  'apps_paths' =>
  array (
    0 =>
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 =>
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'preview_max_x' => '2048',
  'enabledPreviewProviders' =>
  array (
    0 => 'OC\\Preview\\PNG',
    1 => 'OC\\Preview\\JPEG',
    2 => 'OC\\Preview\\GIF',
    3 => 'OC\\Preview\\BMP',
    4 => 'OC\\Preview\\XBitmap',
    5 => 'OC\\Preview\\MP3',
    6 => 'OC\\Preview\\TXT',
    7 => 'OC\\Preview\\MarkDown',
    8 => 'OC\\Preview\\SVG',
    9 => 'OC\\Preview\\TIFF',
  ),
  'updater.release.channel' => 'stable',
  'app_install_overwrite' =>
  array (
    0 => 'files_readmemd',
    1 => 'jsloader',
    2 => 'phonetrack',
    3 => 'cookbook',
    4 => 'side_menu',
    5 => 'apporder',
    6 => 'maps',
    7 => 'contacts',
    8 => 'bookmarks',
    9 => 'tasks',
    10 => 'previewgenerator',
  ),
 'preview_max_y' => '2048',
  'jpeg_quality' => '60',
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'tls',
  'mail_sendmailmode' => 'smtp',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'mail.example.tld',
  'mail_from_address' => 'nextcloud',
  'mail_domain' => 'example.tld',
  'mail_smtpname' => 'nextcloud',
  'mail_smtppassword' => '**redacted**',
  'mail_smtpport' => '587',
  'mysql.utf8mb4' => true,
);

After some more investigation it appears the DNS_CNAME result (I didn’t check before for some reason) is getting my domain name appended to it. This is obviously why it’s returning my local IP causing the ThrowIfLocalIp issue. I am unsure what would be causing this issue though, I am running my local dns from my pfsense firewall if that helps.

root@aa114ba07de6:/var/www/html# php -r "var_dump(dns_get_record('www.edri.org', DNS_A | DNS_AAAA| DNS_CNAME));"
array(2) {
  [0]=>
  array(5) {
    ["host"]=>
    string(12) "www.edri.org"
    ["class"]=>
    string(2) "IN"
    ["ttl"]=>
    int(10800)
    ["type"]=>
    string(1) "A"
    ["ip"]=>
    string(12) "45.66.33.123"
  }
  [1]=>
  array(5) {
    ["host"]=>
    string(26) "www.edri.org.example.tld"
    ["class"]=>
    string(2) "IN"
    ["ttl"]=>
    int(300)
    ["type"]=>
    string(5) "CNAME"
    ["target"]=>
    string(13) "example.tld"
  }
}

Ok so some further debugging shows this is a DNS resolver issue. In Unbound there is a setting called local-zone type, in pfsense it is set to Transparent by default.

This is the default behavior. If the query is for a name that does not exist locally, it is resolved as usual. If the name has a local match but the type is different, a NOERROR, NODATA response is sent to the client

This appears to be what’s appending the local domain name to the cname query. Setting it to static appears to solve the issue, however I’m not to sure, if anyone knows the answer (I know it’s not really a Nextcloud issue after all) I would appreciate the hints.

Static

Returns a NODATA or NXDOMAIN response to the client.

Edit: further reading: https://forum.netgate.com/topic/151192/confused-about-dns-forwarding-and-local-domains/5

Further update,

It turns out using my domain in pfsense wasn’t a great idea. Changing the pfsense domain to pfsense.example.arpa rather than pfsense.example.tld and updating my servers /etc/resolv.conf has finally allowed me to set the local-zone type back to Transparent as it should be.

/etc/resolv.conf:

domain example.arpa
search example.arpa
nameserver 10.1.1.1

Edit: I also had to adjust the DHCP domain and search names and pfsense was issuing the incorrect settings for some reason.