Nextcloud version: 17.0.2.1
Operating system and version: Solus, KDE Neon, Manjaro ARM, Android
Apache or nginx version: Nginx 1.16.1
PHP version: 7.1.33
The issue you are facing:
I have Nextcloud running in a FreeNAS jail, and route it through an Nginx proxy in another jail. For whatever reason, my server works fine from OUTSIDE my home network, but from INSIDE my network, it only works about 10% of the time, and the results are different each time.
Anything outside my network works just fine. Android app works, linux desktop sync client works, web browser works… everything.
But INSIDE my network, it’s a mess. Sometimes Nextcloud loads in the browser, other times it doesn’t (timeout). Sometimes it loads instantly in the browser, sometimes it takes 30+ seconds to load the page. The Nextcloud desktop client almost never works (rare, but it has). The Android app never works. Sometimes Nextcloud will load on one of my computers, but it won’t load on my other two computers. Then 10 minutes later, it will work on a different computer, but not the other two. Most of the time, when it works on LAN, it’s painfully slow (from seconds to several minutes to load a page).
This has been going on for months and I’ve spent countless hours troubleshooting this to no avail.
The output of your Nextcloud log in Admin > Logging:
Nothing here other than some update messages and ldap errors I already fixed
The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):
<?php
$CONFIG = array (
'apps_paths' =
array (
0 =
array (
'path' ='/usr/local/www/nextcloud/apps',
'url' ='/apps',
'writable' =true,
),
1 =
array (
'path' ='/usr/local/www/nextcloud/apps-pkg',
'url' ='/apps-pkg',
'writable' =true,
),
),
'logfile' ='/var/log/nextcloud/nextcloud.log',
'memcache.local' ='\\OC\\Memcache\\APCu',
'instanceid' ='ocoiwtfac9a',
'passwordsalt' ='NO SOUP FOR YOU',
'secret' ='NO SOUP FOR YOU',
'trusted_domains' =
array (
0 ='nextcloud.domain.xyz',
1 ='nc.domain.xyz',
2 ='192.168.1.215',
),
'forwarded_for_headers' =
array (
0 ='X-Forwarded-For',
1 ='HTTP_X_FORWARDED_FOR',
),
'datadirectory' ='/mnt/storage',
'dbtype' ='mysql',
'version' ='17.0.2.1',
'dbname' ='nextcloud',
'dbhost' ='localhost:5432',
'dbport' ='',
'dbtableprefix' ='oc_',
'dbuser' ='oc_ac0831-nextcl',
'dbpassword' ='NO SOUP FOR YOU',
'installed' =true,
'maintenance' =false,
'theme' ='',
'loglevel' =2,
'updater.secret' ='NO SOUP FOR YOU',
'mysql.utf8mb4' =true,
'overwrite.cli.url' ='https://nextcloud.domain.xyz',
'overwriteprotocol' ='https',
'overwritehost' ='nextcloud.domain.xyz',
'trusted_proxies' =array('192.168.1.222'),
);
The output of your Apache/nginx/system log in /var/log/____:
Local network (snippet, when it works)
192.168.1.58 - 192.168.1.210 [14/Jan/2019:09:31:14 -0800] "GET /index.php/css/core/7838-c5eb-css-variables.css?v=db81cddf52fdb3c8ca1e4c859e214124?v=76320180-0 HTTP/1.1" 200 429 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0" "-"rt=0.048 ut=0.048 cs=-
192.168.1.58 - 192.168.1.210 [14/Jan/2019:09:31:14 -0800] "GET /index.php/css/core/7838-c5eb-jquery-ui-fixes.css?v=db81cddf52fdb3c8ca1e4c859e214124?v=76320180-0 HTTP/1.1" 200 927 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0" "-"rt=0.073 ut=0.073 cs=-
192.168.1.58 - 192.168.1.210 [14/Jan/2019:09:31:14 -0800] "GET /index.php/css/core/7838-c5eb-server.css?v=db81cddf52fdb3c8ca1e4c859e214124?v=76320180-0 HTTP/1.1" 200 18993 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0" "-"rt=0.095 ut=0.096 cs=-
192.168.1.58 - 192.168.1.210 [14/Jan/2019:09:31:14 -0800] "GET /index.php/css/firstrunwizard/0d6d-c5eb-firstrunwizard.css?v=db81cddf52fdb3c8ca1e4c859e214124?v=76320180-0 HTTP/1.1" 200 1669 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0" "-"rt=0.117 ut=0.117 cs=-
192.168.1.58 - 192.168.1.210 [14/Jan/2019:09:31:14 -0800] "GET /index.php/css/notifications/70e2-c5eb-styles.css?v=db81cddf52fdb3c8ca1e4c859e214124?v=76320180-0 HTTP/1.1" 200 857 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0" "-"rt=0.140 ut=0.140 cs=-
192.168.1.58 - 192.168.1.210 [14/Jan/2019:09:31:14 -0800] "GET /index.php/css/core/7838-c5eb-jquery.ocdialog.css?v=db81cddf52fdb3c8ca1e4c859e214124?v=76320180-0 HTTP/1.1" 200 609 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0" "-"rt=0.162 ut=0.162 cs=-
192.168.1.58 - 192.168.1.210 [14/Jan/2019:09:31:14 -0800] "GET /index.php/css/core/7838-c5eb-results.css?v=db81cddf52fdb3c8ca1e4c859e214124?v=76320180-0 HTTP/1.1" 200 504 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0" "-"rt=0.136 ut=0.135 cs=-
192.168.1.58 - 192.168.1.210 [14/Jan/2019:09:31:14 -0800] "GET /index.php/css/files/e8ed-c5eb-merged.css?v=db81cddf52fdb3c8ca1e4c859e214124?v=76320180-0 HTTP/1.1" 200 5236 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0" "-"rt=0.132 ut=0.132 cs=-
On mobile data (snippet):
192.168.1.222 - nextcloud.domain.xyz [27/Dec/2019:16:25:40 -0600] "GET /apps/recommendations/api/recommendations HTTP/1.0" 200 1172 "-" "Mozilla/5.0 (X11; Linux aarch64; rv:71.0) Gecko/20100101 Firefox/71.0" "12.34.56.78"rt=0.120 ut=0.120 cs=-
192.168.1.222 - nextcloud.domain.xyz [27/Dec/2019:16:25:40 -0600] "GET /index.php/apps/files/ajax/getstoragestats.php?dir=%2F HTTP/1.0" 200 217 "-" "Mozilla/5.0 (X11; Linux aarch64; rv:71.0) Gecko/20100101 Firefox/71.0" "12.34.56.78"rt=0.031 ut=0.030 cs=-
192.168.1.222 - nextcloud.domain.xyz [27/Dec/2019:16:25:40 -0600] "GET /svg/core/actions/add?color=fff&v=1 HTTP/1.0" 200 176 "-" "Mozilla/5.0 (X11; Linux aarch64; rv:71.0) Gecko/20100101 Firefox/71.0" "12.34.56.78"rt=0.042 ut=0.042 cs=-
192.168.1.222 - nextcloud.domain.xyz [27/Dec/2019:16:25:40 -0600] "GET /apps/theming/icon/files?v=4 HTTP/1.0" 499 0 "-" "Mozilla/5.0 (X11; Linux aarch64; rv:71.0) Gecko/20100101 Firefox/71.0" "12.34.56.78"rt=0.001 ut=0.001 cs=-
192.168.1.222 - nextcloud.domain.xyz [27/Dec/2019:16:25:40 -0600] "GET /apps/theming/favicon/files?v=4 HTTP/1.0" 499 0 "-" "Mozilla/5.0 (X11; Linux aarch64; rv:71.0) Gecko/20100101 Firefox/71.0" "12.34.56.78"rt=0.001 ut=0.001 cs=-
192.168.1.222 - nextcloud.domain.xyz [27/Dec/2019:16:25:42 -0600] "GET /core/preview?fileId=10278&c=665aee10046126c5cee5294aded202bd&x=250&y=250&forceIcon=0 HTTP/1.0" 200 27483 "-" "Mozilla/5.0 (X11; Linux aarch64; rv:71.0) Gecko/20100101 Firefox/71.0" "12.34.56.78"rt=0.052 ut=0.052 cs=-
192.168.1.222 - nextcloud.domain.xyz [27/Dec/2019:16:25:42 -0600] "GET /core/preview?fileId=10288&c=685bd729b7646be0675bc0f0deec8e8c&x=250&y=250&forceIcon=0 HTTP/1.0" 200 38466 "-" "Mozilla/5.0 (X11; Linux aarch64; rv:71.0) Gecko/20100101 Firefox/71.0" "12.34.56.78"rt=0.072 ut=0.071 cs=-
192.168.1.222 - nextcloud.domain.xyz [27/Dec/2019:16:25:42 -0600] "GET /svg/core/actions/triangle-n?color=fff&v=1 HTTP/1.0" 200 147 "-" "Mozilla/5.0 (X11; Linux aarch64; rv:71.0) Gecko/20100101 Firefox/71.0" "12.34.56.78"rt=0.108 ut=0.108 cs=-
Nginx jail (192.168.1.222)
File: /usr/local/etc/nginx/nginx.conf
worker_processes auto; load_module /usr/local/libexec/nginx/ngx_stream_module.so; error_log /var/log/nginx/error.log; #pid logs/nginx.pid; events { worker_connections 1024; } http { include mime.types; include /usr/local/etc/nginx/conf.d/domain.xyz/*.conf; default_type text/html; client_max_body_size 0; sendfile on; }
File: /usr/local/et/nginx/conf.d/nextcloud.domain.xyz.conf
server { listen 8080; #listen [::]:8080; server_name nextcloud.domain.xyz nc.domain.xyz; return 301 https://$server_name$request_uri; } server { listen 8443 ssl; #listen [::]:8443 ssl; server_name nextcloud.domain.xyz nc.domain.xyz; include /usr/local/etc/nginx/conf.d/domain.xyz/ssl.settings; location / { proxy_pass http://192.168.1.215; proxy_set_header Host $host; proxy_set_header X-Forwarded-Host $server_name; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_read_timeout 90; proxy_set_header X-Forwarded-Proto $scheme; set $xforwardedssl "off"; if ($scheme = https) { set $xforwardedssl "on"; } proxy_set_header X-Forwarded-Ssl $xforwardedssl; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; } location = /.well-known/carddav { return 301 $scheme://$host/remote.php/dav; } location = /.well-known/caldav { return 301 $scheme://$host/remote.php/dav; } }
Nextcloud Jail (192.168.1.215)
File: /usr/local/etc/nginx/conf.d/nextcloud.conf
upstream php-handler { server 127.0.0.1:9000; #server unix:/var/run/php/php7.2-fpm.sock; } server { listen 80; listen [::]:80; server_name localhost; add_header X-Frame-Options SAMEORIGIN; add_header X-Content-Type-Options nosniff; add_header X-XSS-Protection "1; mode=block"; add_header X-Robots-Tag none; add_header X-Download-Options noopen; add_header X-Permitted-Cross-Domain-Policies none; add_header Referrer-Policy no-referrer; # Remove X-Powered-By, which is an information leak fastcgi_hide_header X-Powered-By; # Path to the root of your installation root /usr/local/www/nextcloud; location = /robots.txt { allow all; log_not_found off; access_log off; } location = /.well-known/carddav { return 301 $scheme://$host:$server_port/remote.php/dav; } location = /.well-known/caldav { return 301 $scheme://$host:$server_port/remote.php/dav; } # set max upload size client_max_body_size 512M; fastcgi_buffers 64 4K; # Enable gzip but do not remove ETag headers gzip on; gzip_vary on; gzip_comp_level 4; gzip_min_length 256; gzip_proxied expired no-cache no-store private no_last_modified no_etag auth; gzip_types application/atom+xml application/javascript application/json application/ld+json application/manifest+json application/rss+xml application/vnd.geo+json application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/bmp image/svg+xml image/x-icon text/cache-manifest text/css text/plain text/vcard text/vnd.rim.location.xloc text/vtt text/x-component text/x-cross-domain-policy; # Uncomment if your server is build with the ngx_pagespeed module # This module is currently not supported. #pagespeed off; location / { rewrite ^ /index.php; #rewrite ^ /index.php$request_uri; } location ~ ^\/(?:build|tests|config|lib|3rdparty|templates|data)\/ { deny all; } location ~ ^\/(?:\.|autotest|occ|issue|indie|db_|console) { deny all; } location ~ ^\/(?:index|remote|public|cron|core\/ajax\/update|status|ocs\/v[12]|updater\/.+|oc[ms]-provider\/.+)\.php(?:$|\/) { fastcgi_split_path_info ^(.+?\.php)(\/.*|)$; set $path_info $fastcgi_path_info; try_files $fastcgi_script_name =404; include fastcgi_params; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_param PATH_INFO $path_info; fastcgi_param HTTPS on; # Avoid sending the security headers twice fastcgi_param modHeadersAvailable true; # Enable pretty urls fastcgi_param front_controller_active true; fastcgi_pass php-handler; fastcgi_intercept_errors on; fastcgi_request_buffering off; } location ~ ^\/(?:updater|oc[ms]-provider)(?:$|\/) { try_files $uri/ =404; index index.php; } # Adding the cache control header for js, css and map files # Make sure it is BELOW the PHP block location ~ \.(?:css|js|woff2?|svg|gif|map)$ { try_files $uri /index.php$request_uri; add_header Cache-Control "public, max-age=15778463"; add_header X-Frame-Options SAMEORIGIN; add_header X-Content-Type-Options nosniff; add_header X-XSS-Protection "1; mode=block"; add_header X-Robots-Tag none; add_header X-Download-Options noopen; add_header X-Permitted-Cross-Domain-Policies none; add_header Referrer-Policy no-referrer; # Optional: Don't log access to assets access_log off; } location ~ \.(?:png|html|ttf|ico|jpg|jpeg|bcmap)$ { try_files $uri /index.php$request_uri; # Optional: Don't log access to other assets access_log off; } }
Local Network → Domain (HTTPS Redirect):
$ curl -v http://nextcloud.domain.xyz/status.php --trace-time
15:44:00.044498 * Trying 12.34.56.78...
15:44:00.044689 * TCP_NODELAY set
15:44:00.048513 * Connected to nextcloud.domain.xyz (12.34.56.78) port 80 (#0)
15:44:00.048676 > GET /status.php HTTP/1.1
15:44:00.048676 > Host: nextcloud.domain.xyz
15:44:00.048676 > User-Agent: curl/7.58.0
15:44:00.048676 > Accept: */*
15:44:00.048676 >
15:44:00.058516 < HTTP/1.1 301 Moved Permanently
15:44:00.058595 < Server: nginx/1.16.1
15:44:00.058669 < Date: Fri, 27 Dec 2019 21:44:00 GMT
15:44:00.058708 < Content-Type: text/html
15:44:00.058769 < Content-Length: 169
15:44:00.058828 < Connection: keep-alive
15:44:00.058886 < Location: https://nextcloud.domain.xyz/status.php
15:44:00.058945 <
<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.16.1</center>
</body>
</html>
15:44:00.059027 * Connection #0 to host nextcloud.domain.xyz left intact
Local Network → Domain (HTTPS):
NOTE: When issuing this command from LAN, sometimes it actually works. Occasionally it instantly connects, sometimes it takes forever to connect, but most times it fails just as it does below. No rhyme or reason to it.
$ curl -v https://nextcloud.domain.xyz/status.php --trace-time
14:58:06.487613 * Trying 12.34.56.78...
14:58:06.487693 * TCP_NODELAY set
15:00:17.584952 * connect to 12.34.56.78 port 443 failed: Connection timed out
15:00:17.585032 * Failed to connect to nextcloud.domain.xyz port 443: Connection timed out
15:00:17.585110 * Closing connection 0
curl: (7) Failed to connect to nextcloud.domain.xyz port 443: Connection timed out
Outside Network → Domain (HTTPS):
$ curl -v https://nextcloud.domain.xyz --trace-time
16:35:00.764919 * Trying 12.34.56.78:443...
16:35:00.765113 * TCP_NODELAY set
16:35:00.820717 * Connected to nextcloud.domain.xyz (12.34.56.78) port 443 (#0)
16:35:00.825550 * ALPN, offering h2
16:35:00.825914 * ALPN, offering http/1.1
16:35:00.854749 * successfully set certificate verify locations:
16:35:00.854835 * CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none
16:35:00.855220 * TLSv1.3 (OUT), TLS handshake, Client hello (1):
16:35:00.920998 * TLSv1.3 (IN), TLS handshake, Server hello (2):
16:35:00.921415 * TLSv1.2 (IN), TLS handshake, Certificate (11):
16:35:00.923115 * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
16:35:00.923732 * TLSv1.2 (IN), TLS handshake, Server finished (14):
16:35:00.937047 * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
16:35:00.937227 * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
16:35:00.937496 * TLSv1.2 (OUT), TLS handshake, Finished (20):
16:35:00.983783 * TLSv1.2 (IN), TLS handshake, Finished (20):
16:35:00.984083 * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
16:35:00.984246 * ALPN, server accepted to use http/1.1
16:35:00.984427 * Server certificate:
16:35:00.984601 * subject: CN=*.domain.xyz
16:35:00.984782 * start date: Dec 26 18:08:41 2019 GMT
16:35:00.984951 * expire date: Mar 25 18:08:41 2020 GMT
16:35:00.985158 * subjectAltName: host "nextcloud.domain.xyz" matched cert's "*.domain.xyz"
16:35:00.985398 * issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
16:35:00.985666 * SSL certificate verify ok.
16:35:00.986062 > GET / HTTP/1.1
16:35:00.986062 > Host: nextcloud.domain.xyz
16:35:00.986062 > User-Agent: curl/7.67.0
16:35:00.986062 > Accept: */*
16:35:00.986062 >
16:35:01.073049 * Mark bundle as not supporting multiuse
16:35:01.073262 < HTTP/1.1 302 Found
16:35:01.073452 < Server: nginx/1.16.1
16:35:01.073642 < Date: Fri, 27 Dec 2019 22:35:00 GMT
16:35:01.073836 < Content-Type: text/html; charset=UTF-8
16:35:01.074032 < Transfer-Encoding: chunked
16:35:01.074203 < Connection: keep-alive
16:35:01.074393 < Set-Cookie: ocoxlslmcu9a=td241vimqscqi28uk76vt26rqo; path=/; secure; HttpOnly
16:35:01.074588 < Expires: Thu, 19 Nov 1981 08:52:00 GMT
16:35:01.074777 < Cache-Control: no-store, no-cache, must-revalidate
16:35:01.074960 < Pragma: no-cache
16:35:01.075152 < Set-Cookie: oc_sessionPassphrase=o0spwzlgXTirqhb7uvBvr%2ByPJ%2Bg0youasdfcCv41%2BswIKYQmzMK0IfmH%2FZLl9BLr%2B7LRMv07sF%2B9BcH5ScLob2BcUyBFPNwUepXnTdvVcX1%2FTDWA3be71bwwo3Ngx; path=/; secure; HttpOnly
16:35:01.075346 < Content-Security-Policy: default-src 'self'; script-src 'self' 'nonce-SDl3aFhyVjR5adsfsIwS08xbHVUa3QvRzBasdfKzVySW1uR0pYc09lQT06YmIasdfWlBCZGFqasdfUThHYXphMjcwM3E3OUR1VnViWUFaST0='; style-src 'self' 'unsafe-inline'; frame-src *; img-src * data: blob:; font-src 'self' data:; media-src *; connect-src *; object-src 'none'; base-uri 'self';
16:35:01.075545 < Set-Cookie: __Host-nc_sameSiteCookielax=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax
16:35:01.075737 < Set-Cookie: __Host-nc_sameSiteCookiestrict=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict
16:35:01.075940 < Location: https://nextcloud.domain.xyz/login
16:35:01.076133 < X-Frame-Options: SAMEORIGIN
16:35:01.076325 < X-Content-Type-Options: nosniff
16:35:01.076516 < X-XSS-Protection: 1; mode=block
16:35:01.076708 < X-Robots-Tag: none
16:35:01.076901 < X-Download-Options: noopen
16:35:01.077080 < X-Permitted-Cross-Domain-Policies: none
16:35:01.077277 < Referrer-Policy: no-referrer
16:35:01.077476 <
16:35:01.077744 * Connection #0 to host nextcloud.domain.xyz left intact
The desktop app fails the same way (only from LAN, otherwise it works):
Does anyone see anything wrong with my configs that could be causing this? It is driving me absolutely insane… I can’t find anything wrong.