Loosing connection after big sync / directory not writeable

Hello there, im facing a connection issue with using a raspberry 4 with 4GB RAM and an external 1TB SSD attached. After a while my sync status becomes unresponsive and im loosing connection to my nextcloud. I can’t even access the webinterface anymore for more logs etc.

How can I provide you more logs via ssh?
Thanks in advance!

grafik

Your data directory is not writable

Berechtigungen können zumeist korrigiert werden indem dem Web-Server Schreibzugriff auf das Wurzel-Verzeichnis eingeräumt wird. Siehe auch https://docs.nextcloud.com/server/16/go.php?to=admin-dir_permissions.

Nextcloud version 16.0.6
Raspian Buster

The issue you are facing:

At first connection seems stable and fast, but after a while it gets slower and completly looses conection, sometimes with a “Bad Gateway 504” error in the nextcloud syncclient on mac

Is this the first time you’ve seen this error? (Y/N):
no

Steps to replicate it:

  1. Sync large folder with lots of files ( ca. 10.000)
  2. watch sync client after ca. 30mins
  3. other clients (android app) loose connection too

The output of your Nextcloud log in Admin > Logging:

NextCloudPi diagnostics

cat: /usr/local/etc/ncp-version: No such file or directory
{"reqId":"fZkhJL5GCLwUGtr8mhPC","level":2,"time":"2019-11-13T12:11:11+00:00","remoteAddr":"","user":"--","app":"no app in context","method":"","url":"--","message":"Temporary directory \/media\/myCloudDrive\/ncdata\/tmp is not present or writable","userAgent":"--","version":"16.0.6.1"}
{"reqId":"fZkhJL5GCLwUGtr8mhPC","level":3,"time":"2019-11-13T12:11:12+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fopen(\/media\/myCloudDrive\/ncdata\/data_dir_writability_test_5dcbf2e02eea4.tmp): failed to open stream: Read-only file system at \/var\/www\/nextcloud\/lib\/private\/legacy\/util.php#796","userAgent":"--","version":"16.0.6.1"}
{"reqId":"fZkhJL5GCLwUGtr8mhPC","level":3,"time":"2019-11-13T12:11:12+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fopen(\/media\/myCloudDrive\/ncdata\/nextcloud.log): failed to open stream: Read-only file system at \/var\/www\/nextcloud\/lib\/private\/Log\/File.php#139","userAgent":"--","version":"16.0.6.1"}
NextCloudPi version
distribution         Raspbian GNU/Linux 10 \n \l
automount            yes
USB devices          sda 
datadir              /media/myCloudDrive/ncdata
data in SD           no
data filesystem      btrfs
data disk usage      19G/932G
rootfs usage         2.3G/29G
swapfile             /var/swap
dbdir                /var/lib/mysql
Nextcloud check      error
HTTPD service        up
PHP service          up
MariaDB service      up
Redis service        up
Postfix service      up
internet check       ok
port check 80        closed
port check 443       closed
IP                   ***REMOVED SENSITIVE VALUE***
gateway              ***REMOVED SENSITIVE VALUE***
interface            eth0
certificates         ***REMOVED SENSITIVE VALUE***
NAT loopback         no
uptime               22:41

Nextcloud configuration

{"reqId":"54AKrGfZ7L8wf2FJ8kkQ","level":2,"time":"2019-11-13T12:11:18+00:00","remoteAddr":"","user":"--","app":"no app in context","method":"","url":"--","message":"Temporary directory \/media\/myCloudDrive\/ncdata\/tmp is not present or writable","userAgent":"--","version":"16.0.6.1"}
{"reqId":"54AKrGfZ7L8wf2FJ8kkQ","level":3,"time":"2019-11-13T12:11:18+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fopen(\/media\/myCloudDrive\/ncdata\/data_dir_writability_test_5dcbf2e6a3798.tmp): failed to open stream: Read-only file system at \/var\/www\/nextcloud\/lib\/private\/legacy\/util.php#796","userAgent":"--","version":"16.0.6.1"}
Your data directory is not writable
Permissions can usually be fixed by giving the webserver write access to the root directory. See https://docs.nextcloud.com/server/16/go.php?to=admin-dir_permissions.

An unhandled exception has been thrown:
Exception: Environment not properly prepared. in /var/www/nextcloud/lib/private/Console/Application.php:166
Stack trace:
#0 /var/www/nextcloud/console.php(96): OC\Console\Application->loadCommands(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#1 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#2 {main}{"reqId":"54AKrGfZ7L8wf2FJ8kkQ","level":3,"time":"2019-11-13T12:11:18+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fopen(\/media\/myCloudDrive\/ncdata\/nextcloud.log): failed to open stream: Read-only file system at \/var\/www\/nextcloud\/lib\/private\/Log\/File.php#139","userAgent":"--","version":"16.0.6.1"}

HTTPd logs

[Wed Nov 13 08:22:18.208194 2019] [authz_host:error] [pid 777:tid 2822480928] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /img/defaults-white.svg failed, reason: unable to get the remote host name, referer: https://192.168.0.73:4443/css/ncp.css
[Wed Nov 13 08:22:18.208191 2019] [authz_host:error] [pid 777:tid 2906407968] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /img/dashboard.svg failed, reason: unable to get the remote host name, referer: https://192.168.0.73:4443/css/ncp.css
[Wed Nov 13 08:22:18.212735 2019] [authz_host:error] [pid 777:tid 2822480928] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /img/wizard.svg failed, reason: unable to get the remote host name, referer: https://192.168.0.73:4443/css/ncp.css
[Wed Nov 13 08:22:18.212735 2019] [authz_host:error] [pid 777:tid 2872837152] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /img/info-white.svg failed, reason: unable to get the remote host name, referer: https://192.168.0.73:4443/css/ncp.css
[Wed Nov 13 08:22:18.212741 2019] [authz_host:error] [pid 777:tid 2906407968] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /img/settings-white.svg failed, reason: unable to get the remote host name, referer: https://192.168.0.73:4443/css/ncp.css
[Wed Nov 13 08:22:18.212969 2019] [authz_host:error] [pid 777:tid 2872837152] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /img/poweroff.svg failed, reason: unable to get the remote host name, referer: https://192.168.0.73:4443/css/ncp.css
[Wed Nov 13 08:22:18.252061 2019] [authz_host:error] [pid 777:tid 2755290144] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /ncp-output.php failed, reason: unable to get the remote host name
[Wed Nov 13 08:22:18.252061 2019] [authz_host:error] [pid 777:tid 2872837152] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /ncp-launcher.php failed, reason: unable to get the remote host name
[Wed Nov 13 08:22:19.797667 2019] [authz_host:error] [pid 777:tid 2906407968] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /img/favicon.png failed, reason: unable to get the remote host name
[Wed Nov 13 08:22:20.329500 2019] [authz_host:error] [pid 777:tid 2906407968] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /img/files.svg failed, reason: unable to get the remote host name, referer: https://192.168.0.73:4443/css/ncp.css
[Wed Nov 13 08:22:34.580542 2019] [authz_host:error] [pid 777:tid 2872837152] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /ncp-launcher.php failed, reason: unable to get the remote host name
[Wed Nov 13 08:22:45.908515 2019] [authz_host:error] [pid 777:tid 2906407968] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /img/info.svg failed, reason: unable to get the remote host name, referer: https://192.168.0.73:4443/css/ncp.css
[Wed Nov 13 08:23:26.033340 2019] [authz_host:error] [pid 777:tid 2906407968] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /ncp-launcher.php failed, reason: unable to get the remote host name
[Wed Nov 13 08:25:48.211224 2019] [authz_host:error] [pid 777:tid 2872837152] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /ncp-launcher.php failed, reason: unable to get the remote host name
[Wed Nov 13 08:25:48.211331 2019] [authz_host:error] [pid 777:tid 2814088224] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /img/delete.svg failed, reason: unable to get the remote host name
[Wed Nov 13 08:25:48.211332 2019] [authz_host:error] [pid 777:tid 2822480928] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /img/download.svg failed, reason: unable to get the remote host name
[Wed Nov 13 08:25:50.826579 2019] [authz_host:error] [pid 777:tid 2906407968] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /ncp-launcher.php failed, reason: unable to get the remote host name
[Wed Nov 13 08:25:59.438232 2019] [authz_host:error] [pid 777:tid 2906407968] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /img/green-circle.svg failed, reason: unable to get the remote host name, referer: https://192.168.0.73:4443/css/ncp.css
[Wed Nov 13 08:26:30.684419 2019] [authz_host:error] [pid 777:tid 2906407968] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /ncp-launcher.php failed, reason: unable to get the remote host name
[Wed Nov 13 08:27:35.917829 2019] [authz_host:error] [pid 777:tid 2906407968] [client 192.168.0.157:63533] AH01753: access check of 'localhost' to /ncp-launcher.php failed, reason: unable to get the remote host name

Database logs

2019-11-12 13:30:37 0 [Note] InnoDB: Uses event mutexes
2019-11-12 13:30:37 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2019-11-12 13:30:37 0 [Note] InnoDB: Number of pools: 1
2019-11-12 13:30:37 0 [Note] InnoDB: Using generic crc32 instructions
2019-11-12 13:30:37 0 [Note] InnoDB: Initializing buffer pool, total size = 1.625G, instances = 1, chunk size = 128M
2019-11-12 13:30:37 0 [Note] InnoDB: Completed initialization of buffer pool
2019-11-12 13:30:37 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-11-12 13:30:37 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2019-11-12 13:30:37 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-11-12 13:30:37 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2019-11-12 13:30:37 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2019-11-12 13:30:37 0 [Note] InnoDB: 10.3.17 started; log sequence number 56374542; transaction id 55887
2019-11-12 13:30:37 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2019-11-12 13:30:37 0 [Note] Plugin 'FEEDBACK' is disabled.
2019-11-12 13:30:38 0 [Note] Server socket created on IP: '127.0.0.1'.
2019-11-12 13:30:38 0 [Note] Reading of all Master_info entries succeeded
2019-11-12 13:30:38 0 [Note] Added new Master_info '' to hash table
2019-11-12 13:30:38 0 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.3.17-MariaDB-0+deb10u1'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Raspbian 10
2019-11-12 13:30:38 0 [Note] InnoDB: Buffer pool(s) load completed at 191112 13:30:38

I can’t paste the last details here in discourse, too much characters it says, so i posted the screenshot

could it be that my SSD is somewhat causing the error? how can i check if it has enough power?

after a reboot, I can access the webUI again.
but the sync still fails

admin log, recent errors:

Well, I may have found a fix. Using APCu for caching instead of Redis seems to be much more stable for the RPi 4. My large folder syncs nicely.

https://docs.nextcloud.com/server/11/admin_manual/configuration_server/caching_configuration.html#recommendations-based-on-type-of-deployment

I did the following:

  1. sudo apt-get install php-apcu

  2. sudo service apache2 restart

  3. sudo nano /var/www/nextcloud/config/config.php
    change the memcache.local to ACPu
    'memcache.local' => '\OC\Memcache\APCu',

  4. sudo nano /etc/php/7.3/cli/php.ini
    add at the very end
    apc.enable_cli=1

  5. sudo reboot

Hope that helps!