Nextcloud sync errors (>100 files), no clear error, can't find the culprit

Nextcloud version: 16.0.5
Operating system and version: Raspian 10
Apache version: Apache 2.4.38
PHP version: 7.3.9

The issue you are facing:
When I sync a few files, everything works fine. When I try to sync a lot of files (> 100), Nextcloud client says:

“Error transferring https://x.com/index.php/apps/files - server replied:”

And that’s it. I’ve posted the output of the Client log at the very end of this post. As a test, I’ve created 100 empty files in a directory. Nextcloud uploads them and then gets stuck at the 95th file with this error (server replied). After some time, the client continues and successfully syncs.

With some GB of data it’s rather impossible so sync as it takes forever. This happens with clients on Windows as well as on Debian machines.

Any ideas? Thank you!
-Richard

Is this the first time you’ve seen this error? N

Steps to replicate it:

I don’t know… I’ve had the problem since August 2019. I create 100 files, or I try to sync everything (new client install) and it gets stuck.

The output of your Nextcloud log in Admin > Logging:
No log available, everything is reportedly running fine…

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'x',
  'passwordsalt' => 'x',
  'secret' => 'x',
  'trusted_domains' => 
  array (
	  0 => 'x.firewall-gateway.com',
  ),
  'datadirectory' => '/mnt/raid1/nextcloud/data',
  'dbtype' => 'pgsql',
  'version' => '16.0.5.1',
  'overwrite.cli.url' => 'https://x.firewall-gateway.com',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'nextcloud',
  'dbpassword' => 'x',
  'installed' => true,
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 2,
  'mail_from_address' => 'x',
  'mail_smtpmode' => 'smtp',
  'mail_domain' => 'x',
  'mail_smtpport' => '587',
  'mail_smtphost' => 'x',
  'mail_smtpauth' => 1,
  'mail_smtpname' => 'x',
  'mail_smtppassword' => 'x',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'updater.secret' => 'x',
  "log_type" => "file",
  "logfile" => "/var/log/nextcloud.log",
  'logfilemode' => 0640,
  "loglevel" => "2",
  "logdateformat" => "F d, Y H:i:s",
);

The output of your Apache error log in /var/log/____:

[Thu Oct 17 14:33:53.691868 2019] [evasive20:error] [pid 5598:tid 1577038880] [client 88.70.253.144:50444] client denied by server configuration: /mnt/raid1/nextcloud/remote.php
[Thu Oct 17 14:33:54.081538 2019] [evasive20:error] [pid 5598:tid 1501504544] [client 88.70.253.144:50444] client denied by server configuration: /mnt/raid1/nextcloud/index.php
[Thu Oct 17 14:35:19.006196 2019] [authz_core:error] [pid 2613:tid 1723814944] [client 88.70.253.144:36328] AH01630: client denied by server configuration: /mnt/raid1/nextcloud/config
[Thu Oct 17 14:37:30.111638 2019] [authz_core:error] [pid 2613:tid 1740616736] [client 88.70.253.144:36380] AH01630: client denied by server configuration: /mnt/raid1/nextcloud/data/.ocdata
[Thu Oct 17 14:37:36.315234 2019] [authz_core:error] [pid 2613:tid 1698612256] [client 88.70.253.144:36380] AH01630: client denied by server configuration: /mnt/raid1/nextcloud/data/.ocdata

Client Output (2.5.2, but happens with 2.6 as well):

OCC::AbstractNetworkJob::slotFinished QNetworkReply::NetworkError(ContentAccessDenied) "Error transferring https://x.com/index.php/apps/files/ - server replied: " QVariant(int, 403)

[OCC::WebFlowCredentials::stillValid QNetworkReply::NetworkError(ContentAccessDenied)

[OCC::WebFlowCredentials::stillValid "Error transferring https://x.com/index.php/apps/files/ - server replied: "

[OCC::PUTFileJob::finished PUT of "https://x.com/index.php/apps/files/" FINISHED WITH STATUS "ContentAccessDenied Error transferring https://x.com/index.php/apps/files/ - server replied: " QVariant(int, 403) QVariant(QString, "")

[OCC::createBlacklistEntry Probably firewall error: 403 , blacklisting up to 1h only

[OCC::SyncJournalDb::setErrorBlacklistEntry Setting blacklist entry for "User/test/bspl095" 1 "Error transferring https://x.com/index.php/apps/files/ - server replied: " 1571324337 25 1571324317 "" "" 0

[OCC::PropagateItemJob::done Could not complete propagation of "User/test/bspl095" by OCC::PropagateUploadFileV1(0x562666d841d0) with status 2 and error: "Error transferring https://x.com/index.php/apps/files/ - server replied: "

[OCC::ActivityWidget::slotItemCompleted Item "User/test/bspl095" retrieved resulted in "Error transferring https://x.com/index.php/apps/files/ - server replied: "

[OCC::ActivityWidget::slotItemCompleted Item "User/test/bspl095" retrieved resulted in error "Error transferring https://x.com/index.php/apps/files/ - server replied: "

[OCC::ActivityListModel::addErrorToActivityList Error successfully added to the notification list: "Error transferring https://x.com/index.php/apps/files/ - server replied: "

I would go for redis locking to see if it helps. It is also a lot faster than SQL locks.

  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' =>
  array (
    'host' => '/tmp/redis.sock',
    'port' => 0,
    'timeout' => 0,
    'dbindex' => 0,
  ),

Also check that you do not have gzip compression enabled in Apache.

Thanks so much for answering!

I don’t have gzip compression enabled.
Redis is somehow not working(with your or the config from nextcloudcom- I’ve ibstalled the packages), but I’ll try again tomorrow. Not sure whether SQL locks should get stuck on a hundred files, though.

Best
Richard

Unfortunately, Redis locking doesn’t help…

When Redis is used, there should be no locks in the SQL database. You also have to make sure the /run/redis.sockis accessible by the webserver, or you have to use TCP connections to Redis. My experience is that Unix sockets are slightly faster.

Edit:
I added “Satisfy Any” into my apache2 site conf file under the “Directory” tag - it worked very well again, BUT this is a very bad choice. Satisfy Any exposes your files to world access without authentification. So anyone could download the files with a direct link to the files.

I’m still not sure what could solve the original issue, which still persists.

Richard

Hi, anyone knows if this has been solved? I am still having this problem with v20.0.1.1. Any help appreciated.

1 Like