"Syncing files stopping after short time"

make · March 2, 2025, 12:17pm

Since some weeks I am setting up a Nextcloud instance on my homebrew NAS. The structure of the server is like so:

Proxmox\
       \CT1:pihole
       \CT2:nginx
       \VM1:TrueNas\
                   \CT1:Nextcloud\
                                 \CT1:cron            - running
                                 \CT2:nextcloud       - running
                                 \CT3:postgres        - running
                                 \CT4:postgres-update - exited
                                 \CT5:redis           - running
                                 \CT6:permissions     - exited

RAM:

Proxmox Host 64GB, thereof given TrueNas VM 28 GB, thereof given Nextcloud CT 4 GB

CPUs:

Proxmox Host 16, thereof given TrueNas VM 8, thereof given Nextcloud CT 4

Storage:

2 SSDs, ZFS mirror for Proxmox, CTs and VMs
2 harddrives, GPIO passthrough to TrueNAS VM, ZFS mirror in TrueNAS

I can access Nextcloud from inside and outside of my home network without issues.

The Basics

Nextcloud Server version:
- started with 30.0.5 / 1.5.15 later same with 1.5.18
- actual 31.0.0 / 1.6.9
Operating system and version:
- Proxmox: Linux 6.8.12-8-pve
- TrueNAS VM: Debian 12, 6.6.44-production+truenas x86_64
Web server and version:
- Server version: Apache/2.4.62 (Debian)
Reverse proxy and version:
- Nginx Proxy Manager v2.12.2 (runs in other Proxmox container)
PHP version:
- 8.3.17 (cli)
- Zend Engine with Zend OPcache v8.3.17
Is this the first time you’ve seen this error?:
- yes, but fresh installation
When did this problem seem to first start?
- It starts on first attempt when syncing a larger amount of files via the desktop client
Installation method:
- Proxmox host running TrueNAS (ElectricEel-24.10.1) in a virtual machine. Nextcloud installed in a Docker container on TrueNAS
- nginx and pihole run in Proxmox containers
Are you using CloudfIare, mod_security, or similar?
- No

Summary of the issue you are facing:

When I try to sync a larger amount of files (5000 files/1.4 GB) via the desktop client, it starts fast, syncing about 1200 files/510 MB, then gets slower and slower, finally stops. When restarting the desktop client, sometimes it starts syncing again (with uploading files), sometimes not or it crashes.

Steps to replicate it:

A) with the installation described above

delete user account in Windows Desktop client
delete user account in Nextcloud web interface
on Windows system, delete the hidden files in the synced folder
(nextcloudsync.log and .sync_???.db)
create a user new account in Nextcloud web interface
on Windows system, connect to the Nextcloud, select the Windows folder to be synced
syncing starts but then stops

B) create a new installation on Truenas

I did this to rule out I accidently misconfigured something

Set up new datasets for a new Nextcloud instance in TrueNAS
Shutdown production instance, install and deploy 2nd instance for testing

=> same result

C) create a new installation on Proxmox

Because of multiple nested VM/CTs and to rule out a misconfiguration with certificates and proxies, I tried an installation in a Proxmox Container, accessing this instance only via IP address
=> same result

D) create a new installation on Proxmox

This time with Christian Rieger script
=> same result
Then, with this installation I tried to sync from with different computers:
Windows PC/LAN => stops
Linux Laptop/WiFi => OK!
Windows Laptop/WiFi
NC 3.4.1 client => error messages, but runs through
NC 3.15.3 client => network error 7
Because it worked with slow WiFi connection I think it could be a problem for the server to accept many files in short time?

Log entries

Nextcloud

Please provide the log entries from your Nextcloud log that are generated during the time of problem (via the Copy raw option from Administration settings->Logging screen or from your nextcloud.log located in your data directory). Feel free to use a pastebin/gist service if necessary.

On the Windows desktop client the file .nextcloudsync.log is zero length

In the server web interface There are no errors shown with standard error level ‘warning’.
When I switch to highest logging level, warnings show up, some examples:

{"reqId":"cScRdgLNLro6keS2WnXe","level":0,"time":"2025-03-02T09:23:02+00:00","remoteAddr":"192.168.178.35","user":"wolke_admin","app":"app_api","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"ExApp \"survey_client\" not found.","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0","version":"31.0.0.18","data":{"app":"app_api"},"id":"67c4237f93e4a"}
{"reqId":"hirdeIUH39jxZfgldX0n","level":0,"time":"2025-03-02T09:22:50+00:00","remoteAddr":"192.168.178.11","user":false,"app":"core","method":"HEAD","url":"/ocm-provider/","message":"The loading of lazy AppConfig values have been triggered by app \"core\"","userAgent":"Nextcloud Server Crawler","version":"31.0.0.18","exception":{"Exception":"RuntimeException","Message":"The loading of lazy AppConfig values have been triggered by app \"core\"","Code":0,"Trace":[{"file":"/var/www/html/lib/private/AppConfig.php","line":442,"function":"loadConfig","class":"OC\\AppConfig","type":"->","args":["core",true]},{"file":"/var/www/html/lib/private/AppConfig.php","line":327,"function":"getTypedValue","class":"OC\\AppConfig","type":"->","args":["core","ocm_providers","OCA\\CloudFederationAPI\\Capabilities",true,4]},{"file":"/var/www/html/core/Controller/OCMController.php","line":59,"function":"getValueString","class":"OC\\AppConfig","type":"->","args":["core","ocm_providers","OCA\\CloudFederationAPI\\Capabilities",true]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":200,"function":"discovery","class":"OC\\Core\\Controller\\OCMController","type":"->","args":[]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":114,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OC\\Core\\Controller\\OCMController"},"discovery"]},{"file":"/var/www/html/lib/private/AppFramework/App.php","line":161,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OC\\Core\\Controller\\OCMController"},"discovery"]},{"file":"/var/www/html/lib/private/Route/Router.php","line":307,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["OC\\Core\\Controller\\OCMController","discovery",{"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},{"_route":"core.ocm.discovery"}]},{"file":"/var/www/html/lib/base.php","line":1018,"function":"match","class":"OC\\Route\\Router","type":"->","args":["/ocm-provider/"]},{"file":"/var/www/html/index.php","line":24,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"/var/www/html/lib/private/AppConfig.php","Line":1208,"message":"The loading of lazy AppConfig values have been triggered by app \"core\"","exception":[],"CustomMessage":"The loading of lazy AppConfig values have been triggered by app \"core\""},"id":"67c4237599853"}
{"reqId":"2r6KJJe88APn5rwWJahS","level":0,"time":"2025-03-02T09:25:01+00:00","remoteAddr":"","user":false,"app":"no app in context","method":"","url":"--","message":"dirty table reads: SELECT * FROM `*PREFIX*jobs` WHERE (`reserved_at` <= :dcValue1) AND (`last_checked` <= :dcValue2) AND (`time_sensitive` = :dcValue3) ORDER BY `last_checked` ASC LIMIT 1","userAgent":"--","version":"31.0.0.18","exception":{"Exception":"Exception","Message":"dirty table reads: SELECT * FROM `*PREFIX*jobs` WHERE (`reserved_at` <= :dcValue1) AND (`last_checked` <= :dcValue2) AND (`time_sensitive` = :dcValue3) ORDER BY `last_checked` ASC LIMIT 1","Code":0,"Trace":[{"file":"/var/www/html/lib/private/DB/ConnectionAdapter.php","line":50,"function":"executeQuery","class":"OC\\DB\\Connection","type":"->","args":["SELECT * FROM `*PREFIX*jobs` WHERE (`reserved_at` <= :dcValue1) AND (`last_checked` <= :dcValue2) AND (`time_sensitive` = :dcValue3) ORDER BY `last_checked` ASC LIMIT 1",{"dcValue1":1740864301,"dcValue2":1740907501,"dcValue3":1},{"dcValue1":1,"dcValue2":1,"dcValue3":1}]},{"file":"/var/www/html/lib/private/DB/QueryBuilder/QueryBuilder.php","line":289,"function":"executeQuery","class":"OC\\DB\\ConnectionAdapter","type":"->","args":["SELECT * FROM `*PREFIX*jobs` WHERE (`reserved_at` <= :dcValue1) AND (`last_checked` <= :dcValue2) AND (`time_sensitive` = :dcValue3) ORDER BY `last_checked` ASC LIMIT 1",{"dcValue1":1740864301,"dcValue2":1740907501,"dcValue3":1},{"dcValue1":1,"dcValue2":1,"dcValue3":1}]},{"file":"/var/www/html/lib/private/BackgroundJob/JobList.php","line":192,"function":"executeQuery","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/html/cron.php","line":150,"function":"getNext","class":"OC\\BackgroundJob\\JobList","type":"->","args":[true,null]}],"File":"/var/www/html/lib/private/DB/Connection.php","Line":402,"message":"dirty table reads: SELECT * FROM `*PREFIX*jobs` WHERE (`reserved_at` <= :dcValue1) AND (`last_checked` <= :dcValue2) AND (`time_sensitive` = :dcValue3) ORDER BY `last_checked` ASC LIMIT 1","tables":["oc_appconfig","oc_jobs","oc_user_status","oc_direct_edit"],"reads":["oc_jobs"],"exception":[],"CustomMessage":"dirty table reads: SELECT * FROM `*PREFIX*jobs` WHERE (`reserved_at` <= :dcValue1) AND (`last_checked` <= :dcValue2) AND (`time_sensitive` = :dcValue3) ORDER BY `last_checked` ASC LIMIT 1"},"id":"67c423f022911"}

Web Browser

The problem seems not to be related with the web interface

Web server / Reverse Proxy

The output of your Apache/nginx/system log in /var/log/____:

Configuration

Nextcloud

The output of occ config:list system or similar is best, but, if not possible, the contents of your config.php file from /path/to/nextcloud is fine (make sure to remove any identifiable information!):

# occ config:list system
{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apc.shm_size": "128M",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "upgrade.disable-web": true,
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "31.0.0.18",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "maintenance_window_start": 1,
        "maintenance": false,
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "ssl",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "default_phone_region": "DE",
        "loglevel": 0,
        "forbidden_filename_basenames": [
            "con",
            "prn",
            "aux",
            "nul",
            "com0",
            "com1",
            "com2",
            "com3",
            "com4",
            "com5",
            "com6",
            "com7",
            "com8",
            "com9",
            "com\u00b9",
            "com\u00b2",
            "com\u00b3",
            "lpt0",
            "lpt1",
            "lpt2",
            "lpt3",
            "lpt4",
            "lpt5",
            "lpt6",
            "lpt7",
            "lpt8",
            "lpt9",
            "lpt\u00b9",
            "lpt\u00b2",
            "lpt\u00b3"
        ],
        "forbidden_filename_characters": [
            "<",
            ">",
            ":",
            "\"",
            "|",
            "?",
            "*",
            "\\",
            "\/"
        ],
        "forbidden_filename_extensions": [
            " ",
            ".",
            ".filepart",
            ".part"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "https:\/\/nextcloud.sub.domain.de",
        "overwriteprotocol": "https",
        "overwritehost": "nextcloud.sub.domain.de",
        "trusted_domains": [
            "127.0.0.1",
            "192.168.178.11",
            "localhost",
            "nextcloud",
            "nextcloud.sub.domain.de"
        ]
    }
}

Apps

The output of occ app:list (if possible).

# occ app:list
Enabled:
  - activity: 4.0.0
  - app_api: 5.0.2
  - bruteforcesettings: 4.0.0
  - calendar: 5.1.2
  - cloud_federation_api: 1.14.0
  - comments: 1.21.0
  - contacts: 7.0.1
  - contactsinteraction: 1.12.0
  - dashboard: 7.11.0
  - dav: 1.33.0
  - federatedfilesharing: 1.21.0
  - federation: 1.21.0
  - files: 2.3.1
  - files_downloadlimit: 4.0.0
  - files_pdfviewer: 4.0.0
  - files_reminders: 1.4.0
  - files_sharing: 1.23.1
  - files_trashbin: 1.21.0
  - files_versions: 1.24.0
  - firstrunwizard: 4.0.0
  - logreader: 4.0.0
  - lookup_server_connector: 1.19.0
  - nextcloud_announcements: 3.0.0
  - notifications: 4.0.0
  - oauth2: 1.19.1
  - password_policy: 3.0.0
  - privacy: 3.0.0
  - profile: 1.0.0
  - provisioning_api: 1.21.0
  - recommendations: 4.0.0
  - related_resources: 2.0.0
  - serverinfo: 3.0.0
  - settings: 1.14.0
  - sharebymail: 1.21.0
  - support: 3.0.0
  - survey_client: 3.0.0
  - systemtags: 1.21.1
  - text: 5.0.0
  - theming: 2.6.1
  - twofactor_backupcodes: 1.20.0
  - updatenotification: 1.21.0
  - user_status: 1.11.0
  - viewer: 4.0.0
  - weather_status: 1.11.0
  - webhook_listeners: 1.2.0
  - workflowengine: 2.13.0
Disabled:
  - admin_audit: 1.21.0
  - circles: 31.0.0-dev.0 (installed 30.0.0)
  - encryption: 2.19.0
  - files_external: 1.23.0
  - photos: 4.0.0-dev.1 (installed 3.0.2)
  - suspicious_login: 9.0.1
  - twofactor_nextcloud_notification: 5.0.0
  - twofactor_totp: 13.0.0-dev.0
  - user_ldap: 1.22.0

PHP Information

make · March 15, 2025, 8:53pm

I made more trials in between:

So far it looks like the issue is not related to my Nextcloud installations. This, because I have he same issue when I try to synchronize a larger amount of files to a Nextcloud which is hosted at a professional provider (1-Click-Install).

Now I think that issue is related to the Windows version of the desktop client.

As written above it works from a Linux laptop. I thought that might be because of the slow WiFi connection not causing so much load.

But today I made a test from a Linux VM on the same Proxmox server. From this machine it works perfect, too.

Today’s test files were photos and videos, 319 files / 3.7 GB

Versions of the desktop client:

Linux Mint VM, client 3.11.0-1.1build4
Windows 10 pro 22H2, client 3.16.1

Are there any tips on how I can narrow down the problem?

make · March 16, 2025, 9:20am

Finally I found the solution here:

This fixed my problem.