@KarlF12 I appreciate your help. from my feeling disk speed is not an issue as big files transfer fast, but I would like to proof/verify the assumptions. Copying the data from/to the system via NFS or SMB is fast and saturates the 1GBit/s network adapter. Could you point me to the way how I could verify the problem is caused by slow disk access? Is there any logging I can see delays when the system access the disk/database?
download via Browser
Upload via Browser
I made a test and uploaded a video vile with 134MB with a browser - it took~20 sec with very little pauses every 10MB (chunks) the speed shown in the progress bar was 3MB/s to 20MB/s. The file appeared on the local directory 30 sec later - sync took ~6 sec according to the client log, which is absolutely acceptable and shows disk and network perform fast. The problem I have is when lot of small files exist on a directory it takes ages…
2020-10-27 13:30:03:934 [ info nextcloud.gui.application ]: Sync state changed for folder "https://<my.nextcloud.fqdn>/remote.php/dav/files/<user>/Documents" : "Sync Running"
2020-10-27 13:30:03:935 [ info nextcloud.sync.propagator ]: Starting INSTRUCTION_NEW propagation of "Ask me anything, with Mark Russinovich on cloud, Sysinternals, security, and more-P1pk9VBdwUs.mp4" by OCC::PropagateDownloadFile(0x219c4d63090)
2020-10-27 13:30:03:936 [ info nextcloud.sync.accessmanager ]: 2 "" "https://<my.nextcloud.fqdn>/remote.php/dav/files/<user>/Documents/Ask me anything, with Mark Russinovich on cloud, Sysinternals, security, and more-P1pk9VBdwUs.mp4" has X-Request-ID "d30cff50-5b8f-4fb6-92f8-2a3fc1ce6c0a"
2020-10-27 13:30:03:937 [ info nextcloud.sync.networkjob ]: OCC::GETFileJob created for "https://<my.nextcloud.fqdn>" + "/Documents/Ask me anything, with Mark Russinovich on cloud, Sysinternals, security, and more-P1pk9VBdwUs.mp4" "OCC::PropagateDownloadFile"
2020-10-27 13:30:04:491 [ info nextcloud.sync.credentials.webflow ]: request finished
2020-10-27 13:30:04:491 [ info nextcloud.sync.networkjob.etag ]: Request Etag of QUrl("https://<my.nextcloud.fqdn>/remote.php/dav/files/<user>/Contacts") FINISHED WITH STATUS "OK"
2020-10-27 13:30:08:313 [ info nextcloud.sync.credentials.webflow ]: request finished
2020-10-27 13:30:08:326 [ info nextcloud.sync.checksums ]: Computing "SHA1" checksum of "W:/Nextcloud/Documents/.Ask me anything, with Mark Russinovich on cloud, Sysinternals, security, and more-P1pk9VBdwUs.mp4.~fa9" in a thread
2020-10-27 13:30:08:660 [ info nextcloud.sync.database ]: Updating file record for path: "Ask me anything, with Mark Russinovich on cloud, Sysinternals, security, and more-P1pk9VBdwUs.mp4" inode: 53 modtime: 1507790579 type: 0 etag: "e8cc653832d587e1493e7a777b34c60f" fileId: "00475173ocshh0oigtc5" remotePerm: "WDNVR" fileSize: 140892641 checksum: "SHA1:d2021affa8db72d65f0220b44f07004a74eef3e9" e2eMangledName: "" isE2eEncrypted: false
2020-10-27 13:30:08:662 [ info nextcloud.sync.propagator ]: Completed propagation of "Ask me anything, with Mark Russinovich on cloud, Sysinternals, security, and more-P1pk9VBdwUs.mp4" by OCC::PropagateDownloadFile(0x219c4d63090) with status 4
2020-10-27 13:30:08:662 [ warning nextcloud.gui.activity ]: Item "Ask me anything, with Mark Russinovich on cloud, Sysinternals, security, and more-P1pk9VBdwUs.mp4" retrieved resulted in ""
2020-10-27 13:30:08:663 [ warning nextcloud.gui.activity ]: Item "Ask me anything, with Mark Russinovich on cloud, Sysinternals, security, and more-P1pk9VBdwUs.mp4" retrieved successfully.
2020-10-27 13:30:08:663 [ info nextcloud.gui.activity ]: Successfully added to the activity list: ""
2020-10-27 13:30:08:664 [ info nextcloud.gui.folderwatcher ]: Detected changes in paths: QSet("W:/Nextcloud/Documents/Ask me anything, with Mark Russinovich on cloud, Sysinternals, security, and more-P1pk9VBdwUs.mp4")
2020-10-27 13:30:08:665 [ info nextcloud.sync.database ]: Closing DB "C:/Users/<user>/AppData/Roaming/Nextcloud/._sync_6d0d8033c51e.db"
2020-10-27 13:30:08:675 [ info nextcloud.sync.engine ]: CSync run took 6145 ms
from the log it look like the client i doing many many thing just to replicate one file - is it expected?
The file appeared on the local client few seconds later. I checked the client log during the sync and see it executed bunch SQL queries for ALL existing files in the directory where I put the upload is this expected?
2020-10-27 13:16:02:962 [ info nextcloud.sync.csync.updater ]: Database entry found for Readme.md, compare: 1597168289 <-> 1597168289, etag: 0aba3361429e9afd483f17bb8be7644a <-> 0aba3361429e9afd483f17bb8be7644a, inode: 0 <-> 1116, size: 136 <-> 136, perms: 9f <-> 9f, checksum: <-> , ignore: 0, e2e:
2020-10-27 13:16:02:962 [ info nextcloud.sync.csync.updater ]: file: Readme.md, instruction: INSTRUCTION_NONE <<=
2020-10-27 13:16:02:962 [ debug nextcloud.sync.csync.updater ] [ csync_walker ]: file: Welcome to Nextcloud Hub.docx [file_id=00000210ocshh0oigtc5 size=25150]
2020-10-27 13:16:02:963 [ debug nextcloud.sync.database.sql ] [ OCC::SqlQuery::bindValue ]: SQL bind 1 QVariant(qlonglong, 3622897607555501721)
2020-10-27 13:16:02:963 [ debug nextcloud.sync.database.sql ] [ OCC::SqlQuery::exec ]: SQL exec "SELECT path, inode, modtime, type, md5, fileid, remotePerm, filesize, ignoredChildrenRemote, contentchecksumtype.name || ':' || contentChecksum, e2eMangledName, isE2eEncrypted FROM metadata LEFT JOIN checksumtype as contentchecksumtype ON metadata.contentChecksumTypeId == contentchecksumtype.id WHERE phash=?1"
I already tuned my mysql with the following settings:
innodb_flush_log_at_trx_commit = 2
innodb_flush_log_at_timeout = 5
max_connections = 250
which made the system feel little more responsive but still not really good to sync thousands of files. having max_connections = 100 I had an error in the logs that DB stopped responding because of “too many connections” - why does the system with 2 users and 3 clients need more then 100 sql connections?