# Syncing 25k small files - need faster solution **URL:** \ **Category:** ℹ️ Support\ **Tags:** desktop-client, performance\ **Created:** [October 19, 2025, 4:38pm UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301 "2025-10-19T16:38:20Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![Catrik](https://help.nextcloud.com/user_avatar/help.nextcloud.com/catrik/32/67577_2.png) [@Catrik](https://help.nextcloud.com/u/Catrik)\ **Post date:** [October 19, 2025, 4:38pm UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/1 "2025-10-19T16:38:21Z")
I have a folder with only 700 MB, but with some 25 000 files. This is not a problem in normal use when only a few files change then an now, but sometimes I update it to a new version and create a backup folder, and it takes literally days to sync them. What are my options here to make it faster? Turning off the client and manually uploading the files to the server via web browser does not help, because the client still checks them all for ages. Server is owned by me, running on Yunohost install. Clients are a W10 and W11 PC. ---
**Author:** ![wwe](https://help.nextcloud.com/user_avatar/help.nextcloud.com/wwe/32/72963_2.png) [@wwe](https://help.nextcloud.com/u/wwe)\ **Post date:** [October 19, 2025, 7:53pm UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/2 "2025-10-19T19:53:08Z")
Hello @Catrik, You started a topic in support category. Unfortunately you ignored the template and a lot of information to help you is missing. Please edit your original post and add all required details like Nextcloud version, webserver type and version, os version, related log file content. Use the [support template](https://help.nextcloud.com/t/support-template-please-use-this-when-you-request-support/137402). Without additional information the community members cannot help you. In my instance I see a sync speed of 10+ files per second so 25k should sync in less than an hour.. shortly I setup new client and t synced 500k-700 files in less than a day.. Regards, wwe ---
**Author:** ![Catrik](https://help.nextcloud.com/user_avatar/help.nextcloud.com/catrik/32/67577_2.png) [@Catrik](https://help.nextcloud.com/u/Catrik)\ **Post date:** [October 19, 2025, 8:47pm UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/3 "2025-10-19T20:47:02Z")
I should have clarified my question better. I intentionally ignored the template, because I’m not looking for system support, but rather another method to accomplish this specific syncing. Like how to ignore file checks for one specific time, as long as the file names match, or something like that. The files on the server are on a 2,5” HDD which probably explains the slow speed and it has already been looked at by a few people and no issues are found. So I’m really just looking for bandaid workaround and I did not see it relevant to include all the details and logs for that goal. ---
**Author:** ![wwe](https://help.nextcloud.com/user_avatar/help.nextcloud.com/wwe/32/72963_2.png) [@wwe](https://help.nextcloud.com/u/wwe)\ **Post date:** [October 19, 2025, 9:09pm UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/4 "2025-10-19T21:09:30Z")
I see your goal more or less like “I look for support to overcome this system” but obviously this is out of scope here. in fact Nextcloud is not the fastest sync tool in the world for .. reasons.. - if your personal need differs from what it provides - feel free to choose another one. ---
**Author:** ![Catrik](https://help.nextcloud.com/user_avatar/help.nextcloud.com/catrik/32/67577_2.png) [@Catrik](https://help.nextcloud.com/u/Catrik)\ **Post date:** [October 19, 2025, 9:25pm UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/5 "2025-10-19T21:25:29Z")
Out of scope because you personally do not want to help “overcome the system”, or not allowed to discuss it at all in this forum? ---
**Author:** ![Ok\_Know](https://help.nextcloud.com/user_avatar/help.nextcloud.com/ok_know/32/69839_2.png) [@Ok\_Know](https://help.nextcloud.com/u/Ok_Know)\ **Post date:** [October 19, 2025, 11:47pm UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/6 "2025-10-19T23:47:23Z")
![image](https://help.nextcloud.com/uploads/default/original/3X/3/d/3d31fd99bacfbe78e52f7bb2a0aff1205a1c52f9.png) ![image](https://help.nextcloud.com/uploads/default/original/3X/4/d/4d109ec1c388c7144a6eba49a4b83a975a1f70a4.png)Hello, maybe you can try to modify the database’s my.cnf to increase these parameters. This is my configuration. I have only tried to upload a CS1.6 folder with 5600 files in it. After I modified the database parameters, the response was quite fast. ---
**Author:** ![Ok\_Know](https://help.nextcloud.com/user_avatar/help.nextcloud.com/ok_know/32/69839_2.png) [@Ok\_Know](https://help.nextcloud.com/u/Ok_Know)\ **Post date:** [October 19, 2025, 11:51pm UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/7 "2025-10-19T23:51:26Z")
Uploading this folder (800mb/5600 files) in nextcloud webui takes about 5-7 minutes. It will wait for a while and then start uploading. It will be faster if you use the client synchronization method ---
**Author:** ![bb77](https://help.nextcloud.com/letter_avatar/bb77/32/5_5575768a8748004e209b776fc1b2916d.png) [@bb77](https://help.nextcloud.com/u/bb77)\ **Post date:** [October 20, 2025, 5:38am UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/8 "2025-10-20T05:38:46Z")
> [@Catrik](#): > > Out of scope because you personally do not want to help “overcome the system”, or not allowed to discuss it at all in this forum? What he probably meant is that Nextcloud itself has certain limitations that can’t be overcome, except by using an alternative, and this discussion would go beyond the scope of this forum. Also, you haven’t quantified ‘slow’, which makes this discussion somewhat pointless in the first place. I mean, what does ‘slow’ mean? Five minutes, five hours, five days or five weeks? 😉 Without any details, we can only make general statements, such as it being normal (at least to a certain extent) for uploading 25,000 files to take some time, particularly with rotating hard drives. Some general things you could do, if you already optimzed the software as good as possible, and it’s still to slow: - Buy SSDs or generally faster hardware. However, it will always be considerably slower to upload many small files than fewer large ones. It will probably be faster with SSDs than with a spinning 2.5-inch drive though. 😉 - Try upload the files directly to the data folder (via rsync or another tool) and then use `occ files:scan` to make Nextcloud aware of them: [Using the occ command — Nextcloud latest Administration Manual latest documentation](https://docs.nextcloud.com/server/stable/admin_manual/occ_command.html#file-operations-label). However, that will likely take some time as well, and I’m not sure if the total time taken for the upload and scanning of the files would end up being quicker. - Use an SMB or NFS share or an external disk and add it as external storage to Nextcloud. ---
**Author:** ![tflidd](https://help.nextcloud.com/letter_avatar/tflidd/32/5_5575768a8748004e209b776fc1b2916d.png) [@tflidd](https://help.nextcloud.com/u/tflidd)\ **Post date:** [October 20, 2025, 12:06pm UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/9 "2025-10-20T12:06:14Z")
If you haven’t done anything with the database config, @Ok_Know gives you a few good pointers. Also file locking should be done with a redis cache. [Some time back](https://github.com/owncloud/core/issues/20967#issuecomment-205474772), I managed to get a few thousand files per minute, and even a good 100 per minute on a RPi2. If you want to know more, you need to share more details. And if you managed to improve it, let us know what made the difference. ---
**Author:** ![Catrik](https://help.nextcloud.com/user_avatar/help.nextcloud.com/catrik/32/67577_2.png) [@Catrik](https://help.nextcloud.com/u/Catrik)\ **Post date:** [October 21, 2025, 7:51pm UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/10 "2025-10-21T19:51:02Z")
I will try this asap. What does this exactly do, are there any downsides for changing the parameters? ---
**Author:** ![Rob4Bob](https://help.nextcloud.com/letter_avatar/rob4bob/32/5_5575768a8748004e209b776fc1b2916d.png) [@Rob4Bob](https://help.nextcloud.com/u/Rob4Bob)\ **Post date:** [October 22, 2025, 6:31am UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/11 "2025-10-22T06:31:56Z")
Be sure to check whether your current Nextcloud Desktop Client is limited in speed by one of the latest updates. In the connection settings, the values for upload and download bandwidth were automatically set to “Limit automatically” due to a bug. For me, this led to exactly the performance problem you described with many small files. ![upload01](https://help.nextcloud.com/uploads/default/original/3X/8/4/84b2cef3f8e89089e166ac67a69686af91ea4fd1.jpeg) ---
**Author:** ![bb77](https://help.nextcloud.com/letter_avatar/bb77/32/5_5575768a8748004e209b776fc1b2916d.png) [@bb77](https://help.nextcloud.com/u/bb77)\ **Post date:** [October 22, 2025, 7:36am UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/12 "2025-10-22T07:36:34Z")
Indeed, it was set that way for me, too. I wouldn’t have noticed if you hadn’t posted about it. Thanks! I did a little digging and found the following thread: > [@Very slow per file action since 3.17.0](https://help.nextcloud.com/t/very-slow-per-file-action-since-3-17-0/230994): > > Hello EDIT: Issues are since 3.17.0. I’ve confirmed this on different server and different clients. When installing 3.16.4 everything is good. Whats in common between my servers is TruneNAS. I’ve updated docker app to latest version with no success. Still a looot of time between files. I’ve updated to version 3.17.1 on Wiundows 11, and since then, I’ve been experiencing major performance issues, both on local and internet networks. When downloading or uploading a single large file, everything… Related Bug Reports: > > > \### ⚠️ Before submitting, please verify the following: ⚠️ > > \- \[x\] This is a \*\*bug…\*\*, not a question or a configuration issue. > \- \[x\] This issue is \*\*not\*\* already reported on Github (I've searched it). > \- \[x\] Nextcloud Server and Desktop Client are \*\*up to date\*\*. See \[Server Maintenance and Release Schedule\](https://github.com/nextcloud/server/wiki/Maintenance-and-Release-Schedule) and \[Desktop Releases\](https://nextcloud.com/install/#install-clients) for supported versions. > \- \[x\] I agree to follow Nextcloud's \[Code of Conduct\](https://nextcloud.com/contribute/code-of-conduct/) > > \### Bug description > > Since 3.17 upgrade, the sync is extremely slow. > > \### Steps to reproduce > > 1. Upgrade from 3.16.6 to 3.17 > 2. Sync a folder > > \### Expected behavior > > No sync regression > > \### Which files are affected by this bug > > all files, even small ones > > \### Operating system > > Linux > > \### Which version of the operating system you are running. > > Arch > > \### Package > > Distro package manager > > \### Nextcloud Server version > > 31.0.9 > > \### Nextcloud Desktop Client version > > 3.17.1 > > \### Is this bug present after an update or on a fresh install? > > Updated to a major version (ex. 3.16.3 to 3.17.0) > > \### Are you using the Nextcloud Server Encryption module? > > Encryption is Disabled > > \### Are you using an external user-backend? > > \- \[\] Default internal user-backend > \- \[\] LDAP/ Active Directory > \- \[\] SSO - SAML > \- \[x\] Other > > \### Nextcloud Server logs > > \`\`\`shell > none > \`\`\` > > \### Additional info > > \_No response\_ > > > \### ⚠️ Before submitting, please verify the following: ⚠️ > > \- \[x\] This is a \*\*bug…\*\*, not a question or a configuration issue. > \- \[x\] This issue is \*\*not\*\* already reported on Github (I've searched it). > \- \[x\] Nextcloud Server and Desktop Client are \*\*up to date\*\*. See \[Server Maintenance and Release Schedule\](https://github.com/nextcloud/server/wiki/Maintenance-and-Release-Schedule) and \[Desktop Releases\](https://nextcloud.com/install/#install-clients) for supported versions. > \- \[x\] I agree to follow Nextcloud's \[Code of Conduct\](https://nextcloud.com/contribute/code-of-conduct/) > > \### Bug description > > When downloading or uploading a single large file, everything works fine and the speed is normal. However, when transferring multiple smaller files, it creates one file every few seconds, regardless of the file size. I can see in logs every one of them. > I’ve updated to 3.17.2 and still have the same issue. I have to use 3.16.6. > I’ve checked on two different servers, (They are on different networks, and hardware, locally or internet, it doesn’t matter.) > The only thing in common is that both servers are running as Docker containers on TrueNAS. > Every file takes a few seconds to start syncing, even if its few KBs. > > \### Steps to reproduce > > Just syncing anything > > > \### Expected behavior > > Normal speed, like it used to be before 3.17.0 client. > > \### Which files are affected by this bug > > Its not particular file > > \### Operating system > > Other (e.g. FreeBSD) > > \### Which version of the operating system you are running. > > TrueNAS-24.10.2.2 > > \### Package > > Other > > \### Nextcloud Server version > > 31.0.8 > > \### Nextcloud Desktop Client version > > 3.17.0 and above > > \### Is this bug present after an update or on a fresh install? > > Updated to a major version (ex. 3.16.3 to 3.17.0) > > \### Are you using the Nextcloud Server Encryption module? > > Encryption is Enabled > > \### Are you using an external user-backend? > > \- \[\] Default internal user-backend > \- \[\] LDAP/ Active Directory > \- \[\] SSO - SAML > \- \[\] Other > > \### Nextcloud Server logs > > \`\`\`shell > > \`\`\` > > \### Additional info > > \_No response\_ > > > \### ⚠️ Before submitting, please verify the following: ⚠️ > > \- \[X\] This is a \*\*bug…\*\*, not a question or a configuration issue. > \- \[X\] This issue is \*\*not\*\* already reported on Github (I've searched it). > \- \[X\] Nextcloud Server and Desktop Client are \*\*up to date\*\*. See \[Server Maintenance and Release Schedule\](https://github.com/nextcloud/server/wiki/Maintenance-and-Release-Schedule) and \[Desktop Releases\](https://nextcloud.com/install/#install-clients) for supported versions. > \- \[X\] I agree to follow Nextcloud's \[Code of Conduct\](https://nextcloud.com/contribute/code-of-conduct/) > > \### Bug description > > > MacOS 14.0 > NC Desktop 3.10.1 > NC 27.0.2 > > Sync stops randomly - nevertheless I could sync very large folders with many GB and many files without problems for a while > suddenly stops syncing 5 4MB image files - and uses 100% CPU > look at the negative (!) number .... which was frozen. > I didn't find any error logs. > !\[Image 28 10 23 at 18 11\](https://github.com/nextcloud/desktop/assets/6248560/14ae901a-beb2-42b8-89dd-120f8ea37629) > > > \### Steps to reproduce > > 1. After update of MacOS to 14.0 on "old" chip and update of NC Desktop > 2. move files to synced directory > 3. NC Desktop stalls sometimes with 100% CPU > I tried the last 3 releases of NC Desktop - same problem > I tried to move 1-6 jpeg files to the synced directory, this works sometimes, sometimes not. > the sync directory is a group folder > ... > > > \### Expected behavior > > sync should (as before) sync all newly detected files. > ... > > > \### Which files are affected by this bug > > jpeg > > \### Operating system > > Mac OS > > \### Which version of the operating system you are running. > > 14.0.1 > > \### Package > > Appimage > > \### Nextcloud Server version > > 27.0.2 > > \### Nextcloud Desktop Client version > > 3.10, 3.10.1, 3.9.4 > > \### Is this bug present after an update or on a fresh install? > > Updated from a minor version (ex. 3.4.2 to 3.4.4) > > \### Are you using the Nextcloud Server Encryption module? > > Encryption is Disabled > > \### Are you using an external user-backend? > > \- \[\] Default internal user-backend > \- \[\] LDAP/ Active Directory > \- \[\] SSO - SAML > \- \[\] Other > > \### Nextcloud Server logs > > \`\`\`shell > no errors > \`\`\` > > > \### Additional info > > \_No response\_ ---
**Author:** ![tflidd](https://help.nextcloud.com/letter_avatar/tflidd/32/5_5575768a8748004e209b776fc1b2916d.png) [@tflidd](https://help.nextcloud.com/u/tflidd)\ **Post date:** [October 22, 2025, 8:33am UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/13 "2025-10-22T08:33:52Z")
> [@Catrik](#): > > are there any downsides for changing the parameters? If I remember right one was, the flush trx commit you potentially can loose a few queries if the system crashes. With the caches, connections, pool\_sizes, you need to adjust to your system, if they are too small, you have a bad performance, if they are too large, you consume too much RAM for the DB. There are tools to check the db cache settings for your use case (tuning-primer.sh, mysqltuner, …). ---
**Author:** ![Catrik](https://help.nextcloud.com/user_avatar/help.nextcloud.com/catrik/32/67577_2.png) [@Catrik](https://help.nextcloud.com/u/Catrik)\ **Post date:** [October 22, 2025, 3:25pm UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/14 "2025-10-22T15:25:36Z")
Well damn!!! That was it! Swithing the settings during syncing, it will immediately go from 1 file per second to tens per second. Thank you 💯 for pointing that out. Time estimation went from 3 days to 15 minutes. ---
**Author:** ![Ok\_Know](https://help.nextcloud.com/user_avatar/help.nextcloud.com/ok_know/32/69839_2.png) [@Ok\_Know](https://help.nextcloud.com/u/Ok_Know)\ **Post date:** [October 23, 2025, 8:46am UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/15 "2025-10-23T08:46:34Z")
Thank you. From today on, I will do my best to help more people.😀 ---
**Author:** ![disgustipated](https://help.nextcloud.com/letter_avatar/disgustipated/32/5_5575768a8748004e209b776fc1b2916d.png) [@disgustipated](https://help.nextcloud.com/u/disgustipated)\ **Post date:** [October 23, 2025, 9:23am UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/16 "2025-10-23T09:23:31Z")
I just happened to be reading this thread since it came up in my email summary, but my last sync from my photos offload from my camera had seemed pretty slow. i checked the bandwidth limit settings mentioned about thinking nah, im the only user on mine, that cant be it and im sure i had mine set to no limit… mine were set to limit automatically. they must have made that the default somehow with a client update. so odd. thank you all ---
**Author:** ![system](https://help.nextcloud.com/user_avatar/help.nextcloud.com/system/32/29333_2.png) [@system](https://help.nextcloud.com/u/system)\ **Post date:** [October 31, 2025, 9:24am UTC](https://help.nextcloud.com/t/syncing-25k-small-files-need-faster-solution/234301/17 "2025-10-31T09:24:02Z")
This topic was automatically closed 8 days after the last reply. New replies are no longer allowed.