Hello, I’m writing this in case anyone else has experienced it. I’m about to stop using Nextcloud for the company that hired me and suggest another tool for storing their files. I haven’t been able to get the desktop client to work properly. I have good computing capacity, I’m an expert in cloud services, I have implemented a cluster, PostgreSQL in multi-region, and a primary storage in S3 for users to store millions of files. This client is extremely slow, with S3 I have had many problems. It works well for each user but when it comes to sharing folders, it is impossible to rename a folder. I have created an individual implementation without S3 as primary storage and the results are much better, but the Desktop client is still extremely slow and I can’t stop using VirtualDrive, which is where I think the problem lies.
I have purchased a Webdav client and it works well, reaching the conclusion that this method is quite bad. I don’t understand why it has to count the files before showing them to the user, even in this, Microsoft Sharepoint works better. It takes more than 80 minutes to display 40,000 files, while Webdav is immediate and even the Android client does not limit this, and the user does not have to wait.
Nextcloud has seemed like a good open-source tool to me, but this desktop client is really quite bad. Does anyone else have the same problem? Can someone tell me if there is a possibility to continue using VirtualDrive but without counting the files initially? Am I the only one suffering from this problem?
That is even without the S3 storage? And the initial sync only, or does it continue to be slow when checking for changes?
I suppose it the beginning it does a certain sync of a local db of the available files with the database. I have a similar amount of files but I don’t remember taking it so long (it was a long time ago that I did the initial sync). Just out of experience with Nextcloud, caches (redis/apcu) as well as the database caches themselves can have a huge impact on sync performance.
I don’t really know these details and here on the forum are mostly user questions. On the bugtracker are more details about the development and there are probably a few topics, where you see more details about the implementation. NC being open source, you can even make changes yourself and check if omitting certain steps improve the performance. Supposing it needs some kind of local db, and if it syncs everything in the beginning, you can’t just skip it, you might need to fill it slowly over time (as you navigate through the folders). But I have no idea how it is actually done, so it can be very different…
I have a similar platform, unfortunately I have users with millions of files, my nextcloud is on S3 as primary storage I have postgres and 4 clustered machines, I had around 350 simultaneous users and I had the same problem with this client simply does not work for large repositories, on the other hand s3 is quite slow for production.
I tried everything, I even bought webdrive as an alternative, but the price is not interesting anymore.
The use of regular expressions for a user “is complicated” and users will always tell you that Google Drive is better or that Onedrive or even dropbox is better.
My recommendation is: Abandon Nextcloud as main storage service, I use now OneDrive which is quite limited in amount of objects but is faster than Nextcloud desktop, but I still use Nextcloud only for archiving.
I think Nextcloud client can improve but I see that they depend a lot on virtual Drive and who knows if they should do it again which I don’t think.
I stopped insisting and abandoned this product as my primary storage solution. I wish you luck if you can continue.
Did you consider enterprise support? They are specialized on running Nextcloud in such environments, and if they get some money they can afford more on resources to fix your problems.
My Nextcloud has 45 million files 10TB in primary storage and the rest in buckets (external storage), the problem is not the amount of files you can host, the problem is the Nextcloud client, for example if you have a user who unfortunately needs to access 650,000 files, they must wait days for the client to show a file, this perception could be eliminated if all the files were simply shown at once without counting them as webdav or the android client does.
If I have for example 1000 files it works fine (except for the problems of S3 where if you share a folder, renaming it becomes impossible, understanding the architecture of S3… if the folder is yours there is no problem)
Nextcloud works very well standalone, in NAS in small environments, considering an enterprise solution makes sense if you don’t want to use Google or Microsoft, I came from Google Workspace and my users were happy with this client, at least with office the lock worked very well and warned that the file was being used, now I use Microsoft and I have done well with Onedrive, and I have left Nextcloud for archiving.
the first synchronisation is extremely slow, then it can work well, some users often have more than 400,000 documents, many hours for the files to be displayed.
I have looked at other forums and see that this is a common problem that has been asked for a few years now
Microsoft Onedrive works very well with its virtualdrive but it is too limited in number of objects, for me it is not an option, I will think then in another solution, it is sad to know that a tool as good as Nextcloud lacks such basic elements as properly lock Office files, and make a client that works like Google Drive, why at least does not work as the android client (osea for what and why they have to count the files?).
This is just a bug report from a user like you and me. They decided themselves to not use it with VirtualSync, there is no answer to that.
What I don’t know is to which extent the server might be involved, and if there is potential for improvements, or if all problems are just related to the client itself. It can help to look a bit in the topics on github, how they debugged things, what bugs are known and confirmed and help out if you can:
Is it that slow when using the Windows desktop client specifically?
I’m running a different setup, but have 1.1m files at 2.2 TB on the server, with virtual files activated, and when connecting via the Mac Nextcloud client, the time it takes from pressing the Nextcloud icon to finish syncing changes is around 10-11s. Of that, the sync process itself takes roughly 5-6s.
It happens the first time, then the client can work relatively well, but I have given up, I will try other similar tools, nextcloud is definitely not competitive with market solutions, lockiles does not work well, their client is slow and I do not see solutions in the short term, the web interface works very well for me even in S3 (except for renaming folders that have been shared with me).
finally my configuration was ubuntu dedicated server with 64GB RAM (all tuning recommendations ok) 8 Cpus, S3 primary Storage, LDAP (Azure AD) php 8.2 nginx reverse proxy, SSD,
I also tried to remove s3 and started a clean configuration with a fixed disk I put 2.2 TB with a single folder of 2.000.000 files, and it worked better the renaming is much faster, webdav clients do not crash among many things, but I still complain about the client, I do not like that I have to count the files and lockfile for me is necessary but this does not work.
actually the product on the web works fine, I argue is that the client is just bad and has a lot to improve, I tried webdav but microsoft insists on killing it, you have to modify the registry etc etc. Onedrive is very limited, I love Google Drive but I don’t need the Google suite…Anyway…Thanks a lot, maybe I’ll come back when the product works well.
Hi, if you are still around…if you mount nextcloud in GCP you are going to have headaches…the buckets if you use them as primary storage do not allow a user to rename inside a shared folder, if you use a bucket as external extorage you will not be able to delete or rename folders.
On amazon it works very well (considering transfer costs).
However, the client will still be extremely slow when you have thousands of files to sync.
I don’t use S3 (to me the whole point is self hosting), but regarding the client, I did want to add something. There seems to be a perception that the client should just display what is on the server, but there is actually more than that going on. Even with virtual files, it builds out the local directory structure and sets up for tracking changes, meaning it has to create an inventory of those files. This I assume is the “counting files” you refer to? Web and mobile clients don’t do this because they actually are just displaying what is on the server side.
For a setup of the size you describe, you should be getting professional support for Nextcloud from the company. If not then I would say you need to rethink how you approach your infrastructure services because you should always budget for support on critical infrastructure.
It’s also entirely possible that server side conditions and/or configuration are impacting your initial sync speed.
3 servers in cluster 120GB of RAM that are used to the maximum, postgresql, redis…1500 users, SSO…400 active sessions… 20 million files, I came from drobpox, now I’m going back. My mistake was not having seen the mess that has this client, if I have few files works fine … but when you have millions not … I have done tests comparing it with Onedrive and in my opinion definitely the desktop client nextcloud should do it again, I’m not the only one who complains about this, the lock to open files or does not work or is poorly implemented, Google Drive is an example of this, should create a plugin for office and avoid the lock as does at least google Drive, which by the way works wonderfully the user does not have to wait for your files appear. Anyway thanks to all, I retire with the hope that someday this wonderful tool and the whole set of applications will allow not to depend on products like Microsoft or google itself.
It is really interesting that you have no problem paying google, microsoft or dropbox but you complain about Nextcloud, and even with a clear business case in the back, you avoid getting enterprise support at all costs.
They have build the enterprise support in order to implement the solution in such customized and specific environments. I imagine, their university customers have tens or hundreds of thousands of users with probably a lot of files. Did you try to contact them: Get A Quote - Nextcloud
You can team up with other companies using Nextcloud for free and develop such a desktop client.
We are using Sharepoint, Teams and Ondrive. It works only with Microsoft Programs together. It does not work with other Programs as example, Corel Draw or Libre Office. It is a a toy for the Office workers. I think this Cloud Solutions are very good for work in a projekt and keep everything up to date… but it is not a file server. The Internet is way to slow for handle such big amounts of Data in folders.
The problem is not being able to pay for any product like the ones you mention; they all (except for Dropbox) fall short in capabilities, and that’s one of the things I admire most about Nextcloud.
My problem is not with Nextcloud server; the problem is this damn client, which doesn’t work correctly if you have to use many files. The Android version or any WebDAV client works better.
I don’t think that by paying Nextcloud they will come to solve the problem that everyone has with file locks. I’m not the only one asking them to review the virtual drive or come up with something better
Android as far as I know just browses the folder, it doesn’t keep an index itself. And you can use any other webdav client, Windows has a build-in version (with draw-backs, and I think they will stop the support at some time), mountain duck or netdrive even have a virtual drive implementation (not sure how well it works).
If you checkout the podcasts, release news, certain features in new versions, a lot of things were some requests of paying customers in the beginning. And if you want to hire developers, money can help I don’t know how that works in detail, but that is the reason you might want to reach out to them.
Problem on the forum is, we can complain all day, that won’t change anything. The community itself can do a few things to improve features, work on features/app or help testing, giving feedback etc. but we don’t steer the overall development of the company.
Microsoft Onedrive works very well with its virtualdrive but it is too limited in number of objects, for me it is not an option, I will think then in another solution, it is sad to know that a tool as good as Nextcloud lacks such basic elements as properly lock Office files, and make a client that works like Google Drive, why at least does not work as the android client (osea for what and why they have to count the files?).
I don’t know how that works in detail, but that is the reason you might want to reach out to them.