I think there will be no short term a fix to recover from this issue. I think at the moment restore is the best option you have. Fortunately this bug doesn’t cause loss of files itself… only metadata is (created/modified timestamp) is lost and there is useless network and storage consumption… but depending on your hosting model this could become a problem as well…
I created a feature request on Github to get some command line support for managing versions and trashbin so you could recover your files - please support it!
Maybe related is that the main dialog won’t open as well.
I’m on linux ubuntu and downloaded the app image 3.4.0. That seems to be working ok. main dialog opens and the sync has completed and now is idle so maybe it’s related to the build and dependenices at least for unbuntu focal since the app image build seems ok.
FYI I have 3.3.5 loaded on another machine. no issues with it.
Yah, restore is what I ended up doing. Thankfully the nature of this bug is it only effected files that were synced under 3.4.0. So first I downgraded back to 3.3.6 and restored only the groups of files that nextcloud kept retrying to sync.
It needs more incentives. Sync performance on small files is an issue for a very long time and if you promote such a solution, people are perhaps more willing to test this. A long time ago, I tested a few this on a raspberry system myself to see the influence of redis and the database caches (Upload performance OC 8.2.1 (50 mysql queries per file) · Issue #20967 · owncloud/core · GitHub). But it takes you easily a day and with all the follow up more time.
For you that is obvious. There is nothing in the RC1 news. Yes you can go back, you can check on github, … You announce it one day, then there is a release, you don’t mention it again, even the discussion from last time with the announcement, you don’t repost to say that now there is something out, that people might want to test. Also tons of posts with sync performance with git repositories etc.
The thing is, you want people to test it, so don’t make it harder than it has to be.
The issue isn’t solved yet, but perhaps it can be figured out which platforms and environments need better testing.
yeah, I know I failed to mention that more clearly in the requests for testing. But my reply was more at your comment that testing takes a lot of time because the file sync is so slow - that should, going forward, be a lot faster, which hopefully makes testing less of a pain.
Ok, I didn’t mean that either It’s just in general, setting up an extra environment, installing another client etc. takes a lot of time. If the upload takes 2h, I don’t have to sit there and watch. And if I do this effort, I either want to learn something, or get some benefit ot of it, like a new feature I was waiting for a long time.
I don’t think “incentives” would help. In my eyes testing is the hardest part of the development process as you don’t have clear defined goals like “it compiles let’s sell it” ( only joke ). working through well designed runbook and carefully looking for issues which might be not obvious… especially client testing is really hard - running this in production is bad idea, finding good test coverage with production like usage (don’t forget all the moving parts like client reboots, hibernation, network outage, software updates, interaction with other software packages like anti-virus, different OS…) is a nightmare - I don’t think you can count on this forum to find highly trained people+willing to spend big amount of their spare time+to test beta software…
If you want good results - testing must be done by the most experienced people you have - this is the reason majority of testing should happen close to the devs - they are expected to be familiar with the product, know where to look in case of issues and can involve their colleagues fast… This forum can help when it comes to real-life testing on different platforms and cover all the variations of real-world installations… but this must be the very last testing stage with solid, well tested versions you already consider production ready but need last confirmation.
I’m the last one who blaims you for this specific bug - I know the competing market, demanding customers - but I think there is huge room for improvement in communications between Nextcloud GmbH and the community as discussed in depth here
I don’t think for regular updates. But regarding new feaures, they did it differently for the virtual file system, first it was an alpha-feature, disabled by default.
The problem with the community, you don’t know what systems are tested and which environments. As you said, you need to cover a huge amount of different scenarios.
The first releases after the fork, I had the impression that the whole story of reliable updates was improving. People trust Nextcloud with data, sure you should have backup and everything. But how often will you do a restore from backup before you ditch a product?
@jospoortvliet You’re absolutely right we need more users to start testing the RC versions. Id be happy to volunteer my time to test and report issues.
Yes, RC version can cause issues with users data but for testing why not have a test profile, test computer or vm so it only has the RC version and incase of an issues you don’t have to worry about your data. Thats the whole point about the project we all have to put in some work. The devs can only test so much. I get we can’t get everyone onboard with testing but if we want a better product we should. Yes, I agree with everyone that V3.4.0 caused alot of issues and i was one of the admins that had to do a restore as well but at the end of the day we cant only blame Nextcloud. Thats the reason why this wonderful application is free and open source so the community “us” can make the product better.
Thanks for the input all. This time I really should’ve done better by talking about the new bulk upload feature when asking about testing, as this feature was already talked about, that was really a mistake. It might have made a difference in testing.
In the future I will try to do that, too. Sadly generally we don’t yet want to spill the big features, while with small features we’d be happy to talk about the motivation is probably low, but that’s a problem for the future.
In the mean time, I can tell you the desktop client team is debugging hard, working with the server team - there are both server and client issues, it seems, though I guess the server issues are in the new code for bulk upload. As soon as we have a fix we’ll publish it of course.
Would it make sense to have the announcements around RC1, instead of Release? Maybe more people would be willing to test, if it’s clear, what’s coming. Would also give you time to fix any critical bugs towards release and would be a good time to remind app developers to make sure their apps are compatible with the latest version / for release.
@jospoortvliet Thanks for the update and the explanation!
To answer your question about how to make more people test the RC, a few ideas:
Provide a test server, including dummy files, folders, … so that any tester can log in, install the client and start testing, without having the risk to lose any data (similar to what @AndyXheli proposes)
I like the ideas above to communicate about RCs and new feature, to make it more appealing
Propose a bonus to people who report feedback (notably the ones finding bugs). Bonus could be “2 hours of consulting/advice/audit provided by a Nextcloud engineer” (so that the tester receives advice on their own Nextcloud setup for example), a higher priority given to features suggested by the tester, …
At each release, remind to the community that testing RCs is super important and that Nextcloud needs it
Regarding 3.4.1, do you plan to release a RC that we could test? [edit, oups, yes, you wrote it above ]
In general, I think this question is key and would deserve a dedicated discussion, with a lot of publicity so that the community gets really involved.
It’s just in general, setting up an extra environment, installing another client etc. takes a lot of time. If the upload takes 2h, I don’t have to sit there and watch. And if I do this effort, I either want to learn something, or get some benefit ot of it, like a new feature I was waiting for a long time.
]