Desktop client 3.4.0 destroys local time stamp and keeps uploading data to server

It needs more incentives. Sync performance on small files is an issue for a very long time and if you promote such a solution, people are perhaps more willing to test this. A long time ago, I tested a few this on a raspberry system myself to see the influence of redis and the database caches (Upload performance OC 8.2.1 (50 mysql queries per file) Ā· Issue #20967 Ā· owncloud/core Ā· GitHub). But it takes you easily a day and with all the follow up more time.

But this time, unless you are excited about the backup app (šŸ›  Nextcloud 23 RC 1 is here - testing time!), I wouldnā€™t get the idea of testing the sync process.

That was the major feature for this release: fast syncing of small files:

I hope this will then make testing consume a lot less timeā€¦

For you that is obvious. There is nothing in the RC1 news. Yes you can go back, you can check on github, ā€¦ You announce it one day, then there is a release, you donā€™t mention it again, even the discussion from last time with the announcement, you donā€™t repost to say that now there is something out, that people might want to test. Also tons of posts with sync performance with git repositories etc.
The thing is, you want people to test it, so donā€™t make it harder than it has to be.

The issue isnā€™t solved yet, but perhaps it can be figured out which platforms and environments need better testing.

yeah, I know I failed to mention that more clearly in the requests for testing. But my reply was more at your comment that testing takes a lot of time because the file sync is so slow - that should, going forward, be a lot faster, which hopefully makes testing less of a pain.

Ok, I didnā€™t mean that either :wink: Itā€™s just in general, setting up an extra environment, installing another client etc. takes a lot of time. If the upload takes 2h, I donā€™t have to sit there and watch. And if I do this effort, I either want to learn something, or get some benefit ot of it, like a new feature I was waiting for a long time.

I donā€™t think ā€œincentivesā€ would help. In my eyes testing is the hardest part of the development process as you donā€™t have clear defined goals like ā€œit compiles letā€™s sell itā€ ( only joke :wink: ). working through well designed runbook and carefully looking for issues which might be not obviousā€¦ especially client testing is really hard - running this in production is bad idea, finding good test coverage with production like usage (donā€™t forget all the moving parts like client reboots, hibernation, network outage, software updates, interaction with other software packages like anti-virus, different OSā€¦) is a nightmare - I donā€™t think you can count on this forum to find highly trained people + willing to spend big amount of their spare time + to test beta softwareā€¦

If you want good results - testing must be done by the most experienced people you have - this is the reason majority of testing should happen close to the devs - they are expected to be familiar with the product, know where to look in case of issues and can involve their colleagues fastā€¦ This forum can help when it comes to real-life testing on different platforms and cover all the variations of real-world installationsā€¦ but this must be the very last testing stage with solid, well tested versions you already consider production ready but need last confirmation.

Iā€™m the last one who blaims you for this specific bug - I know the competing market, demanding customers - but I think there is huge room for improvement in communications between Nextcloud GmbH and the community as discussed in depth here

4 Likes

I donā€™t think for regular updates. But regarding new feaures, they did it differently for the virtual file system, first it was an alpha-feature, disabled by default.

The problem with the community, you donā€™t know what systems are tested and which environments. As you said, you need to cover a huge amount of different scenarios.

The first releases after the fork, I had the impression that the whole story of reliable updates was improving. People trust Nextcloud with data, sure you should have backup and everything. But how often will you do a restore from backup before you ditch a product?

2 Likes

@jospoortvliet Youā€™re absolutely right we need more users to start testing the RC versions. Id be happy to volunteer my time to test and report issues.

Yes, RC version can cause issues with users data but for testing why not have a test profile, test computer or vm so it only has the RC version and incase of an issues you donā€™t have to worry about your data. Thats the whole point about the project we all have to put in some work. The devs can only test so much. I get we canā€™t get everyone onboard with testing but if we want a better product we should. Yes, I agree with everyone that V3.4.0 caused alot of issues and i was one of the admins that had to do a restore as well but at the end of the day we cant only blame Nextcloud. Thats the reason why this wonderful application is free and open source so the community ā€œusā€ can make the product better.

3 Likes

Thanks for the input all. This time I really shouldā€™ve done better by talking about the new bulk upload feature when asking about testing, as this feature was already talked about, that was really a mistake. It might have made a difference in testing.

In the future I will try to do that, too. Sadly generally we donā€™t yet want to spill the big features, while with small features weā€™d be happy to talk about the motivation is probably low, but thatā€™s a problem for the future.

In the mean time, I can tell you the desktop client team is debugging hard, working with the server team - there are both server and client issues, it seems, though I guess the server issues are in the new code for bulk upload. As soon as we have a fix weā€™ll publish it of course.

1 Like

Would it make sense to have the announcements around RC1, instead of Release? Maybe more people would be willing to test, if itā€™s clear, whatā€™s coming. Would also give you time to fix any critical bugs towards release and would be a good time to remind app developers to make sure their apps are compatible with the latest version / for release.

2 Likes

Yeah, we have done that in the past and might do it again, letā€™s see. It is also nice to be able to say ā€œand you can get it NOWā€ :wink:

anyhow, stay tuned - a test version of 3.4.1 will come soon.

1 Like

@jospoortvliet Thanks for the update and the explanation!

To answer your question about how to make more people test the RC, a few ideas:

  1. Provide a test server, including dummy files, folders, ā€¦ so that any tester can log in, install the client and start testing, without having the risk to lose any data (similar to what @AndyXheli proposes)
  2. I like the ideas above to communicate about RCs and new feature, to make it more appealing
  3. Propose a bonus to people who report feedback (notably the ones finding bugs). Bonus could be ā€œ2 hours of consulting/advice/audit provided by a Nextcloud engineerā€ (so that the tester receives advice on their own Nextcloud setup for example), a higher priority given to features suggested by the tester, ā€¦
  4. At each release, remind to the community that testing RCs is super important and that Nextcloud needs it

Regarding 3.4.1, do you plan to release a RC that we could test? [edit, oups, yes, you wrote it above :blush: ]

In general, I think this question is key and would deserve a dedicated discussion, with a lot of publicity so that the community gets really involved.

2 Likes

Just for info, 3.4.0 is still the version proposed in the stable Ubuntu PPA. https://launchpad.net/~nextcloud-devs/+archive/ubuntu/client

1 Like

Yes - and been working well here (Kubuntu 21.10). Maybe itā€™s a Windows related bug?

Hi,

A 3.4.1RC is here for testing. We improved the reliability of the sync process to better deal with the bug and it seems to fix it. Of course I still should point out that this is a test version, you should backup your data etc!

That said, the more realistic the test case, the more useful the testing is of courseā€¦ As usual, any feedback is welcome below here!

:apple: Mac OS X: https://cloud.nextcloud.com/s/Lp6kJ46KPdQLtFD
:penguin: Linux: https://cloud.nextcloud.com/s/3RY5FRxKymfYDBe
šŸŖŸ Windows: https://cloud.nextcloud.com/s/29zWbXKdTRbLey3

Edit:
Another update which should fix problems with weird dates.

šŸŖŸ Windows: https://cloud.nextcloud.com/s/b6wdJktaP9PtnHN
:penguin: Linux: https://cloud.nextcloud.com/s/WH6LmjDoxaYW6by
:apple: Mac OS X: https://cloud.nextcloud.com/s/6bCAQFT8cX8Z7wS

1 Like

Almost nobody at Nextcloud runs Windows and we had the problem ourselves too (we dogfood!) so Iā€™m reasonably sure itā€™s not Windows specific :wink:

And yes, the test version aboveā€™s been tested by us internally already.

Just a clarification - this only happens in conjunction with NC23, correct?

1 Like

I read some posts that this issue also occurs with 22.2.3 - i am running 21.0.7 and canā€™t see any problems ā€¦

You may look here: 3.4.0 keep synchronizing files after upgrade Ā· Issue #4016 Ā· nextcloud/desktop Ā· GitHub or here 3.4.0 keep synchronizing files after upgrade Ā· Issue #4016 Ā· nextcloud/desktop Ā· GitHub

Just upgrade from 3.3.6 to 3.4.1 on Win11 and so far no sync issues running