Nextcloud client crashes with robocopy

📱 Desktop & mobile clients 💻 Desktop
1

This is 100% repeatable both my client Nextcloud instance and on my lab environment. In a perverse way, I’m pleased I’m managing to repeat it in my lab as that means is support want to engage, I can try various things.

As I’ve mentioned elsewhere, my main Nextcloud client has sadly reached the end of their tether and has asked me to migrate it to SharePoint. A fun task in itself. But a core way to do this is to use robocopy to copy from the Nextcloud sync folder to the OneDrive sync folder - and spend a lot of time looking at downloads & uploads. 600GB to copy here.

I started with a relative small folder containing 961MB in 2,643 files. Small enough to allow my script to be tested but big enough to stress is.

Observation: Nextclient client kept crashing during the execution of a command like this:

robocopy “E:\Data\Nextcloud\Malt\Test1” “E:\Temp\Test1” /ndl /xj /mt /r:0 /np /mir /copy:DTA

A pretty innocuous robocopy command used many times by IT people over the years. What happens is that it merrily starts copying but after a while robocopy starts throwing up “The cloud file provider exited unexpectedly” errors and the Nextcloud client crashes/exits. Note that I gave up copying to OneDrive - just copying here to a local drive on my PC.

If I restart Nextcloud and re-run the command, a few more files are copied. Repeated it several times - doesn’t always crash at the same place.

If I tell Nextcloud to keep all the files in Test1 first, the copy works fine. The problem only occurs when the files copied are been downloaded first by Nextcloud.

I can’t supply logs right now because the folder I’m copying is actually a very sensitive folder. Even the folder and file names are sensitive. However, I’ve just spent an hour or so writing a PowerShell script that replaces the contents with random data and scrambles the file/folder names. Once I’ve worked out how to zap all previous logs, I’ll supply the logs.

This isn’t the first time I’ve seen Nextcloud client randomly crashing when lots activity is carried out.

I’m going to repeat the test without the /MT flag - that causes robocopy to use multiple threads to copy multiple files at once.

2

Nextcloud 28.0.2 on server, client 3.12.0.

Later… I’ve just reset everything and re-run robocopy without the /mt switch so it copies one file at once - that worked. Will do a few more runs before can say for certain but at first it looks like Nextclient crashes when there are many multiple file requests at the same time.

Later still… I’ve just installed Nextcloud client on a my plain vanilla Windows 10 VM. Nothing but Windows on there… run robocopy with /mt and it didn’t get very far at all before the client crashed:

image
image816Ă—792 50.1 KB

I took a snapshot of this VM before the 1st copy so will rewind and take a video.

Well that wasn’t hard to repeat. Here is a video. Nextcloud client crashes almost immediately:

Video of Nextcloud client crash using robocopy

3

Okay to make this totally repeatable, try using robocopy with /mt:64 and it crashes pretty much immediately on my main PC.

On the test VM above, it crashes even with /mt omitted… it can’t even handle single thread copy…

This is the crash from the event log. Not much use:

Faulting application name: nextcloud.exe, version: 3.12.0.55125, time stamp: 0x65cb7aa4
Faulting module name: Qt5Qml.dll, version: 5.15.10.0, time stamp: 0x652318d0
Exception code: 0xc0000005
Fault offset: 0x0000000000176cb0
Faulting process ID: 0x27c0
Faulting application start time: 0x01da65de7a3cb30b
Faulting application path: C:\Program Files\Nextcloud\nextcloud.exe
Faulting module path: C:\Program Files\Nextcloud\Qt5Qml.dll
Report ID: 2d85c366-547b-4784-b842-b88dbd8f3c42
Faulting package full name:
Faulting package-relative application ID:

4

I deleted the logs, started Nextcloud and ran robocopy with /mt:64 which causes immediate crash. A link to my OneDrive folder as it’s 400,00 lines long even though it was only running for a minute or two:

https://maltsystems-my.sharepoint.com/:t:/g/personal/rob_nicholson_maltsystems_co_uk/EedQ1fEOGOZHpdKlSCDF16MB07ewB97YIsv_DYdfp-MoRQ?e=M3nKnK

5

We appreciate that you’re still letting us participate in your customers’ migration away from Nextcloud, and all the effort you put in your testing, but honestly, with all the posts you’ve made here and everything you’ve tried, have you ever tried just downgrading the desktop client to an older version?

Really? You’re experminting with robocopy to migarte files from one server to another via a desktop PC, by using the respective desktop clients that are using Virtual Files, managed by the MS Cloud File Provider API, which then causes one of the desktop clients to crash, and now you’re trying to stress it even more by using multiple threads.

Why not just let the nextcloud client download all the files first, and then move it over to the folder of the one drive client, which then also might be done by robocopy if you absolutely have to :wink: Or maybe use WebDAV to download the files, or heck start a samba server on the nextcloud server and mount it on the windows machine. :wink:

Sure, for local files to another local disk or USB disk or to a SAMBA share, but certainly not for folders containing virtual files managed by the Cloud File Provider API to another folder managed by the Cloud File Provider API, of which robocopy clearly is not aware of and doesn’t know how to handle it, according to the yellow warning in your video. :wink:

6

I’ve just tried that - gone back to 3.10.0 from last September. Same problem.

7

OK, sorry, I must have missed that, and I’m not saying that the desktop clients aren’t a bit buggier than they should be. But I don’t use virtual files, and I don’t use Windows, so there’s that :wink:

But still, using robocopy to copy files from one folder to another that both are using Virtaul files managed by the Cloud File Provider API isn’t probably the best idea… :wink:

EDIT: I also found this: windows backup - Robocopy issues with cloud files - Super User So there is at least one person that expirienced a similiar issue, that doesn’t involve the Nextcloud client, but robocopy, VirtualFiles and the Cloud Provider API.

8

Really? You’re experminting with robocopy to migarte files from one server to another via a desktop PC, by using the respective desktop clients that are using Virtual Files, managed by the MS Cloud File Provider API, which then causes one of the desktop clients to crash, and now you’re trying to stress it even more by using multiple threads.

I’m not experimenting - I’m migrating the data. I switched to the lab and experimenting when the Nextcloud client crash was repeatable. The fact that robocopy then throws lots of client errors is because the cloud provider has just crashed. I’ve migrated PB of data using robocopy and relying on the sync.

The same crash occurs when copying from one Nextcloud folder to another using robocopy. With the /mt flag, robocopy really stresses the cloud provider. It looks like the Nextcloud client really can’t handle multiple I/O requests. This rarely occurs in normal operation with the occasional save or File Explorer copy.

Why not just let the nextcloud client download all the files first, and then move it over to the folder of the one drive client, which then also might be done by robocopy if you absolutely have to :wink: Or maybe use WebDAV to download the files, or heck start a samba server on the nextcloud server and mount it on the windows machine. :wink:

Now I know the client can’t handle multiple I/O requests at once, I’ll have to do this. For this particular client, I’ve actually got a copy of their data on local Linux VM and can probably use that for the initial seed.

Sure, for local files to another local disk or USB disk or to a SAMBA share, but certainly not for folders containing virtual files managed by the Cloud File Provider API to another folder managed by the Cloud File Provider API, of which robocopy clearly is not aware of and doesn’t know how to handle it, according to the yellow warning in your video. :wink:

Robocopy is fully aware of the error as it throws the error message about the cloud provider not working. It then carries on to try the next file hence the multiple errors. Whatever one throws at it, the client should not crash with an application fault.

The same “cloud provider” errors occur with other cloud systems inc. OneDrive. It occasionally can’t keep up with the copy demands and throws these errors. Robocopy catches them and carries on. Run it again and it mirrors the files that failed. But at no point does OneDrive (or Dropbox or Google Drive - all used with robocopy) crash.

9

Yes, cloud provider errors occur with nearly all virtual file systems due to the intrinsic nature of the design. They all have their own quirks. For example, when you upload an Office file using OneDrive, it then downloads immediately with some extra metadata baggage. This means that any backup software that checks the timestamp and/or file size often copies the file again as it thinks it’s changed. Many cloud systems don’t preserve the milliseconds value of the modified timestamp which once again makes the backup systems think it’s changed.

I’m not the greatest fan of virtual file systems either coming from the good old days of LAN based file servers with, you know, real file locking that prevent others editing a file at the same time. None of this file clash malarkey :slight_smile:

But those days are long gone so we’re stuck now. Virtual is implemented because if files are going to have to sync, the local laptop 240GB SSD is going to fill up really quickly if the server has several TB of data.

Personally I think they went down the wrong avenue. Webdav should have been continued to be developed esp. with the increase in internet speed. Many WAN connections these days are faster than 10Mbps back in the old days of LAN.

10

It will happen, not in the form of webdav mounts, but by moving the applications to the web. Web-based applications are the future. Cloud sync is just an interim step for those who still need to use desktop applications and/or use files offline. There will be in-browser solutions for the latter, and the former will only be needed for very specialised applications. Standard office and business applications will be 99% web-based by 2030.

I mean, all the electron apps are already web-based, and it’s only a matter of time before more complex applications become web applications as well. Some of them already are, Google Workspace, MS Office online editors and even Collabora are already good enough for most usecases, and there are already many companies that no longer have MS Office installed on all their PCs.