Slow and weird desktop client behavior (Linux AppImage 4.0.4)

đŸ“± Desktop & mobile clients đŸ’» Desktop
, ,
1

I am evaluating Nextcloud (as alternative for Seafile).

Basics

Server: Ubuntu 22.04.5 running Nextcloud AOI 32.0.3
8 cores > 70% idle, > 18 GB ram free: this should not be the bottleneck

Client: Mint 20.3 running Nextcloud-4.0.4-x86_64.AppImage
4 cores >80% idle, > 1GB RAM free
speedtest.net reports upload bandwidth of 28 Mbps (down 475 Mbps)
The client is using almost no CPU, as if it hangs.

I have two folder sync connections:

”etc” is a small folder that I used for experiments, nothing changed there recently, should be fully in sync

“Fotos” is a large folder with photos and some videos: 77476 files, 367GB. About 6.5GB has already been uploaded to the server, the rest is pending.

Observed behavior after starting the client and opening the “Nextcloud settings” GUI:

  • “etc” initially looked fully synced (as it should), now “Synchronizing files in local folder” like forever (> 30 minutes).
  • ”Fotos” also showed fully synced for a few moments, now “Downloading ” for over 30 minutes. Filename changes every 10 seconds or so.
  • Clicking the three dots after “Fotos” takes about 30 seconds just to open the popup menu

Expected behavior:

  • “etc” should be synced and remain synced
  • “Fotos” should not download anything (there is nothing new on the server) and start uploading fast (taking into account available bandwidth to server)
  • Menus should react fast (< 0.5s) on an almost idle computer

Additional info:

  • progress bars (almost) do not move at all
  • “general” menu also opens slowly: takes about 10s to open

image
image573×393 36.6 KB

2

I’d try first to sync the etc only and pause the Foto sync for the moment. And then check with the etc sync what might be the problem. Do you see network activity when it is syncing? What does the server logs say?

Ok, you know it but the client is perhaps trying to check this. Is it just small in size or also small in terms of number of files?

3

Did you try an older version to exclude a bug in the recent version?
https://github.com/nextcloud-releases/desktop/releases
(Expand “Assets” at the end of each “Contribution”-entry)

4

I killed the client, restarted it and then immediately tried to pause syncing of Fotos.
When I restart the client, it writes this to the terminal from which it is started:
```
nextcloud.gui.application: Migrating old config from “/home/johan/.local/share/Nextcloud” to “/home/johan/.config/Nextcloud”
nextcloud.gui.application: Failed to move the old config directory to its new location ( “/home/johan/.local/share/Nextcloud” to “/home/johan/.config/Nextcloud” )
nextcloud.gui.application: Will move the individual files QList(“etc_sync.log”, “Fotos_sync.log”, “Nextcloud_sync.log”)
nextcloud.gui.application: Fallback move of “etc_sync.log” also failed
nextcloud.gui.application: Fallback move of “Fotos_sync.log” also failed
nextcloud.gui.application: Fallback move of “Nextcloud_sync.log” also failed
```
and only a few minutes later, an icon appears in the system tray.

5 minutes after pausing the syncing of Fotos, it is still “syncing”, i.e. still Downloading.

So I decided to kill it and try an older version.

I downloaded Nextcloud-3.17.4-x86_64.AppImage. This produces the same output at startup:
```
nextcloud.gui.application: Migrating old config from “/home/johan/.local/share/Nextcloud” to “/home/johan/.config/Nextcloud”
nextcloud.gui.application: Failed to move the old config directory to its new location ( “/home/johan/.local/share/Nextcloud” to “/home/johan/.config/Nextcloud” )
nextcloud.gui.application: Will move the individual files QList(“etc_sync.log”, “Fotos_sync.log”, “Nextcloud_sync.log”)
nextcloud.gui.application: Fallback move of “etc_sync.log” also failed
nextcloud.gui.application: Fallback move of “Fotos_sync.log” also failed
nextcloud.gui.application: Fallback move of “Nextcloud_sync.log” also failed
```
A few minutes later, the systray icon appears. Right-click, or left-click + Open main dialog does nothing. Right-click + Settings opens the settings window. It reacts with the same as version 4.0.4 to open menus. This time, I was able to pause the syncing of Fotos in less than a minute. It continues to sync etc for a minute or so, then shows etc as synced. Small changes are synced almost immediately.

So I restarted syncing of Fotos. It shows Synchronizing files in local folder for a minute or so, then starts Downloading 
 just like version 4.0.4, and continues to do so for a very long time (it is still running).

FYI: etc is 404 KB spread over 22 folders/files.

Here is the current content of ~/.local/share/Nextcloud and ~/.config/Nextcloud:

johan@morla:~/work/flopro/etc\> l ~/.local/share/Nextcloud
total 748K
-rw-r--r-- 1 johan johan  24K Jan  5 17:58 etc_sync.log
-rw-r--r-- 1 johan johan 692K Jan  5 18:00 Fotos_sync.log
-rw-r--r-- 1 johan johan  20K Dec 29 11:14 Nextcloud_sync.log
johan@morla:~/work/flopro/etc\> l ~/.config/Nextcloud
total 792K
-rw-r--r-- 1 johan johan  302 Jan  5 17:37  cookies0.db
-rw-r--r-- 1 johan johan 2.8K Dec 27 20:02  etc_sync.log
-rw-r--r-- 1 johan johan 728K Dec 31 11:38  Fotos_sync.log
drwxr-xr-x 2 johan johan  12K Jan  5 18:00  logs
-rw-r--r-- 1 johan johan 2.0K Jan  5 17:59  nextcloud.cfg
-rw-r--r-- 1 johan johan  403 Dec 27 19:38  nextcloud.cfg.backup_20251227_193838
-rw-r--r-- 1 johan johan 2.0K Jan  5 17:38 'nextcloud.cfg.backup_20260105_173839_4.0.4 (build 35937)'
-rw-r--r-- 1 johan johan  21K May 24  2019  Nextcloud_sync.log
-rwxr--r-- 1 johan johan  371 May 17  2019  sync-exclude.lst

I have no clue why the client fails to remove ~/.local/share/Nextcloud; I guess I can remove it manually.

What else can I do to debug this, especially the syncing of Fotos?

5

I don’t know, how the Linux-clients work. Maybe the files still are locked by a background process after killing the desktop-client process. So moving the files may fail.

6

there is a longer bug report for this and no solution so far:

For >300 GB, I suppose it take quite some time. But do you see a reasonable data transfer?

7

No, and that is my main issue.

The client shows that it is downloading files, making no or almost no progress. It should be uploading files, and make reasonable progress. I can understand that maybe, when just started, the client wants to check for changes on the server before uploading local changes. But that shouldn’t take very long, and I know that there are no changes on the server. I waited for multiple hours, but noticed no real change.

I understand that uploading >300GB of data can take time, maybe even several days, even with good quality network connections (I have ~30Gbps upload, the server itself is a lot faster). That is not the issue. But now, I see no progress at all.

8

It might be better, to copy such a lot of files via SMB/scp or other into the data folder of the server, set correct file- and folder-permissions (see no 13.) from command-line and finally call occ files:scan

9

Thanks for the clarification. If your folder isn’t empty, it must first detect the situation on your client and on your server, and this part seems to be blocked. Now it depends, if you want to find the issue or just have a workaround.

Find the issue: dig a bit deeper, check the logfiles for some errors. If they are not specific to your system, check if you can find references on the bug tracker: GitHub · Where software is built, if not create a new topic with your scenario. It is extremely helpful, if you have a simple procedure to reproduce the error.
Don’t forget to check the server logs as well, perhaps you have missed something in your setup, and it is trying to do something in the background, or if you haven’t set up caches (shouldn’t be the case for AIO) that slows everything down.

Workaround: First sync with an empty local folder, make sure that the client and the server are in sync. Then move the files into the folder. You split the task, the client then knows that all of the files are new and need to be uploaded. This should then work, and if not, you have a nice scenario to reproduce a problem. If all the files at once are too hard, folder-by-folder can be easier.

Problem, in the end all of this should work without any problem, it is just the task of the sync client. Problem is that it can depend a lot on the server as well.

1 Like
10

I decided to just wait. This morning (about 25 days later), everything is synced.

I don’t see any specific relevant errors in the logfiles. It was just much slower than I expected, and while it was running, the UI was extremely sluggish.

To reproduce: add a folder sync connection for a very large folder, and let it sync (without first populating the server). Then watch the progress bar, and try to use the UI for “Nextcloud Settings”.

PS: I gave a good network bandwidth to the server: a simple copy (using scp) takes about 5 minutes for 1GB, or 30 hours for 367GB (3.3MB/s, or 26Mb/s, upload speed). 30 hours is still a long time, but with a responsive UI and progress bar, it would be doable.

1 Like
11

I really wonder how you made it possible to sync /etc-Folder with NC-Client on Linux. The Users have no write permission in /etc. The client (Nextcloud-4.0.6-x86_64.AppImage) does not accept for that reason /etc as folder for a sync.

etc
etc650×706 37 KB

So did you changed Owner of /etc or do you run Appimage as sudo?
Both is not recommended.

1 Like
12

Once again, someone seems to want to use the Nextcloud desktop client as a backup tool, but that’s simply not what it’s designed for. :wink:

@ygramoel The desktop client is intended to make files on your Nextcloud available on your desktop so that you can work with them locally on your PC and then synchronize changes or new files that you create in the Nextcloud folder on your PC to your Nextcloud, allowing you to continue working with them in the webUI or on other devices that synchronize or connect with your Nextcloud.

The Nextcloud Client is not a backup tool, and definitely not for system directories such as /etc.

13

@mornsgrans Agreed, but even then, I have a problem. Right now, all files are on the server as well as on my local PC. On my local PC, I re-installed the Nextcloud client, and recreated the folder sync connection.

Now the client probably needs to check that the local and remote folders have the same content. It is taking forever to do that (started yesterday, now at maybe 2%).

While it is doing that, the text underneath the progress bar reads “Downloading 
.”. This is confusing, as it is not actually downloading anything (I checked: none of the files in my local folder has changed).

Why does it take weeks just to establish that there are no changes, and why does the progress bar in the Nextcloud settings window show “Downloading” when it is not downloading anything???

14

I am absolutely not using it for /etc. I never synced /etc. I synced ~/work/etc, which is a small test folder nested in my home folder. Just for testing.

I fully agree that it is usually a bad idea to run Nextcloud as root or change the owner of /etc/

I would like to use Nextcloud to sync my photo collection between a few computers (including a laptop and a desktop). It is convenient to be able to access, re-arrange, 
 these photos from several locations. Admitted, this is also a form of backup: if I loose my laptop, or my desktop’s hard disk breaks down, I will not loose my photo collection.

I have previously used Seafile to sync these photos as well as a lot of other files (for the same purposes: convenient access from multiple computers as well as a form of backup of personal files). I would like to switch to Nextcloud, because of the many other functions it offers, that are not available in Seafile. Seafile also has its problems. It did however have less trouble syncing my photo collection.

15

It’s good that you’ve clarified that now. Since /etc is a system folder that exists on every Linux system, your previous information was unclear.

1 Like
16

I’m not sure, to be honest, but I’d say something is out of sync / broken here. Before you start tinkering too much and trying random things, and potentially end up loosing data, I would start over. :wink:

So here’s what I would do


  1. Log out of the client, and completely remove your account from the client.

  2. Quit the client (just to be extra safe :wink: )

  3. if you have enough disk space to keep a second copy temporarly, simply rename the local Nextcloud folder (the main folder where your Nextcloud file structure is synced to). Alternatively, if you don’t have enough space, you can move it to an external drive.

    If you are 100% sure that all files are present on the Nextcloud server — and ideally you also have a backup there — you could as well just delete the local folder.

  4. Start the client, and log in to your Nextcloud account again. This will create a new local Nextcloud folder, and all files will be freshly synchronized / downloaded from the server.

After that, I would strongly recommend keeping things as simple as possible. Don’t configure multiple folder pairs in different locations, unless you absolutely have to for some reason, and especially not in system directories.

If you want to backup system folders, use dedicated backup tool to back up those folders. There are plenty of backup tools that support WebDAV as a storage backend if you want to use your Nextcloud as a backup target. But please don’t sync those backup folders back to your PC again, exclude them from the Nextcloud client sync (uncheck them).

If the idea was to synchronize configuration files in /etc across multiple machines, I would drop that idea entirely. That only makes sense for very specific configurations and use cases (none come to mind right now, to be honest :wink:), and even then I would definitely not use the Nextcloud client for that purpose.

17

@bb77 Thank you for your suggestions.

In case you missed my previous post: I never tried to sync /etc. My first test folder was unfortunately and confusingly named etc . It has no relation with /etc. It was small and located somewhere in my home folder. Right now, I am not even syncing it.

My main concern is the Fotos folder, about 367GB. I have now tested two scenarios. Both are extremely slow (much slower than they should be), and in both cases, the NC client’s progress bar shows confusing or even wrong information. Also, during the operation, the NC client’s UI reacts very sluggishly.

Scenario 1: the Fotos folder is on my local disk, nothing on the NC server, and I create a folder sync connection. In this case, all 367GB need to be uploaded. With full use of the available bandwidth, this should take 30 hours. Several days would have been acceptable. It actually took 25 days.

Scenario 2: the Fotos folder is on my local disk and on the server, with exactly the same contents. NC client is freshly installed, after removing all NC settings files I could find, including those in the local Fotos folder. In this scenario, the NC client only needs to check that both folders are equal; no files need to be copied. This case is still running; after three days, the progress bar is somewhere between 5% and 10%.

Note that scenario 2 is comparable to manually uploading files to the server before creating the folder sync connection (suggestion by @Mornsgrans). Since this seems to take as much time as scenario 1, it doesn’t really help.

In both cases, the text below the progress bar shows “Downloading” all (or almost all) of the time. This is in both bases incorrect and confusing.

Also in both cases, simply creating the folder sync connection takes multiple minutes, during which the UI is frozen. I suspect that it scans the whole folder synchronously before handling more UI events.

When scenario 2 is finished, I can try the scenario you suggested (let’s call it scenario 3): start with files on the server only, and test how long it takes to download all of them. In this case, the label below the progress bar might actually be correct :slight_smile:

I would still very much like to understand why these operations are so slow, and why they make the UI so sluggish. Preferably without diving into the source code myself. I am still hoping that someone has enough internal knowledge of NC to figure this out. In general, I do like the functionality offered by NextCloud. There is no real competition, especially not if you are concerned about privacy, as I am 


18

Looking at the speed test info in your original post, am I understanding correctly that the server is in a data center outside your local network
?

If so, that would at least partly explain why it’s slow. An upload speed of 28 Mbps really isn’t that much. On top of that, there is always some overhead with web uploads / WebDAV, so you’ll probably never fully saturate the available bandwidth, especially when uploading a large number of smaller files, which is very likely the case with a 367 GB photo collection (probably tens of thousands of files).

Yeah, I would definitely expect that to be a lot faster, given a 475 Mbps download speed, which is roughly 16× your upload speed, unless there’s some other underlying issue affecting performance.

That might indeed be the case here, because the UI really shouldn’t become sluggish during the sync process. However, I can’t say that with complete confidence, because
a) I don’t use the AppImage but distro packages instead (Fedora and Arch in my case), and
b) I haven’t synced such a large amount of data in one go for quite some time.

The last time I did this was when I set up my current laptop from scratch. It downloaded around 200 GB in roughly 30 minutes, when I remeber correctly, though that was on my local network over a 1 Gbps wired connection.

So, yeah, maybe there’s a bug in the current client version or specifically in the AppImage that causes this. I’m not sure.

1 Like
19

True. The server is indeed in a data center, not in my LAN. 28Mbps is also the speed I measure while uploading a large (100MB) file to the same server (with scp), so it is a real speed, not theoretical.

I understand that NC will have some WebDAV overhead. However, the speed I observe is only around 1/10 of the number above. That’s a lot of overhead.

In scenario 2, no data needs to be copied, unless NC client copies the data to compare it. Doesn’t it use some kind of signature to detect changes before copying? Scenario 2 is as slow as scenario 1, so clearly, data copying is not the bottleneck here.

BTW: I switched to the NC client from the repo’s for scenario 2. This is my current client version:

Nextcloud version 4.0.6-20260122.174414.847472c1e7-1.0~jammy1
Using Qt 6.2.4, built against Qt 6.2.4
Using Qt platform plugin ‘xcb’
Using ‘OpenSSL 3.0.2 15 Mar 2022’
Running on Linux Mint 21, x86_64

The sluggish interface is probably even more annoying than the slow sync. Is there anything I can do to help identify the cause?

BTW: I talked to other IT’ers about this issue. I am not the only one who perceives syncing in Nextcloud as slow.

20

Yes, definitely too slow and not normal, I would say. And yes, when I hear all this, it sounds like a problem with the client or synchronization mechanism. Unfortunately, I don’t know enough about the technical details. Maybe someone else has an idea..

Also, I can’t really try to replicate it, at the same scale. However, I did upload a smaller folder containing 2,278 files and didn’t experience any of the issues you describe. The performance isn’t great, but it’s also not as bad as yours, and the UI stayed responsive at all times.

grafik
grafik826×110 18.3 KB

Nextcloud Desktop-Client Version 4.0.6daily (Arch)
Arch Linux
Gnome 49