[solved] Hangs when adding lots of files

I added a folder with many mp3 (6Go of mp3s)

It uploads very fast and then suddenly hangs. After a while I get a timeout error in the client.
It then starts the synch process again. It can happens randomly after 5 files uploaded, 8 or once after 200 files got uploaded. I cannot find anything interesting in nginx, php-fpm or nextcloud logs.

[edit] It seems like the files on which Nextcloud hangs are being permanently locked, thus impossible to delete without manually cleaning redis cache

[edit] before most retries I get a notice windows about conflicting files and asking whether to continue normal sync or keep local conflict files

What does

tload

say?

0.02, 0.05, 0.08

1.07 before hanging

Please guys doesn’t anyone have an idea of what could be wrong ?

I hope you don’t use sqlite. You seem to use redis, I haven’t seen such problems yet. What kind of device are you using as server? Can you estimate after how many files your error appears?

Is this your normal primary storage, no shared folders, no external storage?

Intel® Core™ i5-4670 CPU @ 3.40GHz
System memory size: 23GiB

Nextcloud runs in an LXC container, with an Nginx front proxy in another LXC container, both running on the high performance ZFS file system.
The primary storage is different than from the LXC system filesystem. It comes from another ZFS container on large disks (on the same computer):

lxc config device show nextcloud 
data:
  path: /srv/www/nextcloud/data
  source: /tank/cloud/
  type: disk
root:
  path: /
  type: disk

I have other applications like gitlab running this way and no issue at system level, no filesystem errors at all or anything related. I am pretty sure something is wrong in my config (most probably nginxes config) but I can’t iron it out…

For the “how many files” question, it is quite random, but the pattern is around 200 files at first attempt and then less that 10 for each retry.

I upgraded to 11.0.1: exact same issue.

Stucks after 184 mp3s uploaded by the client, the highest load I got was 0.4 … After a while it started again and stucked after 3 files.

mysql --version
mysql  Ver 15.1 Distrib 10.0.29-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

Bug found: Nextant App

When nextant is enabled it caused the issue described.

@Lox could you file a bug here please?

Thanks!

1 Like

Could it be your redis service? I have a similar issue. All properly configured yet redis suddenly stop caching items or passing the requests. When I restart redis the images and file loads correctly.

As semaphore support has been removed, I do not have the issue anymore.
See the github issue referenced.