Nextcloud having lots of trouble handling large amount of files

ℹ️ Support
1

Setup

  • Raspberry Pi 3 server computer
    • OS: Debian/Yunohost
      • Nextcloud server app
  • Regular desktop client computer
    • OS: Arch Linux
      • Nextcloud client app
  • Router/Modem 1Gbps LAN connection

I’m using Nextcloud for the first time, installed the client and assigned a folder with more than few files, around 30,000 or so. The Nextcloud client shows many many Gateway Time-out 504 errors, resulting in the majority files not getting uploaded. It’s also uploading at a very slow at pace, like I’m seeing 92kB out of 186MB uploaded after half an hour or so.
I’ve tried uploading a single large file as well, which although freezing for a minute from time to time, it resumed at a very fast pace, adding MB’s in seconds.

YunoHost info, not sure if this is useful:

{
    "host": "Debian 8.0",
    "kernel": "4.9.35-v7+",
    "packages": {
        "yunohost": "2.7.2",
        "yunohost-admin": "2.7.2",
        "moulinette": "2.7.2",
        "ssowat": "2.7.2"
    },
    "system": {
        "disks": {
            "root": "Mounted on /, 56.7GiB (47.6GiB free)",
            "mmcblk0p1": "Mounted on /boot, 62.0MiB (41.1MiB free)"
        },
        "memory": {
            "ram": "923.4MiB (618.3MiB free)",
            "swap": "100.0MiB (100.0MiB free)"
        }
    },
    "services": {
        "glances": "running (enabled)",
        "nslcd": "running (enabled)",
        "metronome": "running (enabled)",
        "nsswitch": "inactive (not-found)",
        "postfix": "running (enabled)",
        "rspamd": "running (enabled)",
        "yunohost-firewall": "running (disabled)",
        "nginx": "running (enabled)",
        "rmilter": "running (enabled)",
        "slapd": "running (enabled)",
        "dnsmasq": "running (enabled)",
        "fail2ban": "running (enabled)",
        "yunohost-api": "running (enabled)",
        "mysql": "running (enabled)",
        "avahi-daemon": "running (enabled)",
        "dovecot": "running (enabled)",
        "redis-server": "running (enabled)",
        "php5-fpm": "running (enabled)",
        "ssh": "running (enabled)"
    },
    "applications": {
        "cesium": "Cesium",
        "opensondage": "OpenSondage",
        "zerobin": "Zerobin",
        "roundcube": "Roundcube",
        "nextcloud": "Nextcloud"
    }
}

Also, do I run the risk of these files being deleted off the desktop computer, since they are not uploaded on the Pi?

[update]

What info is missing?

2

It looks like you did not spend a lot of time in optimizing your system. File-locking cache should be handled via redis, for your database there is generally a huge potential for improvements (optimize caches: just an example, it must fit for your system: https://github.com/owncloud/core/issues/20967#issuecomment-205474772). The free RAM shows that you still have potential for improvements. And there are of course the number of connections your webserver can establish (should fit the number of possible database connections and php processes).

For a small device, it is much more challenging to find suitable parameters.

1 Like
3

Closing due to inactivity