Uploads of many small files max out CPU

I have Nextcloud on a VPS, with the data directory on an encrypted rclone remote on GDrive. So far it has worked well, but I’ve only given it very light use.

A few weeks ago I made an account for a friend, who started uploading his files for backing up. He has a huge amount of small files, and this is making the server go nuts - mysqld causes the load to go very high, CPU usage at 95% to 100%, and everything else suffers.

I’ve looked at ā€˜and’ to auto-renice the process but it doesn’t touch root processes (and mysqld is a root process, probably because I have installed Nextcloud via snap), and cpulimit (not sure if/how I can automate that as process IDs change all the time).

Is this normal? Do you think it’s happening purely because of the rclone remote setup - can anyone comment on whether this behaviour is seen on Nextcloud servers with local storage?

Any ideas for solutions are very welcome. I would love to somehow automate a check so that if mysqld or a related Nextcloud process starts hogging the CPU, it’s reniced to allow other processes to take over if needed.

Use redis if possible. The database cache can change the speed enormously. I have done a few tests a long time ago (https://github.com/owncloud/core/issues/20967), you find newer topics here. I don’t know if the snap lets you change all these configurations.

Yes, use a cache server such as Redis to assist Nextcloud for large numbers of files or for large file sizes. An alternative to setting up a cache server is to use Seafile instead, which has the best performance out of the box and doesn’t need a cache server, but is pretty much just for cloud storage. Nextcloud has a good selection of apps, so if a Redis server fixes the performance issue, Nextcloud is a great system to use.

As far as I understand, I have set up the snap to use redis. The settings are there in config.php, and a redis-server process is running. I don’t think I can configure it in any way other than the way it’s already configured due to it running inside the snap though.
Redis has always been running and these issues occurred with it running unfortunately.

Have you configured the ā€˜memcache.locking’ => ā€˜\OC\Memcache\Redis’,
in the nextcloud.config file?
And when going to settings --> overview
do you pass all the security & setup warnings?