Upload to database is damn slow

A few days ago I wanted to upload all my pictures to my Nextcloud. No sooner said than done: I copied these files to the Nextcloud folder on my desktop PC and waited until the desktop client uploaded the files. This still hasn’t happened after 2 days of non-stop running, then I started to look for errors …

On the PC the CPU is almost underutilized, just like SSD or LAN. Nothing happened.

Then I found this manual the day before yesterday: Tutorial: How to migrate mass data to a new NextCloud server

So, I loaded the data (about 300GB, about 1k files) directly into the Nextcloud folder on the NAS that took incredible 30 min …

Now I executed the occ files:scan --all … I ran that overnight and then aborted that, with the result:

that only about 7 files per minute were imported into the database. (Data: Ubuntu 16 LTS, OS@M.2, i3-6100, 16GB RAM).

The load was the whole night never over 4% CPU and 4GB RAM, the HDD load is not noteworthy.

Nextcloud is version 15.0.10, (apache2) php 7.3 with mysql and the rest should be up to date.

In the Nextcloud web interface under ‘Settings’, ‘Overview’ I get the warning “No memory cache has been configured. …” But this is not true.

Could this be the reason?

Of course, I followed the instructions in the manual:
I entered the following in /etc/mysql/conf.d/mysql.cnf:

[mysqld]
innodb_buffer_pool_size=1G
innodb_io_capacity=4000"

Unfortunately with the result that when I type mysql into the console the error comes: mysql: [ERROR] unknown variable 'innodb_buffer_pool_size=1G'

In /etc/php/7.3/fpm/pool.d/www.conf I also entered the following:

pm = dynamic
pm.max_children = 120
pm.start_servers = 12
pm.min_spare_servers = 6
pm.max_spare_servers = 18

And in /etc/php/7.3/apache2/php.ini:

opcache.enable=1
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=10000
opcache.memory_consumption=128
opcache.save_comments=1
opcache.revalidate_freq=1

But if I also enter the following in the NC config.ini independently of it, an internal error occurs:

‘memcache.local’ => ‘\OC\Memcache\APCu’,

I don’t know exactly what it depends on, I’ve tried everything filled (apache2, php, mysql reinstalled), but still it doesn’t work.

What else can I try? What do I have to look for?

Have you also set up redis?

@KarlF12: No. Should I?

I finished occ files:scan --all … it took 18h for ~300GB.
Now I want to download those files to my clients … I started it only at one client. The scanning of my folders (by the desktop client app) took around 3h and now it’s downloading with 2-6MB/s … the client is connected with LAN and if I copy the same files with ftp or smb from the NAS it copies with 100-120MB/s.

Why is my Nextcloud so slow?
How can I speed it up?

Yes.

https://docs.nextcloud.com/server/stable/admin_manual/configuration_server/caching_configuration.html

Thanks @KarlF12. I did it, but it doesn’t make a measurable difference.

I’m not sure if it’s the reason for the slow migration (copy, export, … whatever). It’s just a guess. What else could be the reason?

Can you clarify how your Nextcloud server is connected to the NAS backend?

What do you mean by that exactly?

It’s: Ubuntu 16 LTS, at a M.2 SSD, Intel GBit LAN, apache2 with php7.3 and mysql.

It is doing a lot in the database which can take very long. One point is redis which reduces a lot the load on the database and then you can tune the caching of the database. I did some tests some time ago and I went from ~130 files/minute upload to >1000 files/minute (large number of very small files): Upload performance OC 8.2.1 (50 mysql queries per file) · Issue #20967 · owncloud/core · GitHub
There are some people that shared some details how they changed their database settings, don’t expect it to work out of the box, it probably must be adjusted to your use case.

Just for the database test, a large number of small files is a good benchmark (https://github.com/owncloud/core/issues/11588#issuecomment-169728089). Do some tests before you upload the whole bunch of real files. Don’t expect the same speeds like SFTP or SMB but you should reach two digit transfer rates.

I’m asking how your Nextcloud server is connected to your NAS.

Direct, bits and bytes. The cloud is on the NAS.
In this quote I was talking about the “Nextcloud folder” on my PC. It is connected to the NAS ( resp. the Cloud ) as written with GBit LAN.

Thanks @tflidd, I’ll read through this later and report back.