Performance degradation

ℹ️ Support
1

I’m running a Nextcloud server for a couple of users for about a year now. I have noticed that since about November last year, the performance degraded in the sense that the CPU usage of the VM increased a lot.

The server runs:

  • 2 Cores, 2 GB of RAM
  • Ubuntu 18.04
  • at the moment NextCloud 17.0.2
  • the PHP 7.2 that comes with Ubuntu 18.04
  • MariaDB 10.1 that comes with Ubuntu
  • Apache 2.4.29 that comes with Ubuntu

So it is a pretty simple and straight forward setup.

What alarms me is the CPU usage graph from vSphere over a year:

nc-year
nc-year1136×451 40.4 KB

The CPU usage goes up early November last year.
At this time, I updated to NextCloud 16.06 (16.01 was in May 2019)
The next significant increase was 2nd half of December when I updated to 17.0.2 …

Over the last day it looks like this:

nc-day
nc-day1140×450 97.3 KB

So it is not a permanent usage but regular spikes.
The gap this morning was after I updated Ubuntu and rebooted at 8:25.
The spikes are even during the night when I think that not too many users use it.
But there are certainly a number of PC/Smartphones than sync in the background 24 hours.

Using top I can see that most of the CPU is spent in mysqld but also in apache2.
The system checks of NextCloud are all green …

Does anyone else see such a behaviour?
What would you suggest where to start looking for the cause?

2

I’ve personally not seen this behavior yet, but are you e.g. using Redis for caching purposes? At night a cron job most likely is activated which scans and updates the file tables etc. So it would be worse to check that too.

3

No, I did not set up a Redis cache.

4

I think you should give it a try, because its purpose is to speed-up many things.

5

While that might speed up … I did not see a need before for such a small server and user count.
It would still be good to know what caused the CPU usage increase with 16.0.6 and then again with 17.0.2.

6

I recall two performance related issues:

  1. https://github.com/nextcloud/server/pull/17905 a infinite loop caused by xml parsing for some searchdav requests. Fix was released with 17.0.2.

  2. https://github.com/nextcloud/server/issues/17241 a change by php to use higher default values for password hashing. There is no way to change those values without touching the code but a patch to make those values configurable is prepared and will be released to 16 / 17 / 18.

7

To follow up with the “solution”:
I did finally find out that the performance problem was caused by a runaway client causing the high load. It was a browser add-on (“floccus”) for synchronizing bookmarks into Nextcloud with its “Bookmarks” app. For some reason the sync content was corrupted, causing endless re-syncing. Deleting the content and re-creating the connection solved the problem.