Nextcloud version (eg, 12.0.2):13.0.1
Operating system and version (eg, Ubuntu 17.04): DEBIAN 9.4 Stretch
Apache or nginx version (eg, Apache 2.4.25): Apache 2.4.25
PHP version (eg, 7.1): 7.0.27
Hi there!
I wanted to ask whether it is normal for Nextcloud to consume MASSIVE amounts of RAM?
At this moment I only have one user registered (Encryption is ACTIVATED and I´m using MySQL/MariaDB) and I´m initially syncing around 120 GB of files.
I have set up a server with 32 GB of RAM for my Nextcloud instance. Now, I was expecting Nextcloud to be using around 1-2 GB per registered user, when syncing many files maybe 4 GB at best.
At this very moment 16 GB are being used and the amount of used RAM is continuing to grow…
The rest of the hardware is a RAID 10 array with a writing speed of approx. 150-200 mb/s.
The CPU is an Intel i7, which should be way overpowered for one user…
Apart from that there are no other significant services running.
Does anybody know, why this i happening? Is it normal (which would be hard to believe)?
Is this the first time you’ve seen this error? (Y/N): No, I´ve had a test instance on a smaller server with 4 GB RAM and an Intel Atom Processor and when the SWAP started to be filled I figured it was too much to handle.
Steps to replicate it:
Install Nextcloud
Run Nextcloud
Start saving money to buy more of that delicious RAM and order in bulk (I figured, if you have more RAM then data to sync it might work…)
4.???
5.Profit!
P.S.: I haven´t found any errors and the file syncing etc. is working like a charm. Same goes for calendar and contacts sync.
It does look like an awful lot of mysqld instances running at the same time (without error, but still…)
What process(es) are actually consuming the most RAM? I have about 10 users on my server with 16 GB RAM and I hardly hit 4 GB RAM usage. The processes using most of the RAM are mysqld instances (26 to be precise, but that’s normal).
Is your system swapping?
The RAM usage depends on the configuration of DB, PHP and web server by the amount of memory you allow these services to use - which is configured within their config files. So if you were very generous during the setup and configuration that could be the result. However on Unix systems high RAM usage is not an issue as long as the system is not swapping. It usually uses a little bit more RAM for the services as long as there is a lot available. You can definitely investigate if you’re curious and want to detect potential issues, but in general I believe there is nothing to worry about, only because the system is using 50% of its RAM.
It´s the same here – the mysqld instances make up almost all of the RAM usage.
It´s a bit weired, because RAM-usage dropped significantly to around 800mb-2GB yesterday.
I´m not sure why, but maybe it had something to do with the fact that I was transferring a large amount of very small files (thousands of 120kb or similar files).
I will keep an eye on it and maybe try to reproduce the RAM usage.
Hmm check out which process is using the high amount of ram, database, webserver or PHP process?
Are you using mod-php (Apache with mpm-prefork) or php-fpm? Which database?
Database: I configured my MariaDB to provide innodb cache a bid more then the actual database size. This is also around the amount of overall RAM (~128M in my case), mysqld uses after some days, but could vary depending on how Nextcloud is used. No idea, how other databases (postgreSQL) behave with this. Oracle MySQL should be quite similar.
Redis, if just used for transactional file locking, for my experience can be ignored, as the memory usage is (should be) very low compared to the others.
Webserver + PHP hardly depends on configuration. I use, for single/family usage, Apache + mod-php, which opens up a new process for nearly every access. So during heavy simultaneous access, this can rise memory usage quite much. APCu and OPcache (with recommended 128M cache size) is configured, all non-needed Apache and PHP modules were actively disabled. In my/this case every Apache process takes around 30M, max 50M. On heavier usage (looks like your case), I would definitely use dedicated php-fpm, which significantly reduces memory usage. I think whether nginx, Apache (with mpm-worker/event) or even lighttpd does not make a too big differnce then, as for my impression the php-fpm processes (with caches etc) take the majority of RAM.
Ok, I´ve added more users and monitored the RAM consumption over approximately a week.
Turns out that indeed great amounts of very small files tend to recreate the high usage of RAM.
I also noticed, that the Nextcloud instance can freeze up other users/accounts, if one of them has a big amount of data imported into a calendar, but all in all it´s working fine now with approx. 512-2 GB of RAM usage, depending on user activity.
Yes, but if you don’t have it, all this will be handled by your database which can’t do that efficiently (= very bad overall performance when transferring large amount of files).
Ah sorry this was missunderstandable, of course also for me Redis is the way to go if you have any chance to run it. I just mean the impact on overall memory usage of Redis is low, so that it should not be needed to count separately, compared to webserver/PHP and database memory usage .
I did understand that. It’s just you could think that if you don’t run it at all it could be better when it actually takes more load off the server than it uses itself.
.