Nextcloud takes all of my RAM

esbeeb · February 10, 2020, 8:06am

Here is a fun way to watch all your RAM being gobbled up in real time. Before you visit the Files tab, in a terminal, run the following:

while true; do free -h; sleep 3s; echo; done

Let that command run for a while, and watch the far right column, called “available”. This lets you see how much RAM the server is idling at, before you place any demands on it.

Then visit the “Files” tab in Nextcloud, and watch the “available” RAM decrease.

Here is the behaviour seen on my Nextcloud 16 install, running on a VPS with a mere 1GB of RAM, showing before, during, and after I load up “Files”:

total used free shared buff/cache available
Mem: 987Mi 346Mi 106Mi 16Mi 534Mi 465Mi
Swap: 255Mi 60Mi 195Mi

total used free shared buff/cache available
Mem: 987Mi 364Mi 82Mi 19Mi 540Mi 444Mi
Swap: 255Mi 60Mi 195Mi

total used free shared buff/cache available
Mem: 987Mi 373Mi 71Mi 19Mi 542Mi 435Mi
Swap: 255Mi 60Mi 195Mi

total used free shared buff/cache available
Mem: 987Mi 377Mi 64Mi 19Mi 544Mi 430Mi
Swap: 255Mi 60Mi 195Mi

total used free shared buff/cache available
Mem: 987Mi 387Mi 111Mi 22Mi 488Mi 419Mi
Swap: 255Mi 61Mi 194Mi

total used free shared buff/cache available
Mem: 987Mi 387Mi 110Mi 22Mi 488Mi 418Mi
Swap: 255Mi 61Mi 194Mi

total used free shared buff/cache available
Mem: 987Mi 373Mi 125Mi 22Mi 488Mi 433Mi
Swap: 255Mi 61Mi 194Mi

total used free shared buff/cache available
Mem: 987Mi 356Mi 141Mi 22Mi 488Mi 449Mi
Swap: 255Mi 61Mi 194Mi

total used free shared buff/cache available
Mem: 987Mi 356Mi 142Mi 22Mi 488Mi 449Mi
Swap: 255Mi 61Mi 194Mi

Note: Press “Ctrl + C” to end this RAM monitoring.

Before opening “Files”, the server still had 465MB RAM free. During showing Files, this dropped to 418MB temporarily. But then after about 10 seconds, the free RAM raised back up to 449MB (and I was still looking at “Files”).

That “available” RAM must not slam down to zero, or else you very likely must reboot your Nextcloud server!!

Note: the little thumbnail previews shown for each file and folder icon can be very RAM-expensive to generate (especially the very first time)! You would do well to have only a few files in your top-level folder in “Files”. Instead create folders, and store your files deeper inside those.

esbeeb · February 10, 2020, 8:21am

A Raspberry Pi 3 is sure to only have 1GB of RAM. I think with the advent of Nextcloud 18, much more RAM is assumed to be in a server (especially with ONLYOFFICE, which is rumored to be very RAM hungry).

If you want a (relatively) cheap, decent Nextcloud server, maybe buy (for example) a refurbished Lenovo Thinkpad x220, x230, or x250 next time, and max out the RAM?

dungta0321 · February 10, 2020, 9:09am

I think it was the problem of the software. I increased swap size to 2 Gb but Nextcloud eat it all. Why do you recommended thinkpad laptop instead of intel nuc or another chip U, cereon laptop. I just curious:D

esbeeb · February 10, 2020, 12:00pm

gcoter · February 10, 2020, 12:03pm

How much RAM would be necessary to run Nextcloud 18 in good conditions?

My top level directory (the one whose content is displayed when I open the Files tab) contains 10 elements. Also, I am able to open it sometimes. In my case, if I remember well, going to “Files” directly works but coming from another tab (like “Activity”) causes the problem. So I would say that displaying “Files” works most of the time. But for some reasons, when I come from another tab, it crashes. By the way, I encountered this problem first with the Audio Player app (which I thought was responsible for it) before finding out that the problem could also happen with Files. I’m just saying that this problem might not be related to a display problem of “Files” specifically.

gcoter · February 10, 2020, 12:06pm

Yes I have the same problem. I wonder how much swap would be necessary to prevent it from crashing. Most of the time, Nextcloud takes very little RAM in comparison (~140 MB I would say).

esbeeb · February 11, 2020, 1:33pm

Yes, I too want to know that, because my VPS only has 1GB.

Maybe if we’re lucky, someone will write a Nextcloud App called “RAM miser” or something like that…

gcoter · February 12, 2020, 6:54pm

From System requirements — Nextcloud latest Administration Manual latest documentation

Memory requirements for running a Nextcloud server are greatly variable, depending on the numbers of users, apps, files and volume of server activity. Nextcloud needs a minimum of 128MB RAM, and we recommend a minimum of 512MB.

gcoter · February 12, 2020, 6:58pm

And it didn’t change between version 16 and version 18:

esbeeb · February 12, 2020, 10:12pm

Well, I don’t think anyone (no offense) in this thread has yet to establish if this RAM exhaustion is the result of a bug, or the result of Nextcloud working properly, but now requiring more RAM (and maybe those docs stating the same RAM requirements as in older Nextcloud versions are out of date).

gcoter · February 12, 2020, 10:46pm

What do you mean? Personally, I am still wondering if it is a bug or something normal.

If I understand well (tell me if I am wrong ) , you think that it is caused by the new version of Nextcloud. Maybe I should try to come back to a previous version (16 or 17)? Or do you think that running Nextcloud on a Raspberry Pi 3 (considering there is only 1 user) is not feasible, whatever the version?

esbeeb · February 13, 2020, 7:30am

So we are in the same boat there.

This is my suspicion, yes.

I would personally only use a Raspberry Pi 4 for a 1-user Nextcloud install (as they are very reasonably cheap, and a good value for what you get), and yes, I would put NC 17 on it at this time. And this is with the Nextcloud data dir and DB moved out to some decently fast USB3-attached storage.

A Raspberry Pi 3 is just a little too slow for comfort, even for 1 user (and I’ve tried it), I say. I would personally use a Raspberry Pi 3 for, say, a Pi-Hole.

gcoter · February 13, 2020, 8:07pm

Ok thank you for the clarification

gcoter · February 22, 2020, 2:31pm

Hi all,

So the only idea I have left (except buying a Raspberry Pi 4) is to try migrating from version 18 to version 17. Is it easily feasible with docker? If I just change the image tag but keep the same docker volumes, will it work?

gcoter · March 4, 2020, 8:55pm

I think I managed to limit the RAM usage by changing the MPM configuration of Apache (in /etc/apache2/mods-enabled/mpm_prefork.conf):

# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxRequestWorkers: maximum number of server processes allowed to start
# MaxConnectionsPerChild: maximum number of requests a server process serves

<IfModule mpm_prefork_module>
	StartServers			 2
	MinSpareServers		  2
	MaxSpareServers		 3
	MaxRequestWorkers	  3
	MaxConnectionsPerChild   0
</IfModule>

The default values were way too high for my Raspberry Pi 3. For instance, when loading the Photos tab, one Apache process takes 150 MB. With the default values, Apache tries to run up to 150 processes. I reduced it to 3, which is the maximum my Raspberry can handle. As a consequence, loading the Photos page for instance is very slow, but at least now my server doesn’t crash. The other pages load well, because they contain less heavy files.

To conclude, I agree with @esbeeb on the fact that a Raspberry Pi 3 might not be enough to use Nextcloud comfortably. I don’t think it is related to the version of Nextcloud, it is just a matter of configuring Apache accordingly to the server’s capacity, which is weak in my case.

gcoter · March 4, 2020, 9:10pm

Remark : I calculated the number of workers based on the RAM usage when loading the Photos page, but I notice that on other pages a worker doesn’t take much RAM. So I might use more than 3 workers.

esbeeb · March 9, 2020, 3:11am

For Nextcloud 17, I personally wouldn’t buy a Raspberry Pi 4 either. I would suggest a refurbished Lenovo Thinkpad X230 at least. These can be bought with an i5 CPU and 4GB of RAM for $142 USD plus shipping (where I live). The i5 CPU would be a quantum leap in performance over a Raspberry Pi 4. Maybe toss in an SSD, and you’d have true SATA speeds and reliability.

You get so much more all-around horsepower for just a little more money.

Your efforts to fine tune Nextcloud are commendable, but there comes a point where just “going with the flow” (of Nextcloud being designed to expect more horsepower) will save you time and hassle, and be well worth that bit more money.

harvo · September 24, 2020, 10:54pm

I have been having similar issues. I think there are a couple of solutions.

I think the most likely issue is the thumbnail generation, I found that even < 1GB of files, the server will fire off a ton of requests to generate thumbnails, and if you’re using external storage - this will take even longer.

This is where the apache processes above are eating up your RAM. But that’s because they are told how much they’re allowed to serve and how long to live for.

One thing that really helps for this specific issue is to use the preview generator.

https://apps.nextcloud.com/apps/previewgenerator

You can run this on your cron and it will pick up any new images and generate thumbnails for them. Because it doesn’t go through apache, there is barely any overhead, it just works through your images one by one. I’ve been running this on a Digital Ocean droplet, and even after increasing the server to 8GB RAM (as a temporary test of course!) it still ate up all the RAM. But running this script, it sat in the backrgound and generated everything with no issue!

Secondly, something I touched on earlier, there are settings within apache to decide how much a worker is allowed to serve and also how long they live for. I can’t remember the exact settings, just about to look for it now myself. But we have done similar at work.

Essentially the default is 100 for serving, but this can be frequently changed to 2000 or even 5000 for bigger servers. You can give this a gentle increase and see how it goes, if no difference, try a bit more.

The other setting is how long they wait before they die. The default is 5 seconds, you can safely reduce this down to 3. Once the apache workers die, the RAM is freed up.

bryanl · June 22, 2021, 9:49pm

Hmm, I’ve been running Nextcloud on a Raspberry Pi 3 with three users since 2017 without any problems whatsoever. Absolutely fantastic. Now it autoupgraded to the latest version, and now it is using up all the RAM and klling the Pi. Regularly cannot sync files and end up e-mailing stuff from one computer to another.

I’m not very happy. I’m actually more likely to look for a different sync solution now than to buy new hardware.

mooleshacat · January 10, 2022, 4:07pm

I saw this post after searching ram usage related to clamav and I noticed there are additional setup instructions for image preview generator while looking at the app page.

You need to run the cron manually to generate initial image cache. Then you need to enable the cronjob. See the app page and read description for more information.