Nextcloud takes all of my RAM

gcoter · February 1, 2020, 9:37pm

System Information

Nextcloud version (eg, 12.0.2): 18.0.0 (installed with Docker, see details below)
Host system: Linux raspberrypi 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux
Docker version: 19.03.5, build 633a0ea
docker-compose version: 1.24.1, build 4667896
docker-compose file:

version: '2'

volumes:
  db:
    driver: local
  data:
    driver: local

services:
  db:
    image: postgres:alpine
    mem_limit: 50m
    volumes:
      - db:/var/lib/postgresql/data
    env_file:
      - db.env
    networks:
      - internal

  app:
    image: nextcloud:apache
    mem_limit: 300m
    hostname: <my-hostname>
    depends_on:
      - db
    env_file:
      - db.env
      - app.env
    networks:
      - web
      - internal
    ports:
      - 8080:80
    labels:
      - traefik.enable=true
      - traefik.frontend.rule=Host:<my-hostname>
      - traefik.port=80
      - traefik.docker.network=web
    volumes:
      - data:/var/www/html

networks:
  web:
    external: true
  internal:

The issue you are facing:

Sometimes, Nextcloud takes all of the available RAM and Swap memory. When it happens, I have no choice but to stop the container. I noticed that it happens when I switch from one tab to another. For instance, I first visit the “Activity” tab, then click on the “Files” tab, but instead of loading the tab, it will keep loading while the memory is being filled on my Raspberry.

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

Visit the Activity tab
Click on another tab

Sorry, I cannot give more details for now, I don’t understand at all what is going on. Most of the time, it works fine.

The output of your Nextcloud log in Admin > Logging:

https://pastebin.com/c0eaS6NF

The output of your config.php file in `/path/to/nextcloud` (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'passwordsalt' => 'W1ADxxD5S7ZesWZdRinlwPTly6Sujw',
  'secret' => 'xvHTw1FrhQpo2kGTMGMFzLCpl9mzTBriijryuMaDKV+h5Um0',
  'trusted_domains' => 
  array (
    0 => 'localhost',
    1 => '<my-domain>',
  ),
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'pgsql',
  'version' => '18.0.0.10',
  'overwrite.cli.url' => 'http://localhost',
  'overwriteprotocol' => 'https',
  'dbname' => 'nextcloud',
  'dbhost' => 'db',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'oc_admin',
  'dbpassword' => '<db-pasword>',
  'installed' => true,
  'instanceid' => 'oce7wys7ujj6',
  'maintenance' => false,
);

Rello · February 2, 2020, 8:54am

Can you do an htop and sort by memory to see which process is causing this?

webserver
php
database

gcoter · February 2, 2020, 10:50am

Yes, I can see it is apache2. And I confirm that I am able to reproduce the issue by doing these steps:

Visiting the Activity tab
Clicking on the Files tab

gcoter · February 2, 2020, 12:48pm

After googling a bit, I found several posts mentioning that Apache takes a lot of memory, in particular this one: https://askubuntu.com/questions/558788/apache-eating-up-2gb-of-ram

They suggest that MPM Configuration might not be adapted to the OP’s environment. So I searched for the configuration which is used in my case:

root@nextcloud:/var/www/html# cat /etc/apache2/mods-available/mpm_prefork.conf
# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxRequestWorkers: maximum number of server processes allowed to start
# MaxConnectionsPerChild: maximum number of requests a server process serves

<IfModule mpm_prefork_module>
	StartServers			 5
	MinSpareServers		  5
	MaxSpareServers		 10
	MaxRequestWorkers	  150
	MaxConnectionsPerChild   0
</IfModule>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

I am not an Apache expert but I wonder if MaxRequestWorkers 150 is not too big considering I am on a Rapsberry. What do you think?

gcoter · February 5, 2020, 9:58pm

I tried to decrease MaxRequestWorkers to 15 but it didn’t solve my problem. Does someone have another idea?

gcoter · February 6, 2020, 10:43pm

In this thread, someone suggests to “calculate” the ideal value for MaxRequestWorkers like this: https://stackoverflow.com/questions/40562135/apache-httpd-using-all-memory-till-server-down/41308833#41308833

I might try this soon. But I am open to suggestions

dungta0321 · February 9, 2020, 12:23am

I got this problem on rpi 3 too. When i am on file tab then click on photo tab, apache eat all my ram and swap. i have to unplug power to restart server

gcoter · February 9, 2020, 11:59am

It seems really similar to my problem indeed. Which version of Nextcloud do you use? Did you install it with Docker?

dungta0321 · February 10, 2020, 12:36am

Yes Im using nextcloud 18 on docker. Latest tag

esbeeb · February 10, 2020, 8:06am

Here is a fun way to watch all your RAM being gobbled up in real time. Before you visit the Files tab, in a terminal, run the following:

while true; do free -h; sleep 3s; echo; done

Let that command run for a while, and watch the far right column, called “available”. This lets you see how much RAM the server is idling at, before you place any demands on it.

Then visit the “Files” tab in Nextcloud, and watch the “available” RAM decrease.

Here is the behaviour seen on my Nextcloud 16 install, running on a VPS with a mere 1GB of RAM, showing before, during, and after I load up “Files”:

total used free shared buff/cache available
Mem: 987Mi 346Mi 106Mi 16Mi 534Mi 465Mi
Swap: 255Mi 60Mi 195Mi

total used free shared buff/cache available
Mem: 987Mi 364Mi 82Mi 19Mi 540Mi 444Mi
Swap: 255Mi 60Mi 195Mi

total used free shared buff/cache available
Mem: 987Mi 373Mi 71Mi 19Mi 542Mi 435Mi
Swap: 255Mi 60Mi 195Mi

total used free shared buff/cache available
Mem: 987Mi 377Mi 64Mi 19Mi 544Mi 430Mi
Swap: 255Mi 60Mi 195Mi

total used free shared buff/cache available
Mem: 987Mi 387Mi 111Mi 22Mi 488Mi 419Mi
Swap: 255Mi 61Mi 194Mi

total used free shared buff/cache available
Mem: 987Mi 387Mi 110Mi 22Mi 488Mi 418Mi
Swap: 255Mi 61Mi 194Mi

total used free shared buff/cache available
Mem: 987Mi 373Mi 125Mi 22Mi 488Mi 433Mi
Swap: 255Mi 61Mi 194Mi

total used free shared buff/cache available
Mem: 987Mi 356Mi 141Mi 22Mi 488Mi 449Mi
Swap: 255Mi 61Mi 194Mi

total used free shared buff/cache available
Mem: 987Mi 356Mi 142Mi 22Mi 488Mi 449Mi
Swap: 255Mi 61Mi 194Mi

Note: Press “Ctrl + C” to end this RAM monitoring.

Before opening “Files”, the server still had 465MB RAM free. During showing Files, this dropped to 418MB temporarily. But then after about 10 seconds, the free RAM raised back up to 449MB (and I was still looking at “Files”).

That “available” RAM must not slam down to zero, or else you very likely must reboot your Nextcloud server!!

Note: the little thumbnail previews shown for each file and folder icon can be very RAM-expensive to generate (especially the very first time)! You would do well to have only a few files in your top-level folder in “Files”. Instead create folders, and store your files deeper inside those.

esbeeb · February 10, 2020, 8:21am

A Raspberry Pi 3 is sure to only have 1GB of RAM. I think with the advent of Nextcloud 18, much more RAM is assumed to be in a server (especially with ONLYOFFICE, which is rumored to be very RAM hungry).

If you want a (relatively) cheap, decent Nextcloud server, maybe buy (for example) a refurbished Lenovo Thinkpad x220, x230, or x250 next time, and max out the RAM?

dungta0321 · February 10, 2020, 9:09am

I think it was the problem of the software. I increased swap size to 2 Gb but Nextcloud eat it all. Why do you recommended thinkpad laptop instead of intel nuc or another chip U, cereon laptop. I just curious:D

esbeeb · February 10, 2020, 12:00pm

gcoter · February 10, 2020, 12:03pm

How much RAM would be necessary to run Nextcloud 18 in good conditions?

My top level directory (the one whose content is displayed when I open the Files tab) contains 10 elements. Also, I am able to open it sometimes. In my case, if I remember well, going to “Files” directly works but coming from another tab (like “Activity”) causes the problem. So I would say that displaying “Files” works most of the time. But for some reasons, when I come from another tab, it crashes. By the way, I encountered this problem first with the Audio Player app (which I thought was responsible for it) before finding out that the problem could also happen with Files. I’m just saying that this problem might not be related to a display problem of “Files” specifically.

gcoter · February 10, 2020, 12:06pm

Yes I have the same problem. I wonder how much swap would be necessary to prevent it from crashing. Most of the time, Nextcloud takes very little RAM in comparison (~140 MB I would say).

esbeeb · February 11, 2020, 1:33pm

Yes, I too want to know that, because my VPS only has 1GB.

Maybe if we’re lucky, someone will write a Nextcloud App called “RAM miser” or something like that…

gcoter · February 12, 2020, 6:54pm

From System requirements — Nextcloud latest Administration Manual latest documentation

Memory requirements for running a Nextcloud server are greatly variable, depending on the numbers of users, apps, files and volume of server activity. Nextcloud needs a minimum of 128MB RAM, and we recommend a minimum of 512MB.

gcoter · February 12, 2020, 6:58pm

And it didn’t change between version 16 and version 18:

esbeeb · February 12, 2020, 10:12pm

Well, I don’t think anyone (no offense) in this thread has yet to establish if this RAM exhaustion is the result of a bug, or the result of Nextcloud working properly, but now requiring more RAM (and maybe those docs stating the same RAM requirements as in older Nextcloud versions are out of date).

gcoter · February 12, 2020, 10:46pm

What do you mean? Personally, I am still wondering if it is a bug or something normal.

If I understand well (tell me if I am wrong ) , you think that it is caused by the new version of Nextcloud. Maybe I should try to come back to a previous version (16 or 17)? Or do you think that running Nextcloud on a Raspberry Pi 3 (considering there is only 1 user) is not feasible, whatever the version?