Memory usage spike when opening apps

Nextcloud version (eg, 20.0.5): 22.0.0 (seen on 21.0.0 and 21.0.3 as well)
Operating system and version (eg, Ubuntu 20.04): Ubuntu 20.4
Apache or nginx version (eg, Apache 2.4.25): 2.4.41-4ubuntu3.3
PHP version (eg, 7.4): 7.4.3-4ubuntu2.5

The issue you are facing:
When I load an app in the brower, the server memory spikes (apache processes go from ~300M virt to ~2G virt). This completely takes down my server from memory exhaustion.

I don’t know what steps to take next to identify where the problem is coming from, if its a bug, if it a configuration issue, etc.

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

  1. Plain install (extract tar.bz2 into /var/www/html, configure with mysql, etc)
  2. Load an app in the browser (files, photos)
  3. Problem occurs

The output of your Nextcloud log in Admin > Logging:

These all seem to be about losing connection to mysql when the server is rebooted

Error	core	Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2002] No such file or directory		2021-08-03T13:34:13+0100
Error	core	Doctrine\DBAL\Exception\ConnectionLost: An exception occurred while executing a query: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away		2021-08-03T13:34:13+0100
Error	index	Doctrine\DBAL\Exception\ConnectionLost: An exception occurred while executing a query: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away		2021-08-03T13:34:13+0100
Error	no app in context	Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2002] No such file or directory		2021-08-03T13:34:12+0100
Error	PHP	Doctrine\DBAL\Exception\ConnectionLost: An exception occurred while executing a query: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away at /var/vhosts/nextcloud.james-hewitt.co.uk/html/3rdparty/doctrine/dbal/src/Driver/API/MySQL/ExceptionConverter.php#101		2021-08-03T13:34:12+0100
Error	core	Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2002] No such file or directory		2021-08-03T13:34:12+0100
Error	core	Doctrine\DBAL\Exception\ConnectionLost: An exception occurred while executing a query: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away		2021-08-03T13:34:07+0100
Fatal	no app in context	Doctrine\DBAL\Exception\ConnectionLost: An exception occurred while executing a query: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away		2021-08-03T13:34:12+0100
Error	index	Doctrine\DBAL\Exception\ConnectionLost: An exception occurred while executing a query: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away		2021-08-03T13:34:06+0100
Error	PHP	OC\DB\Exceptions\DbalException: An exception occurred while executing a query: SQLSTATE[08S01]: Communication link failure: 1053 Server shutdown in progress at /var/vhosts/nextcloud.james-hewitt.co.uk/html/lib/private/DB/Exceptions/DbalException.php#71

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'redacted',
  'passwordsalt' => 'redacted',
  'secret' => 'redacted',
  'trusted_domains' => 
  array (
    0 => 'redacted',
  ),
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/var/vhosts/redacted/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/vhosts/redacted/html/apps-addons',
      'url' => '/apps-addons',
      'writable' => true,
    ),
  ),
  'datadirectory' => '/var/vhosts/redacted/data/',
  'dbtype' => 'mysql',
  'version' => '22.0.0.11',
  'overwrite.cli.url' => 'http://redacted/nextcloud',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud',
  'dbpassword' => 'redacted',
  'installed' => true,
  'mail_from_address' => 'nextcloud',
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'mail_domain' => 'redacted',
  'mail_smtphost' => 'localhost',
  'mail_smtpport' => '25',
  'default_phone_region' => 'GB',
  'blacklisted_files' => 
  array (
  ),
  'objectstore' => 
  array (
    'class' => '\\OC\\Files\\ObjectStore\\S3',
    'arguments' => 
    array (
      'bucket' => 'jammy-nextcloud',
      'autocreate' => true,
      'key' => 'redacted',
      'secret' => 'redacted',
      'hostname' => 's3.dualstack.eu-west-1.amazonaws.com',
      'port' => 443,
      'use_ssl' => true,
      'use_path_style' => true,
    ),
  ),
  'maintenance' => false,
  'updater.secret' => 'redacted',
  'loglevel' => 2,
);

The output of your Apache/nginx/system log in /var/log/____:

From error.log (these were on reboot I think, the mysql connection is fine)

[Tue Aug 03 12:34:12.316455 2021] [php7:error] [pid 4367] [client 90.255.1.94:48338] PHP Fatal error:  Uncaught Doctrine\\DBAL\\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2002] No such file or directory in /var/vhosts/nextcloud.james-hewitt.co.uk/html/lib/private/DB/Connection.php:87\nStack trace:\n#0 /var/vhosts/nextcloud.james-hewitt.co.uk/html/3rdparty/doctrine/dbal/src/Connection.php(1486): OC\\DB\\Connection->connect()\n#1 /var/vhosts/nextcloud.james-hewitt.co.uk/html/3rdparty/doctrine/dbal/src/Connection.php(1014): Doctrine\\DBAL\\Connection->getWrappedConnection()\n#2 /var/vhosts/nextcloud.james-hewitt.co.uk/html/lib/private/DB/Connection.php(231): Doctrine\\DBAL\\Connection->executeQuery()\n#3 /var/vhosts/nextcloud.james-hewitt.co.uk/html/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php(210): OC\\DB\\Connection->executeQuery()\n#4 /var/vhosts/nextcloud.james-hewitt.co.uk/html/lib/private/DB/QueryBuilder/QueryBuilder.php(287): Doctrine\\DBAL\\Query\\QueryBuilder->execute()\n#5 /var/vhosts/nextcloud.james-hewitt.co.uk/html/lib/private/AppConfig.php(344): OC\\DB\\QueryB in /var/vhosts/nextcloud.james-hewitt.co.uk/html/lib/private/DB/Connection.php on line 87
[Tue Aug 03 12:34:13.670645 2021] [php7:error] [pid 1439] [client 90.255.1.94:48330] PHP Fatal error:  Uncaught Doctrine\\DBAL\\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2002] No such file or directory in /var/vhosts/nextcloud.james-hewitt.co.uk/html/lib/private/DB/Connection.php:87\nStack trace:\n#0 /var/vhosts/nextcloud.james-hewitt.co.uk/html/3rdparty/doctrine/dbal/src/Connection.php(1486): OC\\DB\\Connection->connect()\n#1 /var/vhosts/nextcloud.james-hewitt.co.uk/html/3rdparty/doctrine/dbal/src/Connection.php(1014): Doctrine\\DBAL\\Connection->getWrappedConnection()\n#2 /var/vhosts/nextcloud.james-hewitt.co.uk/html/lib/private/DB/Connection.php(231): Doctrine\\DBAL\\Connection->executeQuery()\n#3 /var/vhosts/nextcloud.james-hewitt.co.uk/html/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php(210): OC\\DB\\Connection->executeQuery()\n#4 /var/vhosts/nextcloud.james-hewitt.co.uk/html/lib/private/DB/QueryBuilder/QueryBuilder.php(287): Doctrine\\DBAL\\Query\\QueryBuilder->execute()\n#5 /var/vhosts/nextcloud.james-hewitt.co.uk/html/lib/private/AppConfig.php(344): OC\\DB\\QueryB in /var/vhosts/nextcloud.james-hewitt.co.uk/html/lib/private/DB/Connection.php on line 87

From access.log:

90.255.1.94 - - [03/Aug/2021:12:36:47 +0000] "GET /ocs/v2.php/apps/text/workspace?path=%2F HTTP/1.1" 404 991 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
90.255.1.94 - - [03/Aug/2021:12:36:47 +0000] "GET /index.php/apps/systemtags/lastused HTTP/1.1" 200 887 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
90.255.1.94 - - [03/Aug/2021:12:36:47 +0000] "PROPFIND /remote.php/dav/files/jammy/ HTTP/1.1" 207 2175 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
90.255.1.94 - - [03/Aug/2021:12:36:48 +0000] "GET /index.php/apps/files/ajax/getstoragestats.php?dir=%2F HTTP/1.1" 200 1056 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
90.255.1.94 - - [03/Aug/2021:12:36:47 +0000] "GET /index.php/apps/recommendations/api/recommendations HTTP/1.1" 200 1189 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
90.255.1.94 - - [03/Aug/2021:12:36:48 +0000] "GET /index.php/apps/theming/img/core/filetypes/folder.svg?v=0 HTTP/1.1" 200 1227 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
90.255.1.94 - - [03/Aug/2021:12:36:48 +0000] "GET /ocs/v1.php/apps/files_external/api/v1/mounts?format=json HTTP/1.1" 200 969 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
90.255.1.94 - - [03/Aug/2021:12:36:48 +0000] "GET /index.php/apps/theming/img/core/filetypes/text.svg?v=0 HTTP/1.1" 200 1265 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
90.255.1.94 - - [03/Aug/2021:12:36:48 +0000] "GET /index.php/apps/theming/img/core/filetypes/application-pdf.svg?v=0 HTTP/1.1" 200 2036 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
90.255.1.94 - - [03/Aug/2021:12:36:48 +0000] "GET /index.php/core/preview?fileId=5225&c=6042352417e3d&x=250&y=250&forceIcon=0&a=0 HTTP/1.1" 200 4360 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"

On a hunch (preview was the last access log entry) I turned off previews and that seems to have solved the problem.

So, how do I work out why previews are breaking my server?