Out Of Memory And Disk When Download Big File

Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version (eg, 12.0.2):17.0.2
Operating system and version (eg, Ubuntu 17.04):centos6
Apache or nginx version (eg, Apache 2.4.25): nginx 1.16.1
PHP version (eg, 7.1):7.2

The issue you are facing:
The vps I am using have 512M ram, 1G swap, and 55% used of disk. When I am download file from my server, it will out of memory or fill up disk space with fastcgi_temp files in a minute. Cause mysql or php-fpm crash.

Is this the first time you’ve seen this error? (Y/N):N

Steps to replicate it:

  1. download some big file from server.
  2. out of memory -> mysql crash / out of disk -> php-fpm no responded.
  3. nextcloud crash.

The output of your Nextcloud log in Admin > Logging:

Sabre\DAV\Exception\ServiceUnavailable: Doctrine\DBAL\DBALException: Failed to connect to the database: An exception occurred in driver: SQLSTATE[HY000] [2002] Connection refused

Doctrine\DBAL\DBALException: Failed to connect to the database: An exception occurred in driver: SQLSTATE[HY000] [2002] Connection refused

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

$CONFIG = array (
  'instanceid' => '',
  'passwordsalt' => '',
  'secret' => '',
  'trusted_domains' => 
  array (
    0 => 'pan.tamabi.de',
  ),
  'datadirectory' => '/blockStorage0/site/nextcloud',
  'dbtype' => 'mysql',
  'version' => '17.0.2.1',
  'overwrite.cli.url' => 'https://pan.tamabi.de',
  'dbname' => '',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => '',
  'dbpassword' => '',
  'installed' => true,
  'mail_smtpmode' => '',
  'mail_smtpsecure' => '',
  'mail_sendmailmode' => '',
  'mail_smtpauth' => 1,
  'mail_from_address' => '',
  'mail_domain' => '',
  'mail_smtphost' => '',
  'mail_smtpport' => '',
  'mail_smtpname' => '',
  'mail_smtppassword' => '',
  'mail_smtpauthtype' => '',
  'maintenance' => false,
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'updater.secret' => '',
  'theme' => '',
  'loglevel' => 2,
);

The output of your Apache/nginx/system log in /var/log/____:

2019/12/23 09:16:31 [crit] 14157#14157: *60581 pwritev() "/var/cache/nginx/fastcgi_temp/7/31/0000000317" has written only 3953 of 8184 while reading upstream, client: 120.84.12.153, server: pan.tamabi.de, request: "GET /remote.php/webdav/Downloads/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ.mkv?downloadStartSecret=vpbdi64mcn HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-cgi.sock:", host: "pan.tamabi.de"
2019/12/23 09:16:31 [alert] 14157#14157: *60581 write() to "/var/log/nginx/access.log" was incomplete: 183 of 402 while logging request, client: 120.84.12.153, server: pan.tamabi.de, request: "GET /remote.php/webdav/Downloads/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ.mkv?downloadStartSecret=vpbdi64mcn HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-cgi.sock", host: "pan.tamabi.de"
2019/12/23 09:16:31 [crit] 14157#14157: *60576 pwritev() "/var/cache/nginx/fastcgi_temp/8/31/0000000318" has written only 5217 of 8184 while reading upstream, client: 120.84.12.153, server: pan.tamabi.de, request: "GET /remote.php/webdav/Downloads/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ.mkv?downloadStartSecret=vpbdi64mcn HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-cgi.sock:", host: "pan.tamabi.de"
2019/12/23 09:16:32 [crit] 14157#14157: *60577 pwritev() "/var/cache/nginx/fastcgi_temp/6/31/0000000316" has written only 4305 of 8184 while reading upstream, client: 120.84.12.153, server: pan.tamabi.de, request: "GET /remote.php/webdav/Downloads/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ.mkv?downloadStartSecret=vpbdi64mcn HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-cgi.sock:", host: "pan.tamabi.de"
2019/12/23 09:16:34 [crit] 14157#14157: *60575 pwritev() "/var/cache/nginx/fastcgi_temp/5/31/0000000315" has written only 4809 of 8184 while reading upstream, client: 120.84.12.153, server: pan.tamabi.de, request: "GET /remote.php/webdav/Downloads/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ.mkv?downloadStartSecret=vpbdi64mcn HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-cgi.sock:", host: "pan.tamabi.de"
2019/12/23 09:16:54 [crit] 14157#14157: *60583 pwritev() "/var/cache/nginx/fastcgi_temp/9/31/0000000319" has written only 5433 of 8184 while readi2019/12/23 09:28:20 [error] 14157#14157: *60650 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 120.84.12.153, server: pan.tamabi.de, request: "GET /ocs/v2.php/apps/notifications/api/v2/notifications HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-cgi.sock", host: "pan.tamabi.de"
2019/12/23 09:29:11 [error] 14157#14157: *60694 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 120.84.12.153, server: pan.tamabi.de, request: "GET /ocs/v2.php/apps/notifications/api/v2/notifications HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-cgi.sock", host: "pan.tamabi.de"
2019/12/23 09:29:20 [error] 14157#14157: *60650 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 120.84.12.153, server: pan.tamabi.de, request: "GET /ocs/v2.php/apps/notifications/api/v2/notifications HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-cgi.sock", host: "pan.tamabi.de"
2019/12/23 09:31:03 [error] 14157#14157: *60700 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 156.243.11.85, server: pan.tamabi.de, request: "GET /remote.php/webdav/Downloads/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ.mkv?downloadStartSecret=vpbdi64mcn HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-cgi.sock", host: "pan.tamabi.de"
2019/12/23 09:31:23 [error] 14157#14157: *60705 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 156.243.11.85, server: pan.tamabi.de, request: "GET /apps/files/?dir=/&fileid=6 HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-cgi.sock:", host: "pan.tamabi.de"
2019/12/23 09:31:23 [error] 14157#14157: *60707 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 156.243.11.85, server: pan.tamabi.de, request: "GET /remote.php/webdav/Downloads/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ/1080p.BluRay.x264.DTS-X.7.1-SWTYBLZ.mkv?downloadStartSecret=vpbdi64mcn HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-cgi.sock:", host: "pan.tamabi.de"
2019/12/23 09:31:24 [crit] 14157#14157: *60705 connect() to unix:/dev/shm/php-cgi.sock failed (2: No such file or directory) while connecting to upstream, client: 156.243.11.85, server: pan.tamabi.de, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://unix:/dev/shm/php-cgi.sock:", host: "pan.tamabi.de"

What have you done to configure your system for 512MB RAM?

Are you aware the stock MariaDB/MySQL uses 300MB RES? Are you aware each php-fpm process uses 50MB? Nginx uses 20MB.

If I recall correctly the default php-fpm goes up to 6 child processes out of the box so that’s 300MB right there.

And just because you have 512MB RAM doesn’t mean you can use 512MB. You want spare resources for the rest of the OS.

Thank you for your answer. I have the extra 1G swap on it. The nextcloud documend said it need minimal 128m but recommand 512m. Ususally the whole system using 300M ram. And got 1200M free. So I guess may be there are some problem else.
image

okay, seems like actually cause by low ram, the download tool I am using will generate 24 connection to server. When I reduce it to 8, the ploblem just gone. :neutral_face: