Can't fix 499 errors after 60 seconds using NGINX/PHP-FPM

ℹ️ Support 📦 Appliances (Docker, Snappy, VM, NCP, AIO)
,
1

Running NextCloud 20 on custom baked docker image using Ubuntu 20.04 as the base image. Runing NGINX/PHP-FPM 7.4.

Large uploads from the mobile app fail and /var/log/nginx/access.log shows 499 status codes for MOVE operations after 60 seconds.

I’ve found plenty of articles around to change the 60 second timeout and I have tried many iterations, but none of them seem to change to 60 second timout/499 response.

Currently I have the following parameter configured:

/etc/php/7.4/fpm/php.ini

memory_limit = 16G
max_execution_time = 600
max_input_time = 600
default_socket_timeout = 360
post_max_size = 8G
upload_max_filesize = 8G

/etc/php/7.4/fpm/pool.d/www.conf
request_terminate_timeout = 600

/etc/nginx/sites-enabled/nextcloud.conf (in location ~ .php section)

fastcgi_read_timeout 360;
fastcgi_send_timeout 360;
fastcgi_connect_timeout 1200;

I am completely dumb at PHP and don’t really understand the differences between php.ini, php-fpm.conf, conf.d/.ini, and pool.d/.conf config files and the various sections ([php] vs [global] vs [www]). So it’s very possible I am putting these settings in the wrong places.

If these settings aren’t in the wrong places, any suggestions for other parameters to change or how to determine why the 60 second timeout is still being enforced?

Thanks in advance for any help!

2

I’ve tried a few more things with no luck

  • Upgraded to 21.0.4
  • Updated parameters in CLI php.ini file (I knew this was a long shot)
  • Updated parameters in .user.ini

I added slow log tracing and here is the trace ~15 second into the transaction:

[28-Sep-2021 14:37:57]  [pool www] pid 2362
script_filename = /var/www/nextcloud/remote.php
[0x00007f8df8017550] hash_hmac() /mnt/ssd/apps/encryption/lib/Crypto/Crypt.php:520
[0x00007f8df80174d0] createSignature() /mnt/ssd/apps/encryption/lib/Crypto/Crypt.php:199
[0x00007f8df80173e0] symmetricEncryptFileContent() /mnt/ssd/apps/encryption/lib/Crypto/Encryption.php:347
[0x00007f8df8017320] encrypt() /var/www/nextcloud/lib/private/Files/Stream/Encryption.php:488
[0x00007f8df8017250] flush() /var/www/nextcloud/lib/private/Files/Stream/Encryption.php:400
[0x00007f8df8017170] stream_write() /var/www/nextcloud/lib/private/legacy/OC_Helper.php:261
[0x00007f8df8017100] fwrite() /var/www/nextcloud/lib/private/legacy/OC_Helper.php:261
[0x00007f8df8017030] streamCopy() /var/www/nextcloud/lib/private/Files/Storage/Wrapper/Encryption.php:1036
[0x00007f8df8016f60] writeStream() /var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php:629
[0x00007f8df8016e80] writeStream() /mnt/ssd/apps/dav/lib/Connector/Sabre/File.php:207
[0x00007f8df8016c70] put() /mnt/ssd/apps/dav/lib/Connector/Sabre/Directory.php:156
[0x00007f8df8016b60] createFile() /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php:305
[0x00007f8df8016a60] copyNode() /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php:133
[0x00007f8df8016990] copy() /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php:163
[0x00007f8df80168a0] move() /mnt/ssd/apps/dav/lib/Upload/ChunkingPlugin.php:95
[0x00007f8df80167d0] performMove() /mnt/ssd/apps/dav/lib/Upload/ChunkingPlugin.php:77
[0x00007f8df80166e0] beforeMove() /var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php:89
[0x00007f8df8016600] emit() /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php:603
[0x00007f8df8016530] httpMove() /var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php:89
[0x00007f8df8016450] emit() /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php:472
[0x00007f8df8016390] invokeMethod() /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php:253
[0x00007f8df8016290] start() /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php:321
[0x00007f8df8016240] exec() /mnt/ssd/apps/dav/lib/Server.php:332
[0x00007f8df80161e0] exec() /mnt/ssd/apps/dav/appinfo/v2/remote.php:35
[0x00007f8df8016140] [INCLUDE_OR_EVAL]() /var/www/nextcloud/remote.php:167

Open to any and all suggestions or debugging tips.

Thanks!

3

So I tested it manually with curl commands and I was able to upload 2GB of random data. The MOVE took 124 seconds and returned a 201 status code. So it appears that my server is configured correctly and the issue is with the app.

I have submitted a bug report on github.

4

I’ve been having the same issue on the Windows client, and I can’t find a solution. Would it be the same bug?