Server hangs for a minute when uploading folder with data

I have server with Nextcloud 14.0.2 RC 1, Debian Buster, PHP 7.2. APCu and OPcache enabled.

Network speed (wired): 150 Mbps incoming / 15 Mbps upcoming

Server works very fast in idle mode, so web interface very responsive in NC pages surfing. When somebody of my users starts to upload file or folder (by Drag’n’Drop), server hangs for a minute. Server really doesn’t start file uploading, so user see “Uploading…” if don’t update page. If he update page, it will not load for a while.

Not sure it’s the same problem, also when I go to “fresh” folder (with photos) and scroll down quickly, NC creating a thumbnails or doing something and then server dies for a minute: page doesn’t load for a while when trying to update it, sometimes it returns “nginx 502 bad gateway” page.

Server configuration

Operating system: Debian Buster

Web server: Nginx/1.14.0

Database: 10.1.35-MariaDB-1

PHP version: PHP 7.2.9-1

Nextcloud version: 14.0.2 RC 1

Signing status:

Signing status
No errors have been found.

Nextcloud configuration:

Config report
{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "14.0.2.0",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "maintenance": false,
        "theme": "",
        "loglevel": 1,
        "updater.release.channel": "beta"
    }
}

Are you using external storage, if yes which one: no

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Browser: Chrome latest

Operating system: Debian Buster

Logs

Web server error log

Web server error log
2018/09/03 01:13:22 [error] 8219#8219: *26902 upstream timed out (110: Connection timed out) while reading response header from upstream, client: *REMOVED IP*, server: *REMOVED SERVER*, request: "GET /settings/ajax/checksetup HTTP/2.0", upstream: "fastcgi://unix:/run/php/php7.2-fpm.sock", host: *REMOVED HOST*

Hardware info

hwinfo --short
cpu:
                       AMD Phenom(tm) II X6 1100T Processor, 803 MHz
                       AMD Phenom(tm) II X6 1100T Processor, 803 MHz
                       AMD Phenom(tm) II X6 1100T Processor, 803 MHz
                       AMD Phenom(tm) II X6 1100T Processor, 803 MHz
                       AMD Phenom(tm) II X6 1100T Processor, 804 MHz
                       AMD Phenom(tm) II X6 1100T Processor, 803 MHz
graphics card:
                       ATI Turks XT [Radeon HD 6670/7670]
sound:
                       ATI Turks HDMI Audio [Radeon HD 6500/6600 / 6700M Series]
                       ASUSTeK M5A88-V EVO
storage:
                       ATI SB7x0/SB8x0/SB9x0 IDE Controller
                       VIA VT6415 PATA IDE Host Controller
                       ATI SB7x0/SB8x0/SB9x0 SATA Controller [IDE mode]
network:
  enp6s0               Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
  wlp4s7               D-Link DWA-525 Wireless N 150 Desktop Adapter (rev.A2)
network interface:
  enp6s0               Ethernet network interface
  lo                   Loopback network interface
  wlp4s7               Ethernet network interface
disk:
  /dev/sda             WDC WD5001AALS-0
partition:
  /dev/sda1            Partition
  /dev/sda2            Partition
  /dev/sda3            Partition
usb controller:
                       ASMedia ASM1042 SuperSpeed USB Host Controller
                       ASUSTeK M5A88-V EVO
                       ASUSTeK M5A88-V EVO
                       ASUSTeK M5A88-V EVO
                       ASUSTeK M5A88-V EVO
                       ASUSTeK M5A88-V EVO
                       ASUSTeK M5A88-V EVO
                       ASUSTeK M5A88-V EVO
bios:
                       BIOS
bridge:
                       AMD Family 10h Processor Miscellaneous Control
                       ATI SB700/SB800/SB900 PCI to PCI bridge (PCIE port 1)
                       AMD Family 10h Processor Address Map
                       ASUSTeK M5A88-V EVO
                       ASUSTeK M5A88-V EVO
                       AMD Family 10h Processor Link Control
                       AMD Family 10h Processor DRAM Controller
                       ATI SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0)
                       AMD Family 10h Processor HyperTransport Configuration
                       ATI SBx00 PCI to PCI Bridge
                       AMD RS780/RS880 PCI to PCI bridge (PCIE port 4)
                       AMD RS780 PCI to PCI bridge (ext gfx port 0)
                       AMD RS780/RS880 PCI to PCI bridge (PCIE port 5)
hub:
                       Linux Foundation 1.1 root hub
                       Linux Foundation 3.0 root hub
                       Linux Foundation 1.1 root hub
                       Linux Foundation 2.0 root hub
                       Linux Foundation 1.1 root hub
                       Linux Foundation 2.0 root hub
                       Linux Foundation 2.0 root hub
                       Linux Foundation 1.1 root hub
                       Linux Foundation 2.0 root hub
memory:
                       Main Memory
firewire controller:
                       ASUSTeK M5A88-V EVO
unknown:
                       FPU
                       DMA controller
                       PIC
                       Keyboard controller
                       PS/2 Controller
                       ATI SBx00 SMBus Controller
  /dev/ttyS0           16550A

Screenshots

User uploads folder and server hangs

Scrolled down and NC updates thumbnails

It looks for me like your mysql is taking too long and then your nginx<->fastcgi link times out before the procedure finishes.

I’d first try to improve the mysql performance. If you don’t already use redis as a file locking cache, do so because it reduces the load on the database. Then you can review your mysql cache settings, there are scripts/tools to help you and you should as well find a few topics here.

Normally you should get it fast enough that you don’t have to increase the timeout settings.

Thanks for the answer!

I installed and configured Redis the server, switched NC to use Redis instead of APCu, but I see no positive effect, only small regression. For example, sometimes I get 504 gateway time-out when surfing over settings tabs, or on “Downloading” stage of NC updater.

After all I checked MariaDB settings manually and using MySQLTuner-perl, but it shows me none of important optimization points.

My settings:
https://pastebin.com/UF787zph - NC config
https://pastebin.com/LYxMJfzT - Redis config
https://pastebin.com/ZH1HDJBd - Nginx config
https://pastebin.com/FksKPv9U - php-fpm config
https://pastebin.com/VPAgVEu3 - mysql tuner output