Slow gallery performance with NC 13.0

First time really posting here, forgive me if I miss something. I’ve been running nextcloud in a kubernetes cluster for a little while now, and overall it works great. The only problem is that it seems to be a little sluggish, especially around the gallery (when in preview mode, list mode works fine). I’m not sure if it’s related, but I notice that when I try to open the gallery, redis performs a TON of activity for about 20 seconds, and then images start to load. The redis log (via monitor in redis-cli) is below:

1534129230.740390 [0 127.0.0.1:44528] "INCRBY" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/1f2ac3c2ee5ccd168b45a06e1f0de5f7" "1"
1534129230.740519 [0 127.0.0.1:44528] "EXPIRE" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/1f2ac3c2ee5ccd168b45a06e1f0de5f7" "3600"
1534129230.740902 [0 127.0.0.1:44528] "WATCH" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/1f2ac3c2ee5ccd168b45a06e1f0de5f7"
1534129230.740988 [0 127.0.0.1:44528] "GET" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/1f2ac3c2ee5ccd168b45a06e1f0de5f7"
1534129230.741089 [0 127.0.0.1:44528] "MULTI"
1534129230.741225 [0 127.0.0.1:44528] "DEL" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/1f2ac3c2ee5ccd168b45a06e1f0de5f7"
1534129230.741255 [0 127.0.0.1:44528] "EXEC"
1534129230.741489 [0 127.0.0.1:44528] "INCRBY" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/3cf8e045b87c8474253c64a2aa4cfb64" "1"
1534129230.741573 [0 127.0.0.1:44528] "EXPIRE" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/3cf8e045b87c8474253c64a2aa4cfb64" "3600"
1534129230.741940 [0 127.0.0.1:44528] "WATCH" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/3cf8e045b87c8474253c64a2aa4cfb64"
1534129230.742023 [0 127.0.0.1:44528] "GET" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/3cf8e045b87c8474253c64a2aa4cfb64"
1534129230.742109 [0 127.0.0.1:44528] "MULTI"
1534129230.742237 [0 127.0.0.1:44528] "DEL" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/3cf8e045b87c8474253c64a2aa4cfb64"
1534129230.742267 [0 127.0.0.1:44528] "EXEC"
1534129230.742446 [0 127.0.0.1:44528] "INCRBY" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/778ad2355dc827ad3f54eae6a7491f52" "1"
1534129230.742528 [0 127.0.0.1:44528] "EXPIRE" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/778ad2355dc827ad3f54eae6a7491f52" "3600"
1534129230.742864 [0 127.0.0.1:44528] "WATCH" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/778ad2355dc827ad3f54eae6a7491f52"
1534129230.742938 [0 127.0.0.1:44528] "GET" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/778ad2355dc827ad3f54eae6a7491f52"
1534129230.743016 [0 127.0.0.1:44528] "MULTI"
1534129230.743135 [0 127.0.0.1:44528] "DEL" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/778ad2355dc827ad3f54eae6a7491f52"
1534129230.743166 [0 127.0.0.1:44528] "EXEC"
1534129230.743394 [0 127.0.0.1:44528] "INCRBY" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/582ea19cfae350a96d1964d9b03beb86" "1"
1534129230.743474 [0 127.0.0.1:44528] "EXPIRE" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/582ea19cfae350a96d1964d9b03beb86" "3600"
1534129230.743819 [0 127.0.0.1:44528] "WATCH" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/582ea19cfae350a96d1964d9b03beb86"
1534129230.743883 [0 127.0.0.1:44528] "GET" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/582ea19cfae350a96d1964d9b03beb86"
1534129230.743946 [0 127.0.0.1:44528] "MULTI"
1534129230.744025 [0 127.0.0.1:44528] "DEL" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/582ea19cfae350a96d1964d9b03beb86"
1534129230.744048 [0 127.0.0.1:44528] "EXEC"
1534129230.744193 [0 127.0.0.1:44528] "INCRBY" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/e56f3238e65ff6828b9eab24ba965a4b" "1"
1534129230.744268 [0 127.0.0.1:44528] "EXPIRE" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/e56f3238e65ff6828b9eab24ba965a4b" "3600"
1534129230.744547 [0 127.0.0.1:44528] "WATCH" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/e56f3238e65ff6828b9eab24ba965a4b"
1534129230.744609 [0 127.0.0.1:44528] "GET" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/e56f3238e65ff6828b9eab24ba965a4b"
1534129230.744675 [0 127.0.0.1:44528] "MULTI"
1534129230.744759 [0 127.0.0.1:44528] "DEL" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/e56f3238e65ff6828b9eab24ba965a4b"
1534129230.744782 [0 127.0.0.1:44528] "EXEC"
1534129230.744914 [0 127.0.0.1:44528] "INCRBY" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/f968d494a03a091f80d9a82c744d6ac0" "1"
1534129230.744981 [0 127.0.0.1:44528] "EXPIRE" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/f968d494a03a091f80d9a82c744d6ac0" "3600"
1534129230.745258 [0 127.0.0.1:44528] "WATCH" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/f968d494a03a091f80d9a82c744d6ac0"
1534129230.745326 [0 127.0.0.1:44528] "GET" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/f968d494a03a091f80d9a82c744d6ac0"
1534129230.745388 [0 127.0.0.1:44528] "MULTI"
1534129230.745469 [0 127.0.0.1:44528] "DEL" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/f968d494a03a091f80d9a82c744d6ac0"
1534129230.745492 [0 127.0.0.1:44528] "EXEC"
1534129230.745646 [0 127.0.0.1:44528] "INCRBY" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/14f409b4a68a7a877c3689727282a6a4" "1"
1534129230.745719 [0 127.0.0.1:44528] "EXPIRE" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/14f409b4a68a7a877c3689727282a6a4" "3600"
1534129230.746014 [0 127.0.0.1:44528] "WATCH" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/14f409b4a68a7a877c3689727282a6a4"
1534129230.746078 [0 127.0.0.1:44528] "GET" "8e2996a5a54edc611c8507db6e8fbf89/lockfiles/14f409b4a68a7a877c3689727282a6a4"
1534129230.746161 [0 127.0.0.1:44528] "MULTI"

I’m running linuxserver.io’s version of nextcloud, tagged release 131, but here are my version numbers:

Nextcloud: 13.04
OS: Ubuntu for K8s, alpine for the container
nginx: 1.12.2
PHP: 7.1.17
Redis: 4.0.11
MariaDB: 10.3.8

Redis, MariaDB and nextcloud are all containers within the same pod, if that helps. Nextcloud’s config folder (php, nginx, www, etc) is stored locally on SSD Raid 5 array on the node, everything else mounts over nfs.

MariaDB data and logs are hosted on an NFS volume with the following options:

 mountOptions:
    - hard
    - intr
    - noexec
    - rsize=16384
    - wsize=16384
    - noatime
    - nfsvers=4.1

logs are mounted:

 mountOptions:
    - hard
    - intr
    - noexec
    - rsize=131072
    - wsize=131072
    - noatime
    - nfsvers=4.1

NFS data for nextcloud to write files to is mounted with:

  mountOptions:
    - hard
    - noexec
    - rsize=32768
    - wsize=32768
    - nfsvers=4.1
    - intr

The output of your Nextcloud log in Admin > Logging:

Can't get this, it just spins

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' =>
  array (
    'host' => '127.0.0.1',
    'port' => 6379,
  ),
  'datadirectory' => '/data',
  'instanceid' => '123456789',
  'passwordsalt' => 'saltvalue',
  'secret' => 'secretvalue',
  'trusted_domains' =>
  array (
    0 => 'redacted',
    1 => 'redacted',
    2 => 'redacted',
  ),
  'overwrite.cli.url' => 'redacted',
  'overwritehost' => 'redacted',
  'dbtype' => 'mysql',
  'version' => '13.0.4.0',
  'installed' => true,
  'maintenance' => false,
  'theme' => '',
  'log_type' => 'syslog',
  'logfile' => '',
  'loglevel' => '3',
  'dbname' => 'nextcloud',
  'dbhost' => '127.0.0.1:3306',
  'dbuser' => 'nextcloud',
  'dbpassword' => 'dbpass',
  'mail_smtpmode' => 'smtp',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_from_address' => 'cloudadmin',
  'mail_domain' => '<maildomain>',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'mail.host.com',
  'mail_smtpport' => '465',
  'mail_smtpsecure' => 'ssl',
  'mail_smtpname' => 'username',
  'mail_smtppassword' => 'password',
  'updater.release.channel' => 'production',
);

The output of your Apache/nginx/system log in /var/log/____:

Empty

The only thing I could think of is that it might be the database being mounted over NFS, but mysql is more forgiving than sqlite, so I’m not sure it’s that, but I can correlate a TON of that redis-cli stuff every time i try to do something…and it hangs for about 20 seconds.

Thanks!