Web UI extremely slow - NC24

ℹ️ Support 💬 Talk (spreed)
1

Ciao,

Nextcloud version (eg, 24.0.1): 24.0.7
Talk Server version (eg, 14.0.2): 14.0.2

we’re experiencing really slow web UI in NC24. This is the first time it happens, we develop NC since a few years. Posting under Talk (spreed) because conversations seems to be the problem, but we’re NOT using the videoconference system - only the chats

  • test with 30 total users, 10 conversations

20 users dispatched in 5 conversations, plus another 10 users in 5 support conversations

install method: manually, Apache on Ubuntu LTS 20.04

NC server (8 core - 8GB RAM) did not show any stress - load average always below 1.0

chatting in conversations were extremely slow - the UI was unresponsive. sending messages was nearly impossible for some users, and very slow for everyone.

we cannot update Talk to the last version because we’re working on a a fork, but this do not involve the videoconference system (we’re not using it, only the chats in conversations). We tested also with a clean Talk 14.0.2 install and the Web UI slow is the same.

The support team is supposed to jump from one support channel to another - 5 support channels. It took up to 40’ to load one conversation. We don’t have any clue.

Nextcloud server specs:

Ubuntu 20.04.1
Nextcloud 24.0.7.1
RAM 8GB
8 core
Redis, OPcache, etc.

share of /serverinfo/api/v1/info

so, nothing… we disabled all possible not-so-important apps,

any idea would be higly appreciated, thanks!

k.

2

Hey, when reading your issue, this one came to my mind: Nextcloud Talk is really slow when using HTTP2 · Issue #2211 · nextcloud/spreed · GitHub

3

thanks, the Syskeeper works well, but that issue is from 2-3 years old. The system we’re talking about uses NC 24 and Apache, which is the webserver officially supported in the doc - Installation on Linux — Nextcloud latest Administration Manual latest documentation

we’ll try turning off HTTP2 but in any case it’s not a good solution,

k.

4

It’s not dependant on how old the issue is and it’s not meant against apache :slight_smile:
But apache, php-fpm, mpm_prefork and http2 is not working well with Talk. See requirements System requirements - Nextcloud Talk API documentation

5

thanks, but I don’t understand.

We have Apache2 + PHP FPM + mpm_events. The server is dedicated, with 8GB RAM and 8 core. Cutting HTTP" does not change the UI extremely slowness - but, this settings make sense for 50 contemporary users chatting? we’re not using the videocall system.

or, it’ better to have mod_php + mpm_prefork?

thanks,

k.

6

That’s a supported configuration and should not result in any slowdown (atleast from this point of view). You might want to look into how many fpm processes are available and if where the bottleneck really is (FPM? DB? etc.).
Maybe you can post your server configuration, so others can take a look.

7

OK. this is now with only 4 users, still very slow UI. filtering fpm processes

Screenshot 2022-11-28 13-51-01
Screenshot 2022-11-28 13-51-011358Ă—668 117 KB

as per the server config,

Nextcloud server specs:

Ubuntu 20.04.1
Nextcloud 24.0.7.1
RAM 8GB
8 core
Apache2, PHP, Redis

share of /serverinfo/api/v1/info

what should be useful to know about server conf?

thanks,

k.

8

Maybe a DB problem then?

9

yes, this is our feeling, too. We changed a few conf in the server, following Memory caching — Nextcloud latest Administration Manual latest documentation namely:

  • added

    'memcache.distributed' => '\OC\Memcache\Redis',

and

 'dbindex'  => 0,

to config.php

restarting the server > the UI is responsive, but only for a few minutes.

Then, apparently, nothing changed. with only 5 active users on 5 conversations, UI is still terribly slow. the main conversation’s window in particular is slow - the sidebars with lists of conversations on the left and new messages in bold are a bit more responsive, but jumping from one conversation to another is nearly to impossible, up to 2 minutes…

Sure, the DB could be optimised. We’re working on MySQL suggestions, slow queries, tables opening, etc.

Still, the strange thing is, on a dev machine with 1/4 of the power (2GB RAM and 2core instead of 8GB RAM and 8core, same datacenter, same OS Ubuntu 20.04, same NC and Talk versions, same confs everywhere), the 5 users jump decently among 5 and even 10 conversations.

The only differences are

  • the number of messages in the conversations (a few hundreds on dev, a few thousands in production)
  • the number of total conversations (20 in dev, 180 in productions) - but no more than 10 active at the same time, in both cases

a final thought is about JS - could javascript minification-optimisation significantly improve the loading of the conversations tabs?

thanks for your suggestions,

k.

10

Ciao,

finally, we found the solution! Looking at the DB, the strange thing was, MySQL query execution was fast from the command line, but then the PHP rendering was slow. This pointed out to review the php-fpm pool of connections, in the end was pretty easy:

increase the number of child in php-fpm,

vi /etc/php/8.0/fpm/pool.d/www.conf

something like this:

; The desired minimum number of idle server processes.
; Note: Used only when pm is set to 'dynamic'
; Note: Mandatory when pm is set to 'dynamic'
pm.min_spare_servers = 100

; The desired maximum number of idle server processes.
; Note: Used only when pm is set to 'dynamic'
; Note: Mandatory when pm is set to 'dynamic'
pm.max_spare_servers = 150

; The number of seconds after which an idle process will be killed.
; Note: Used only when pm is set to 'ondemand'
; Default Value: 10s
;pm.process_idle_timeout = 10s;

; The number of requests each child process should execute before respawning.
; This can be useful to work around memory leaks in 3rd party libraries. For
; endless request processing specify '0'. Equivalent to PHP_FCGI_MAX_REQUESTS.
; Default Value: 0
pm.max_requests = 500

and now, everything is quite good.

we wonder if these configurations can be further improved, if it make sense to set endless request, threads, child, etc.

thank you for your support,

k.