Nextcloud Overload

sorry to hear that,
I hope you’ve looked at /etc/redis/redis.conf?

Look here for guides on how to setup redis:
https://docs.nextcloud.com/server/24/admin_manual/configuration_server/caching_configuration.html#id2

indeed I forgot to modify the redis.conf file

for confirmation, I modify the directives as follows ?

# Accept connections on the specified port, default is 6379 (IANA #815344).
# If port 0 is specified Redis will not listen on a TCP socket.
port 6379

# Unix socket.
#
# Specify the path for the Unix socket that will be used to listen for
# incoming connections. There is no default, so Redis will not listen
# on a unix socket when not specified.
#
#unixsocket /var/run/redis/redis-server.sock
#unixsocketperm 770

Hello and best wishes 2023 to all

On January 4th I made the modifications to use Redis in Nextcloud with a reboot.
Today January 16th, the server is so slow and overloaded that I have to reboot it
Here are the screenshots of Htop

https://zupimages.net/up/23/03/ms4d.jpg
https://zupimages.net/up/23/03/tuz8.jpg

We can see that it is collabora online which is responsible for this RAM overload.

How to solve this problem because it’s really disabling for a server in production.

Thanks for your help

Hello

And here it is, 6 days after the last reboot, the overload problem reappears.
For the moment the only solution I have is to program a reboot of the server every two days.

Now I really need help to find a solution

Thank you in advance.

maybe it’s time to perform systematic troubleshooting. You left many questions about your system unanswered so far. Please start over and fill out the required support template.

From the screenshot it looks you use built-in CODE server - I think not many use such a system with 10+ active users… this is known to cause slow-down:

you should try separate COOL installation (maybe as docker container) and in case the issue happens there as well report the issue at COllaboraOnLine forum/github.

if I didn’t fill in the form it’s because it hasn’t changed from the first post, but I can fill it up again no worries:

Support intro

Nextcloud version (eg, 20.0.5): 24.0.5
Operating system and version (eg, Ubuntu 20.04): Debian 11
Apache or nginx version (eg, Apache 2.4.25): Apache 2.4.54
PHP version (eg, 7.4): 8.1.11

The issue you are facing:

Is this the first time you’ve seen this error? (Y/N): N

Steps to replicate it: Leave the server in production without reboot between 1 and 2 weeks

The output of your Nextcloud log in Admin > Logging:

Nothing special (no error)

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'xxx',
  'passwordsalt' => 'xxx',
  'secret' => 'xxx',
  'trusted_domains' =>
  array (
    0 => 'xxx.xxx.xxx.xxx',
    1 => 'xxx.xxx.com',
    2 => 'xxx.xxx.lan',
  ),
  'datadirectory' => '/DataNextcloud',
  'dbtype' => 'mysql',
  'version' => '24.0.5.1',
  'overwrite.cli.url' => 'https://xxx.xxx.com',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud',
  'dbpassword' => 'xxx',
  'installed' => true,
  'default_phone_region' => 'FR',
  'htaccess.RewriteBase' => '/',
  'ldapProviderFactory' => 'OCA\\User_LDAP\\LDAPProviderFactory',
  'mail_from_address' => 'xxx',
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'mail_domain' => 'xxx.com',
  'mail_smtpport' => '25',
  'mail_smtphost' => 'xxx.xxx.xxx.xxx',
  'lost_password_link' => 'disabled',
  'auth.webauthn.enabled' => false,
  'session_lifetime' => 72000,
  'session_keepalive' => false,
  'auto_logout' => true,
  'filelocking.enabled' => 'true',
  'skeletondirectory' => '',
  'redis' =>
  array (
    'host' => 'localhost',
    'port' => '6379',
    'timeout' => '0.0',
  ),
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'maintenance' => false,
  'logfile' => '/var/log/nextcloud/nextcloud.log',
  'loglevel' => 2,
  'logtimezone' => 'Europe/Paris',
  'log_rotate_size' => 104857600,
  'trashbin_retention_obligation' => 'auto, 60',
  'theme' => '',
  'app_install_overwrite' =>
  array (
    0 => 'integration_whiteboard',
  ),
);

The output of your Apache/nginx/system log in /var/log/____:

Nothing special in log file (no error)

Nothing special in log file (no error)

Output errors in nextcloud.log in /var/www/ or as admin user in top right menu, filtering for errors. Use a pastebin service if necessary.
nothing special in log file (no error)

I’m going to look at docker even if I’ve never used this system and I don’t know yet how to integrate it with nextcloud.

doesn’t help… please review check apache log, increase log level etc… there must be some trace to the faulty activity…

Hello

No problem, I want us to find a solution.

Here is what I could extract from the apache logs (the logs are between 8 and 10 Mb)

in /var/log/apache2/error.log (many lines of this type)

::1 - - [25/Jan/2023:07:46:36 +0100] "OPTIONS * HTTP/1.0" 200 126 "-" "Apache/2.4.54 (Debian) OpenSSL/1.1.1n (internal dummy connection)"

in in /var/log/apache2/access.log (hundreds of lines of this type for exemple)

xxx.domain.lan:443 10.1.0.200 - - [24/Jan/2023:09:59:52 +0100] "POST /apps/richdocumentscode/proxy.php?req=/cool/https%3A%2F%2Fxxx.domain.com%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F127156_ocru8zmmu4cy%3Faccess_token%3DxWQiPOVgzHayI2gMkL7oZ8gY1dMcydp5%26access_token_ttl%3D0/ws?WOPISrc=https%3A%2F%2Fxxx.domain.com%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F127156_ocru8zmmu4cy&compat=/ws/2b86107c0b7126b25ebcd5e49c0f1f31/write/402 HTTP/1.1" 200 567 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:108.0) Gecko/20100101 Firefox/108.0"

and

xxx.domain.lan:443 87.89.7.81 - - [24/Jan/2023:11:23:57 +0100] "POST /apps/richdocumentscode/proxy.php?req=/cool/https%3A%2F%2Fxxx.domain.com%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F2012_ocru8zmmu4cy%3Faccess_token%3DadgFUHTsJxooh68LCiMnW58XjAt7tybI%26access_token_ttl%3D0/ws?WOPISrc=https%3A%2F%2Fxxx.domain.com%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F2012_ocru8zmmu4cy&compat=/ws/6e89d65c1918c949410d0b44170538c2/write/138 HTTP/1.1" 200 567 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0"

in in /var/log/apache2/nextcloud-error.log (many lines of this type for exemple)

[Wed Jan 25 00:00:10.851609 2023] [ssl:warn] [pid 692] AH01909: xxx.domain.lan:443:0 server certificate does NOT include an ID which matches the server name
[Wed Jan 25 08:47:35.337223 2023] [php:notice] [pid 18075] [client 172.16.10.15:60210] richdocumentscode (proxy.php) error exit, PID: 18075, Message: The param should be 'status' or 'req=...', but is: ''
[Wed Jan 25 08:47:36.675712 2023] [php:notice] [pid 18807] [client 172.16.10.15:60276] richdocumentscode (proxy.php) error exit, PID: 18807, Message: The param should be 'status' or 'req=...', but is: ''
[Wed Jan 25 08:47:37.925531 2023] [php:notice] [pid 18807] [client 172.16.10.15:60276] richdocumentscode (proxy.php) error exit, PID: 18807, Message: The param should be 'status' or 'req=...', but is: ''
[Wed Jan 25 08:47:39.167347 2023] [php:notice] [pid 18807] [client 172.16.10.15:60276] richdocumentscode (proxy.php) error exit, PID: 18807, Message: The param should be 'status' or 'req=...', but is: ''
[Wed Jan 25 08:47:57.671284 2023] [access_compat:error] [pid 26585] [client 87.89.7.81:45462] AH01797: client denied by server configuration: /var/www/nextcloud/config
[Wed Jan 25 08:48:25.884724 2023] [php:notice] [pid 18077] [client 172.16.10.15:60274] richdocumentscode (proxy.php) error exit, PID: 18077, Message: No content in reply from coolwsd. Is SSL enabled in error ?

Thank you for your help and your analysis

for me it sounds like richdocuments wants to talk with CODE using local name… this connection fails which might result in additional tries… sounds like major config issue but starting from the point your installation works in general it doesn’t make sense.

please describe more details e.g. how did you install Nextcloud, Apache config, reverse proxy, DNS records, verify DNS resolution (e.g. if you mapped some public DNS to local IPs, is it same for server and clients?) etc…

did you try searching the errors?

seems to result from wrong Apache config. do you access the server on xxx.domain.lan or did you configure xxx.domain.lan as Apache hostname?

Hi

This the tuto i followed for install Nextcloud24 (But the owner has upgraded the page for nextcloud25).

This is my file /etc/apache2/sites-available/nextcloud.conf

<VirtualHost 172.16.0.114:80>
       ServerName xxx.domain.com
       Redirect permanent / https://xxx.domain.com
</VirtualHost>

<VirtualHost 172.16.0.114:443>
        ServerName xxx.domain.lan
        ServerAlias xxx.domain.com
        DocumentRoot "/var/www/nextcloud/"
        <Directory "/var/www/nextcloud/">
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                Allow from all
        </Directory>
        SSLEngine on
        SSLVerifyClient none
        SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0ean-shutdown downgrade-1.0 force-response-1.0
        SSLProtocol -all +TLSv1.2 +TLSv1.3
        SSLCipherSuite SSL ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
        SSLCipherSuite TLSv1.3 TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256
        SSLHonorCipherOrder on
        SSLOpenSSLConfCmd Curves X25519:secp521r1:secp384r1:prime256v1
        SSLCertificateFile /etc/letsencrypt/live/xxx.domain.com/fullchain.pem
        SSLCertificateKeyFile /etc/letsencrypt/live/xxx.domain.com/privkey.pem
        #Include /etc/letsencrypt/options-ssl-apache.conf
        #SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
        ServerSignature Off
        <IfModule mod_dav.c>
                Dav off
        </IfModule>
        <IfModule mod_headers.c>
                Header always set Strict-Transport-Security "max-age=15768000; includeSubDomains; preload"
        </IfModule>
        ErrorLog /var/log/apache2/nextcloud-error.log
        LogLevel warn

</virtualHost>

From the local network, you can connect to the server via the address .com or .lan

My local DNS server is under windows server, the local fqdn xxx.domain.lan is directed to the ip of the nextcloud server (172.16.0.114) and idem for the fqdn xxx.domain.com which is directed to 172.16.0.114.

I have a public IP dedicated to nextcloud which is directly directed to the private IP by my UTM (the server is self-hosted).

My .com domain is managed at OVH and the nextcloud fqdn xxx.domain.com is of course directed to my public IP.

I don’t have a reverseproxy on this IP address since it is dedicated to this Nextcloud server.

Thank you for your help

having both xxx.domain.com and xxx.domain.lan as A records pointing to the same IP (and reverse-lookup possibly pointing to .lan) might confuse the systems when they try to auto-detect it’s hostname. with local DNS in place you can always connect using xxx.domain.com I would recommend you get rid of the .lan hostname everywhere. This definitely simplifies configuration. you can keep the .lan DNS record as CNAME in the DNS just for the case somebody keeps using it.

Give it a try and check the logs if the number of connections decrease after change…

so if I understand correctly what you mean, I have to remove all references to xxx.domain.lan in my apache2 configuration and in my windows server DNS?