Nextcloud version: 15.0.8 ( recently upgraded from 13.0.2)
Operating system and version : Debian 9
Apache or nginx version : Nginx 1.10.3
PHP version : 7.0 via php-fpm
The issue you are facing: 502 gateway timeouts after changing ldap queries to better performing ones. Only way to temporarily recover is to restart nginx + php-fpm and change configuration back
Is this the first time you’ve seen this error? : No, poorly written ldap query in gui would hose server since Owncloud 8?
Background:
Internal enterprise nexcloud server, continously upgraded and running since 2015-2016. Almost no third party apps, besides Calendar and IP range limiter.
After performing manual chain update from nextcloud 13.0.2 (?) via 13.0.8 -> 14.0.0 -> 14.0.8 -> 15.0.0 -> 15.0.8, I had noticed that server was very sluggish, due to php/fpm workers eating all cpu and memory resources. 512 MB limit per worker was exhausted rather quickly, and when it was relaxed, php would exhaust all availble resources.
It seems to be caused bz slight misconfiguration of LDAP addon, base dn of ldap user was too broadly defined. Once corrected, resource usage returned to 13.0.X levels.
We still experienced somewhat slow login times and login denials that should not happen, so we started looking at our ldap queries. We are authenticating via ldaps against global catalogue of AD server. We are recursivelz searching through members of one group, whose members are groups at different trees in our AD forest.
Working user ldap query looks like this (and return 590 potential users):
(&(|(objectclass=person)(objectclass=user))(|(|(memberof:1.2.840.113556.1.4.1941:=CN=Nextcloud users,OU=nextcloud,OU=DVZ,DC=servis,DC=employer,DC=cz)(primaryGroupID=11238))))
After benchmarking its performance via powershell showed us execution time between 25 to 60 seconds, further refinement seemed neccessary.
This ldap query is functionally equivalent, plus it filters out disabled user accounts:
(&(sAMAccountType=805306368)(!userAccountControl:1.2.840.113556.1.4.803:=2)(memberof:1.2.840.113556.1.4.1941:=OU=nextcloud,OU=DVZ,DC=servis,DC=employer,DC=cz))
It works in powershell and succefully executes in 6-10 seconds. Attempting to use it however fails, when attempting to get user count nextcloud just throws 502 error after 50 to 60 seconds waiting for something. Same thing happens when users attempt to log in via ldap backed credentials.
So question is did I miss something critical in our LDAP config? Are there any undocumented caveats with ldap queries in nexcloud?
Supporting data:
The output of your Nextcloud log in Admin > Logging:
Extreme logspamming from some misconfigured or old clients
{"reqId":"pdetGkfxhBg0w6jEz0cf","level":3,"time":"2019-06-05T04:28:23+00:00","remoteAddr":"194.213.41.2","user":"Sample_User","app":"PHP","method":"PROPFIND","url":"\/remote.php\/webdav\/","message":"ldap_control_paged_result_response(): Result is: No such object (32) at \/var\/www\/nextcloud\/apps\/user_ldap\/lib\/LDAP.php#74","userAgent":"Mozilla\/5.0 (Windows) mirall\/2.3.3 (build 1) (Nextcloud)","version":"15.0.8.1"}
More than few hundred thousand times a day
The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):
<?php
$CONFIG = array (
'instanceid' => 'nope',
'passwordsalt' => 'nope',
'secret' => 'nope',
'trusted_domains' =>
array (
0 => 'localhost',
1 => 'external zone',
2 => 'internal zone',
3 => 'instance fqdn',
),
'datadirectory' => '/var/www/nextcloud/data',
'overwrite.cli.url' => 'instance fqdn',
'dbtype' => 'mysql',
'version' => '15.0.8.1',
'dbname' => 'nextcloud',
'dbhost' => 'localhost',
'dbport' => 'localhost:/var/run/mysqld/mysqld.sock',
'dbtableprefix' => 'oc_',
'dbuser' => 'nextcloud',
'dbpassword' => 'nope',
'installed' => true,
'ldapIgnoreNamingRules' => false,
'mail_smtpmode' => 'smtp',
'mail_smtpauthtype' => 'PLAIN',
'mail_from_address' => 'nope',
'mail_domain' => 'domain name',
'mail_smtphost' => 'exchange ip',
'mail_smtpport' => '25',
'auth.bruteforce.protection.enabled' => false,
'maintenance' => false,
'proxy' => 'proxy.adress.cz:3128',
'default_language' => 'cs_CZ',
'loglevel' => 2,
'logfile' => '/var/log/nextcloud.log',
'mysql.utf8mb4' => true,
);
The output of your Apache/nginx/system log in /var/log/____:
just .ocdata denials as usual