Contacts and autocomplete with large LDAP directories

Hi,

I’m testing Nextcloud in the following environment:

  • Nextcloud version 13.0.2
  • CentOS 7.5
  • Apache 2.4.6
  • PHP 7.1.17
  • MariaDB 5.5.56

System specs are:

  • 2 vCPU Xeon E5-2660 @ 2.60GHz
  • 8GB RAM

I have it configured to authenticate using the SAML app and also connected to an LDAP server (for desktop client access). We have a fairly large user base (around 25.000 users).

The main problem I’m facing is that the sharing autocomplete is unbeareably slow (10-15 seconds).

I also have an issue with the Contacts menu, which shows the message There was an error loading your contacts after showing a loading spinner for some seconds, but logs show it’s caused by a memory exhaustion (happens even after raising memory_limit to 512MB). I’m not worried about this one, as I know I can use a custom theme to disable this menu (although it would be nice being able to completely disable it via configuration).

Back to the autocomplete issue, I’ve narrowed it down to the query Nextcloud makes to the oc_cards. I have only logged in as my test user and I already have >25.000 rows on that table. I guess the LDAP module loads the whole user list on the table. The size of the table is not the actual problem, it’s the vCard contents (and especially the automatically generated photos) what makes the query very slow:

MariaDB [nextcloud]> SELECT `c`.`carddata`, `c`.`uri` FROM `oc_cards` `c` WHERE `c`.`id` IN (SELECT DISTINCT `cp`.`cardid` FROM `oc_cards_properties` `cp` WHERE (`cp`.`addressbookid` = '2') AND ((`cp`.`name` = 'EMAIL') OR (`cp`.`name` = 'FN')) AND (`cp`.`value`  COLLATE utf8mb4_general_ci LIKE '%autocomplete-search-filter-here%'));
[...]
104 rows in set (3.83 sec)

Those 4 seconds add up to the time Nextcloud takes to parse and process the results, which gives us a 10-15 seconds delay on the autocomplete dropdown.

Is anyone else having the same issue? Are there any workarounds to make this faster? Maybe something like disabling automatically generated photos. As you can see below, I’m already using redis and APCu as recommended on the docs.

My config.php file looks like this:

$CONFIG = array (
  'passwordsalt' => 'XXXXX',
  'secret' => 'XXXX',
  'trusted_domains' => 
  array (
    0 => 'www.my.tld',
  ),
  'datadirectory' => '/var/files',
  'overwrite.cli.url' => 'https://www.my.tld/nextcloud',
  'dbtype' => 'mysql',
  'version' => '13.0.2.1',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'XXX',
  'dbpassword' => 'XXX',
  'installed' => true,
  'htaccess.RewriteBase' => '/nextcloud',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => '/var/run/redis/redis.sock',
    'port' => 0,
    'timeout' => 0,
  ),
  'default_language' => 'es',
  'knowledgebaseenabled' => false,
  'allow_user_to_change_display_name' => false,
  'mail_domain' => 'my.tld',
  'mail_smtpmode' => 'sendmail',
  'appcodechecker' => false,
  'updatechecker' => false,
  'appstoreenabled' => false,
  'has_internet_connection' => false,
  'config_is_read_only' => true,
  'upgrade.disable-web' => false,
  'log_type' => 'file',
  'logfile' => '/var/log/nextcloud/nextcloud.log',
  'sharing.minSearchStringLength' => 5,
  'activity_expire_days' => 180,
  'enable_previews' => false,
  'instanceid' => 'oc1yrpeohrlp',
  'mail_from_address' => 'nextcloud',
  'mail_smtpauthtype' => 'LOGIN',
  'ldapIgnoreNamingRules' => false,
  'ldapProviderFactory' => '\\OCA\\User_LDAP\\LDAPProviderFactory',
);

Thanks!

You need to add index to these 2 tables