LDAP user attributes for sharing autocompletion not fetched in background

I’m trying to fix this problem and I am thinking this is a bug with the Nextcloud LDAP-sync implementation. I tried hard to track it down but if I am missing something I would really appreciate any help :slight_smile:

When performing a user search in the sharing dialogue of NC, autocompletion performs a full LDAP sync/lookup although ldap time to live is set to 86400s.

This is very problematic in our environment because LDAP consists of ~ 800 members and full sync takes about 4 minutes. That’s way to long and should reliably be performed in background.

APCu is installed and preforming well (local). Redis is also up and performing well (distributed & locking).

Steps to reproduce

  1. Wait one day.
  2. Try to share something via share-dialogue.
  3. Only very few frequently used users for sharing are listet.
  4. Other users are listet after 4 minutes (= complete ldap sync which can be observed on the ldap server).

Expected behaviour

User attributes of all users should be fetched at least twice a day in background as said in the Nextcloud manual:
"The attributes of users are fetched on demand (i.e. for sharing autocompletion or in the user management) and then stored inside the Nextcloud database to allow a better performance on our side. They are typically checked twice a day in batches from all users again. "

Actual behaviour

A complete sync is triggered although attributes should be in cache.

Server configuration

Operating system: Ubuntu 20.04

Web server: Nginx 1.21.3

Database: MariaDB 15.1

PHP version: 8.0

Nextcloud version: 21.0.5

List of activated apps:

App list
  - accessibility: 1.7.0
  - activity: 2.14.3
  - announcementcenter: 5.0.1
  - apporder: 0.13.0
  - bbb: 2.0.0
  - bruteforcesettings: 2.2.0
  - circles: 0.21.4
  - cloud_federation_api: 1.4.0
  - comments: 1.11.0
  - dav: 1.17.1
  - external: 3.8.2
  - extract: 1.3.2
  - federatedfilesharing: 1.11.0
  - federation: 1.11.0
  - files: 1.16.0
  - files_accesscontrol: 1.11.1
  - files_antivirus: 3.2.2
  - files_external: 1.12.0
  - files_pdfviewer: 2.1.0
  - files_rightclick: 1.0.0
  - files_sharing: 1.13.1
  - files_texteditor: 2.14.0
  - files_trashbin: 1.11.0
  - files_videoplayer: 1.10.0
  - groupfolders: 9.0.3
  - logreader: 2.6.0
  - lookup_server_connector: 1.9.0
  - nextcloud_announcements: 1.10.0
  - notifications: 2.9.0
  - oauth2: 1.9.0
  - onlyoffice: 7.1.2
  - password_policy: 1.11.0
  - previewgenerator: 3.1.1
  - privacy: 1.5.0
  - provisioning_api: 1.11.0
  - richdocuments: 4.2.3
  - serverinfo: 1.11.0
  - settings: 1.3.0
  - systemtags: 1.11.0
  - theming: 1.12.0
  - twofactor_backupcodes: 1.10.0
  - updatenotification: 1.11.0
  - user_ldap: 1.11.0
  - viewer: 1.5.0
  - workflowengine: 2.3.1

Nextcloud configuration:

Config report
{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***",
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "log_type": "file",
        "logfile": "\/var\/www\/nextcloud\/data\/nextcloud.log",
        "loglevel": 3,
        "logdateformat": "d. F Y H:i:s",
        "dbtype": "mysql",
        "version": "21.0.5.1",
        "overwritehost": "nextcloud.gymnasium-ettenheim.de",
        "overwrite.cli.url": "https:\/\/nextcloud.gymnasium-ettenheim.de",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "activity_expire_days": 14,
        "auth.bruteforce.protection.enabled": true,
        "blacklisted_files": [
            ".htaccess",
            "Thumbs.db",
            "thumbs.db"
        ],
  "cron_log": true,
        "default_phone_region": "DE",
        "default_locale": "de_DE",
        "default_language": "de",
        "enable_previews": true,
        "enabledPreviewProviders": [
            "OC\\Preview\\PNG",
            "OC\\Preview\\JPEG",
            "OC\\Preview\\GIF",
            "OC\\Preview\\BMP",
            "OC\\Preview\\XBitmap",
            "OC\\Preview\\Movie",
            "OC\\Preview\\PDF",
            "OC\\Preview\\MP3",
            "OC\\Preview\\TXT",
            "OC\\Preview\\MarkDown",
            "OC\\Preview\\Image",
            "OC\\Preview\\TIFF",
            "OC\\Preview\\Font",
            "OC\\Preview\\MKV",
            "OC\\Preview\\SVG",
            "OC\\Preview\\AVI",
            "OC\\Preview\\OpenDocument",
            "OC\\Preview\\MSOfficeDoc",
            "OC\\Preview\\MSOffice2003",
            "OC\\Preview\\MSOffice2007"
        ],
        "filesystem_check_changes": 0,
        "filelocking.enabled": "true",
        "htaccess.RewriteBase": "\/",
        "integrity.check.disabled": false,
        "knowledgebaseenabled": false,
        "log_rotate_size": 104857600,
        "logtimezone": "Europe\/Berlin",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "preview_max_x": 1024,
        "preview_max_y": 768,
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "timeout": 1.5,
            "password": "***REMOVED SENSITIVE VALUE***"
        },
        "quota_include_external_storage": false,
        "skeletondirectory": "\/var\/www\/nextcloud\/core\/skeletonEttenheim",
        "share_folder": "\/mir freigegeben",
        "lost_password_link": "disabled",
        "trashbin_retention_obligation": "auto, 7",
        "versions_retention_obligation": "auto, 14",
        "ldapIgnoreNamingRules": false,
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "ldapUserCleanupInterval": 86400,
        "maintenance": false,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "ssl",
        "forwarded_for_headers": [
            "HTTP_X_FORWARDED"
        ],
        "app_install_overwrite": [
            "files_clipboard"
        ],
        "simpleSignUpLink.shown": false,
        "theme": "",
        "allow_local_remote_servers": true
    }
}

External storage: local/smb

Are you using encryption: no

Are you using an external user-backend, if yes which one: LDAP

LDAP configuration (delete this part if not used)

LDAP config
   "user_ldap": {
            "background_sync_interval": "43200",
            "background_sync_offset": "700",
            "background_sync_prefix": "s01",
            "cleanUpJobOffset": "450",
            "enabled": "yes",
            "installed_version": "1.11.0",
            "s01_lastChange": "1635413244",
            "s01has_memberof_filter_support": "0",
            "s01home_folder_naming_rule": "",
            "s01last_jpegPhoto_lookup": "0",
            "s01ldap_agent_password": "***REMOVED SENSITIVE VALUE***",
            "s01ldap_attributes_for_group_search": "cn",
            "s01ldap_attributes_for_user_search": "uid\ngivenName\nsn",
            "s01ldap_backup_host": "",
            "s01ldap_backup_port": "",
            "s01ldap_base": "dc=paedml-linux,dc=lokal",
            "s01ldap_base_groups": "cn=groups,ou=schule,dc=paedml-linux,dc=lokal",
            "s01ldap_base_users": "dc=paedml-linux,dc=lokal",
            "s01ldap_cache_ttl": "86400",
            "s01ldap_configuration_active": "1",
            "s01ldap_default_ppolicy_dn": "",
            "s01ldap_display_name": "displayname",
            "s01ldap_dn": "uid=ldapsuche,cn=users,dc=paedml-linux,dc=lokal",
            "s01ldap_dynamic_group_member_url": "",
            "s01ldap_email_attr": "mailPrimaryAddress",
            "s01ldap_experienced_admin": "1",
            "s01ldap_expert_username_attr": "uidNumber",
            "s01ldap_expert_uuid_group_attr": "",
            "s01ldap_expert_uuid_user_attr": "",
   "s01ldap_ext_storage_home_attribute": "uid",
            "s01ldap_gid_number": "gidNumber",
            "s01ldap_group_display_name": "cn",
            "s01ldap_group_filter": "(objectclass=univentionGroup)",
            "s01ldap_group_filter_mode": "0",
            "s01ldap_group_member_assoc_attribute": "memberUid",
            "s01ldap_groupfilter_groups": "",
            "s01ldap_groupfilter_objectclass": "",
            "s01ldap_host": "server.paedml-linux.lokal",
            "s01ldap_login_filter": "(&(objectclass=person)(uid=%uid))",
            "s01ldap_login_filter_mode": "0",
            "s01ldap_loginfilter_attributes": "",
            "s01ldap_loginfilter_email": "0",
            "s01ldap_loginfilter_username": "1",
            "s01ldap_matching_rule_in_chain_state": "unknown",
            "s01ldap_nested_groups": "0",
            "s01ldap_override_main_server": "",
            "s01ldap_paging_size": "700",
            "s01ldap_port": "7389",
            "s01ldap_quota_attr": "",
            "s01ldap_quota_def": "0",
            "s01ldap_tls": "0",
            "s01ldap_turn_off_cert_check": "0",
            "s01ldap_turn_on_pwd_change": "0",
            "s01ldap_user_avatar_rule": "default",
            "s01ldap_user_display_name_2": "",
            "s01ldap_user_filter_mode": "0",
            "s01ldap_userfilter_groups": "",
            "s01ldap_userfilter_objectclass": "",
            "s01ldap_userlist_filter": "(|(objectclass=ucsschoolTeacher)(objectclass=ucsschoolStudent)(uid=Administrator))",
            "s01use_memberof_to_detect_membership": "1",
            "types": "authentication",
            "updateAttributesInterval": "10800"
        },
+-------------------------------+------------------------------------------------------------------------------------+
| Configuration                 | s01                                                                                |
+-------------------------------+------------------------------------------------------------------------------------+
| hasMemberOfFilterSupport      | 0                                                                                  |
| homeFolderNamingRule          |                                                                                    |
| lastJpegPhotoLookup           | 0                                                                                  |
| ldapAgentName                 | ***REMOVED SENSITIVE VALUE***                                   |
| ldapAgentPassword             | ***                                                                                |
| ldapAttributesForGroupSearch  | cn                                                                                 |
| ldapAttributesForUserSearch   | uid;givenName;sn                                                                   |
| ldapBackupHost                |                                                                                    |
| ldapBackupPort                |                                                                                    |
| ldapBase                      | dc=paedml-linux,dc=lokal                                                           |
| ldapBaseGroups                | cn=groups,ou=schule,dc=paedml-linux,dc=lokal                                       |
| ldapBaseUsers                 | dc=paedml-linux,dc=lokal                                                           |
| ldapCacheTTL                  | 86400                                                                              |
| ldapConfigurationActive       | 1                                                                                  |
| ldapDefaultPPolicyDN          |                                                                                    |
| ldapDynamicGroupMemberURL     |                                                                                    |
| ldapEmailAttribute            | mailPrimaryAddress                                                                 |
| ldapExperiencedAdmin          | 1                                                                                  |
| ldapExpertUUIDGroupAttr       |                                                                                    |
| ldapExpertUUIDUserAttr        |                                                                                    |
| ldapExpertUsernameAttr        | uidNumber                                                                          |
| ldapExtStorageHomeAttribute   | uid                                                                                |
| ldapGidNumber                 | gidNumber                                                                          |
| ldapGroupDisplayName          | cn                                                                                 |
| ldapGroupFilter               | (objectclass=univentionGroup)                                                      |
| ldapGroupFilterGroups         |                                                                                    |
| ldapGroupFilterMode           | 0                                                                                  |
| ldapGroupFilterObjectclass    |                                                                                    |
| ldapGroupMemberAssocAttr      | memberUid                                                                          |
| ldapHost                      | server.paedml-linux.lokal                                                          |
| ldapIgnoreNamingRules         |                                                                                    |
| ldapLoginFilter               | (&(objectclass=person)(uid=%uid))                                                  |
| ldapLoginFilterAttributes     |                                                                                    |
| ldapLoginFilterEmail          | 0                                                                                  |
| ldapLoginFilterMode           | 0                                                                                  |
| ldapLoginFilterUsername       | 1                                                                                  |
| ldapMatchingRuleInChainState  | unknown                                                                            |
| ldapNestedGroups              | 0                                                                                  |
| ldapOverrideMainServer        |                                                                                    |
| ldapPagingSize                | 700                                                                                |
| ldapPort                      | 7389                                                                               |
| ldapQuotaAttribute            |                                                                                    |
| ldapQuotaDefault              | 0                                                                                  |
| ldapTLS                       | 0                                                                                  |
| ldapUserAvatarRule            | default                                                                            |
| ldapUserDisplayName           | displayname                                                                        |
| ldapUserDisplayName2          |                                                                                    |
| ldapUserFilter                | (|(objectclass=ucsschoolTeacher)(objectclass=ucsschoolStudent)(uid=Administrator)) |
| ldapUserFilterGroups          |                                                                                    |
| ldapUserFilterMode            | 0                                                                                  |
| ldapUserFilterObjectclass     |                                                                                    |
| ldapUuidGroupAttribute        | auto                                                                               |
| ldapUuidUserAttribute         | auto                                                                               |
| turnOffCertCheck              | 0                                                                                  |
| turnOnPasswordChange          | 0                                                                                  |
| useMemberOfToDetectMembership | 1                                                                                  |
+-------------------------------+------------------------------------------------------------------------------------+

Client configuration

Browser: various

Operating system: various

Logs

Nextcloud doesn’t throw any LDAP-related errors or warnings.
APCu is up and running. I did check and observe it’s work via the APCu php test interface.
Redis is also up and running.