Creating users with occ is slow

ℹ️ Support
1

Hi,

I’m trying to get work Nextcloud 11.0.1 with Shibboleth authentication.

To create the users and set the quota and mailaddress I wrote a script which uses occ like this:

export OC_PASS=dfjksgkfdghksjdfhgsfkjghsdfngb; occ user:add --password-from-env --display-name "max muster" username1
occ user:setting username1 files quota "2 GB"
occ user:setting username1 settings email max.muster@example.com

Once the users are imported authentication whith user_saml works like a charm.

The problem is that we have about 11000 users and the creation of each of them takes between 2 and 10 seconds. The time increases with the number of already existing users. By disabling the transactional filelocking during the improt the speed increases by about 20%. I found out that the bottleneck is the cpu (the script an the database take 40 to 50% each). By logging all database statements I found out that there are thousends on SQL-statements for creating a single user. They all look like this:

SELECT `fileid`, `storage`, `path`, `parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum` FROM `oc_filecache` WHERE `storage` = '2' AND `path_hash` = '1acd52c6827490691204f62ea0ca390c'

Now I wander if there is any better way to create this amount of users or if it is possible to reduce the number of SQL-statements per user creation.

Best regards,
Jan

2

Post this directly to the bugtracker on https://github.com/nextcloud/server/issues

3

Just if anybody else have this problem I will describe our solution.

We were upgrading from an ownCloud 9.1.2 whith LDAP users. We found out that during the migration for each LDAP user a new directory under $DATADIR/appdata_ockvw2z18073/avatar/ had been created. The existence of these directories slows down the user creation process. By simply removeing the LDAP configuration before the migration the directories are not created and the performace increases. Each user is now in about 0,8 sec created.

I’ll make a ticket for this later.