Webdav access extremely slow, while regular HTTP access very fast

Nextcloud version (eg, 20.0.5): 24.0.3, using the official Docker image
Operating system and version (eg, Ubuntu 20.04): Debian 11
Apache or nginx version (eg, Apache 2.4.25): Traefik 2.8
PHP version (eg, 7.4): 8.0.21

The issue you are facing:

Since one week, the WebDAV endpoint of my Nextcloud instance has been unbearably slow (multiple minutes per query). The strangeness comes from the fact that (1) the host system has a lot of CPU/RAM/Disk usage to spare, and (2) regular non-DAV HTTP queries are very fast, and I can use all services of the instance using the web interface with now slowdowns at all.

Illustrations.

Normal non-DAV HTTP query to access the dashboard:

time curl 'https://$MYINSTANCE/apps/dashboard/#/' -H 'Cookie:$VALIDCOOKIES'

Result:

<some valid HTTP content>

________________________________________________________
Executed in  319.09 millis    fish           external
   usr time   60.87 millis    0.00 micros   60.87 millis
   sys time   14.57 millis  904.00 micros   13.66 millis

DAV query to list files:

time curl -X PROPFIND -H "Depth: 1" -u $USER:$PASSWORD https://$MYINSTANCE/remote.php/dav/files/$USER/

Result:

<some valid XML content>

________________________________________________________
Executed in   57.09 secs      fish           external
   usr time   52.84 millis    1.24 millis   51.60 millis
   sys time   19.84 millis    0.05 millis   19.79 millis

Hence less than one second in one case, and almost 1 minute in the other case.

Note that this is only one query, and it can take up to 700 seconds with a tool such as Joplin to make a complete sync job of a single small folder.


Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it: I do not know, this started happening without making any special change to my instance.

The output of your Nextcloud log in Admin > Logging: There are zero new log entries when I do DAV queries, hence I think this is probably not relevant.

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php                                                                                                                  
$CONFIG = array (                                          
  'trusted_proxies' =>                                     
  array (                                                  
    0 => '192.168.0.0/16',                                 
  ),                                                       
  'filelocking.enabled' => true,                           
  'htaccess.RewriteBase' => '/',                                                                                       
  'memcache.local' => '\\OC\\Memcache\\APCu',                                                                          
  'apps_paths' =>                                          
  array (                                                  
    0 =>                                                   
    array (                                                
      'path' => '/var/www/html/apps',                      
      'url' => '/apps',                                    
      'writable' => false,                                 
    ),                                                     
    1 =>                                                   
    array (                                                
      'path' => '/var/www/html/custom_apps',                                                                           
      'url' => '/custom_apps',                             
      'writable' => true,                                  
    ),                                                     
  ),                                                       
  'instanceid' => '$ID',                          
  'passwordsalt' => '$SALT',                                                                  
  'secret' => '$SECRET',                                                                      
  'trusted_domains' =>                                     
  array (                                                  
    0 => '$MYINSTANCE_OTHER',                            
    1 => '$MYINSTANCE',                               
  ),                                                       
  'datadirectory' => '/var/www/html/data',                 
  'overwrite.cli.url' => 'https://$MYINSTANCE',                                                                   
  'overwritehost' => $MYINSTANCE',                   
  'overwriteprotocol' => 'https',                          
  'dbtype' => 'mysql',                                     
  'version' => '24.0.3.2',                                                                                             
  'dbname' => 'nextcloud',                                 
  'dbhost' => 'nextcloud-database',                        
  'dbport' => '',                                          
  'dbtableprefix' => 'oc_',                   
  'dbuser' => 'nextcloud',
  'dbpassword' => '$DBPASS',
  'installed' => true,
  'maintenance' => false,
  'loglevel' => 2,
  'app_install_overwrite' => 
  array (
    0 => 'calendar',
    1 => 'passwords',
  ),
  'mysql.utf8mb4' => true,
  'theme' => '',
  'default_phone_region' => 'FR',
  'mail_from_address' => 'cloud',
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'mail_domain' => '$MYDOMAIN',
  'mail_smtpauth' => 1,
  'mail_smtphost' => '$SMTP',
  'mail_smtpsecure' => 'tls',
  'mail_smtpname' => '$SMTPNAME',
  'mail_smtppassword' => '$SMTPPASS',
  'mail_smtpport' => '587',
  'mail_smtpauthtype' => 'LOGIN',
  'allow_local_remote_servers' => true,
);

I’ve disabled Redis, hence nothing in the current configuration file about that.

The output of your Apache/nginx/system log in /var/log/____: I’m using Traefik

Output errors in nextcloud.log in /var/www/ or as admin user in top right menu, filtering for errors. Use a pastebin service if necessary. Again, there are zero new log entries when I do DAV queries, hence I think this is probably not relevant.

Update: I’ve installed a second Nextloud instance next to the first one, on the same machine, with the same overall configuration, and the problem does not appear. Hence, something in my real Nextcloud instance makes DAV (and only DAV) extremely slow, and I still have no clue why…

Update: I’ve tried upgrading my MariaDB instance to the latest version (10.8) and disabling all my apps in Nextcloud, but neither attempt changed the situation.

Update: I’ve tried creating a brand new account in the Nextcloud instance, and tried a DAV access. Result: the slowdown does not happen with a brand new account!

This seems to mean that the problem is related to my own nextcloud user account somehow. Maybe due to the very large amount of content (files, events, tasks)

From now on I will try to find out what in my account is causing this slowdown… but if anyone has any idea, I’m interested!

Update: on the slow account, I’ve reduced my amount of files from 5157 to 1817 (and removed them from the trashbin), but there is no observable improvement.

Update: I’ve manually transfered all my data (files, contacts, calendars, …) from my old account to the new account mentionned above, and the problem still does not appear in the new account. At this point, both my old account and this new accound are, from a user perspective, identical (give or take the sharing options of files/calendars that got lost in the import/export actions).

There must be something deep in the database about my old user account that makes DAV unresponsive, but it seems impossible to find what this could be.

At this point, I am tempted to simply delete my old account and migrate everything into a new one, as I just tried, but this is very annoying as I would loose all my sharing options (which are numerous)…

Issue opened here: [Bug]: DAV queries extremely slow with a specific user account · Issue #33453 · nextcloud/server · GitHub