The Basics
- Nextcloud Server version (e.g., 29.x.x):
- 31.0.7
- Operating system and version (e.g., Ubuntu 24.04):
- Ubuntu 24.04.2
- Web server and version (e.g, Apache 2.4.25):
- Apache 2.4.65
- Reverse proxy and version _(e.g. nginx 1.27.2)
- Apache 2.4.65
- PHP version (e.g, 8.3):
- 8.3.23
- Is this the first time you’ve seen this error? (Yes / No):
- Yes
- When did this problem seem to first start?
- Months
- Installation method (e.g. AlO, NCP, Bare Metal/Archive, etc.)
- Bare Metal
- Are you using CloudfIare, mod_security, or similar? (Yes / No)
- No
Summary of the issue you are facing:
My infrastructure was on under-powered hardware that has since been migrated to properly scaled hardware. A recurring issue I was experiencing was periodic overnight extreme resource loads; I had assumed this was due to the limited hardware, but it is persisting now that I should have no hardware limit. For example, last night at Midnight, my web server and database went into a period of extreme CPU load which took hours to resolve, and actualy disrupted my monitoring for several hours:
Going back 3 months, the spikes seem to happen roughly every 3-4 days:
I have looked into tasks for the web user, the DB user, inside the sql database, but I cannot find a list of recurring maintenance tasks other than the “cron” cronjob running as the apache user on the nextcloud web server, which handles many more tasks than just this periodic spike. I also think it may be mysql tuning, through I’ve been through a lot of mysql tuning and it seems to not be helping; it’s a separate mysql server well-resourced and on flash storage.
I’m looking for help identifying and mitigating the cyclic task that is causing resource spiking impacting the function of the DB server.
Steps to replicate it (hint: details matter!):
Nothing, this is a cyclically recurring issue
Log entries
Nextcloud
There are some log entries about query timeouts exceeded, but nothing specific about what queries they are.
MySQL
There are some errors in the logs
---TRANSACTION 131813048, ACTIVE (PREPARED) 3 sec
mysql tables in use 1, locked 1
2 lock struct(s), heap size 1128, 1 row lock(s), undo log entries 1
MySQL thread id 34342, OS thread handle 131120923432640, query id 868327 sr66-nxt-01.sub.domain.com xx.xx.xx.207 usr_nextcloud waiting for handler commit
UPDATE `oc_preferences` SET `configvalue` = '1754293298', `lazy` = 0, `type` = 0, `flags` = 0, `indexed` = '' WHERE (`userid` = '6c7cc23e-52c9-103b-8e51-0fc031bcb58e') AND (`appid` = 'login') AND (`configkey` = 'lastLogin')
---TRANSACTION 131813047, ACTIVE (PREPARED) 57 sec
mysql tables in use 1, locked 1
2 lock struct(s), heap size 1128, 1 row lock(s), undo log entries 1
MySQL thread id 34241, OS thread handle 131120794339008, query id 868262 sr66-nxt-01.sub.domain.com xx.xx.xx.207 usr_nextcloud waiting for handler commit
UPDATE `oc_jobs` SET `reserved_at` = '1754293244', `last_checked` = '1754293244' WHERE (`id` = '2195') AND (`reserved_at` = '0') AND (`last_checked` = '1754291151')
---TRANSACTION 131813046, ACTIVE (PREPARED) 57 sec
mysql tables in use 1, locked 1
2 lock struct(s), heap size 1128, 1 row lock(s), undo log entries 1
MySQL thread id 34310, OS thread handle 131119778346688, query id 868258 sr66-nxt-01.sub.domain.com xx.xx.xx.207 usr_nextcloud waiting for handler commit
UPDATE `oc_authtoken` SET `last_check` = 1754293244, `last_activity` = 1754293244 WHERE `id` = 468
---TRANSACTION 131813045, ACTIVE (PREPARED) 57 sec
mysql tables in use 1, locked 1
2 lock struct(s), heap size 1128, 1 row lock(s), undo log entries 1
MySQL thread id 34273, OS thread handle 131120791176896, query id 868243 sr66-nxt-01.sub.domain.com xx.xx.xx.207 usr_nextcloud waiting for handler commit
UPDATE `oc_appconfig` SET `configvalue` = '4876', `lazy` = 0, `type` = 2 WHERE (`appid` = 'backgroundjob') AND (`configkey` = 'lastjob')
---TRANSACTION 131813043, ACTIVE (PREPARED) 57 sec
mysql tables in use 1, locked 1
2 lock struct(s), heap size 1128, 1 row lock(s), undo log entries 1
MySQL thread id 34304, OS thread handle 131120796448448, query id 868239 sr66-nxt-01.sub.domain.com xx.xx.xx.207 usr_nextcloud waiting for handler commit
UPDATE `oc_jobs` SET `last_run` = 1754293244 WHERE `id` = 21
Configuration
Nextcloud
The output of occ config:list system or similar is best, but, if not possible, the contents of your config.php file from /path/to/nextcloud is fine (make sure to remove any identifiable information!):
{
"system": {
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"sub.domain.com",
"domain.com",
"nextcloud.domain.com",
"sr66-nxt-01.sub.domain.com"
],
"trusted_proxies": "***REMOVED SENSITIVE VALUE***",
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "mysql",
"defaultapp": "files",
"version": "31.0.7.1",
"overwritehost": "nextcloud.domain.com",
"overwriteprotocol": "https",
"overwritewebroot": "\/",
"overwrite.cli.url": "https:\/\/nextcloud.domain.com",
"htaccess.RewriteBase": "\/",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"mysql.utf8mb4": true,
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"memcache.local": "\\OC\\Memcache\\APCu",
"memcache.distributed": "\\OC\\Memcache\\Redis",
"maintenance": false,
"theme": "",
"localstorage.umask": 2,
"log_type": "file",
"logfile": "\/var\/log\/nextcloud\/nextcloud.log",
"loglevel": 2,
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_smtpmode": "sendmail",
"mail_sendmailmode": "smtp",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"mail_smtpport": "25",
"default_phone_region": "US",
"ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
"app_install_overwrite": [],
"trashbin_retention_obligation": "auto,15",
"allow_local_remote_servers": true,
"mail_smtpsecure": "ssl",
"filelocking.enabled": "true",
"memcache.locking": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 0,
"dbindex": 0,
"password": "***REMOVED SENSITIVE VALUE***",
"timeout": 1.5
},
"enforce_theme": "",
"updater.release.channel": "stable",
"maintenance_window_start": 9
}
}
Apps
Enabled:
- activity: 4.0.0
- admin_audit: 1.21.0
- app_api: 5.0.2
- bruteforcesettings: 4.0.0
- calendar: 5.3.8
- circles: 31.0.0
- cloud_federation_api: 1.14.0
- comments: 1.21.0
- contacts: 7.2.3
- contactsinteraction: 1.12.0
- dashboard: 7.11.0
- dav: 1.33.0
- federatedfilesharing: 1.21.0
- federation: 1.21.0
- files: 2.3.1
- files_3dmodelviewer: 0.0.15
- files_downloadlimit: 4.0.0
- files_external: 1.23.0
- files_pdfviewer: 4.0.0
- files_reminders: 1.4.0
- files_sharing: 1.23.1
- files_trashbin: 1.21.0
- files_versions: 1.24.0
- firstrunwizard: 4.0.0
- logreader: 4.0.0
- lookup_server_connector: 1.19.0
- mail: 5.1.10
- nextcloud_announcements: 3.0.0
- notes: 4.12.2
- notifications: 4.0.0
- oauth2: 1.19.1
- password_policy: 3.0.0
- photos: 4.0.0
- privacy: 3.0.0
- profile: 1.0.0
- provisioning_api: 1.21.0
- related_resources: 2.0.0
- richdocuments: 8.7.3
- serverinfo: 3.0.0
- settings: 1.14.0
- sharebymail: 1.21.0
- spreed: 21.1.2
- support: 3.0.0
- survey_client: 3.0.0
- suspicious_login: 9.0.1
- systemtags: 1.21.1
- tables: 0.9.5
- tasks: 0.16.1
- text: 5.0.0
- theming: 2.6.1
- theming_customcss: 1.18.0
- twofactor_backupcodes: 1.20.0
- updatenotification: 1.21.0
- user_ldap: 1.22.0
- user_status: 1.11.0
- viewer: 4.0.0
- weather_status: 1.11.0
- webhook_listeners: 1.2.0
- workflowengine: 2.13.0
Disabled:
- encryption: 2.19.0 (installed 2.10.0)
- epubviewer: 1.8.0 (installed 1.8.0)
- files_texteditor: 2.15.1 (installed 2.15.1)
- ldap_write_support: 1.13.0 (installed 1.13.0)
- printer: 0.0.5 (installed 0.0.5)
- recommendations: 4.0.0 (installed 3.0.0)
- twofactor_nextcloud_notification: 5.0.0
- twofactor_totp: 13.0.0-dev.0