Cron jobs getting stuck after upgrade to Nextcloud 33

ℹ️ Support
, , ,
1

Dear,

I upgraded our instance from OwnCloud 10 to Nextcloud 25, then performed all successive version jumps up to Nextcloud 33.0.1.

Since the upgrade to version 33, I have a recurring problem:
When I run the cron with the command: occ background-job:worker

The process gets stuck. In the MariaDB database, I can see several transactions that are blocked, and one large transaction that has been in ROLLING BACK state for many hours (currently more than 1.3 million undo log entries).

Even when the rollback transactions eventually finish, the problem reappears as soon as I relaunch the cron jobs. Additionally, there are currently around 6000 jobs waiting in the cron queue.

This is causing high database load and frequent “Lock wait timeout exceeded” errors.
Thank you for your help.

Best regards,

The Basics

  • Nextcloud Server version (e.g., 29.x.x):
    • 33.0.1
  • Operating system and version (e.g., Ubuntu 24.04):
    • Docker on RL 10
  • Web server and version (e.g, Apache 2.4.25):
    • nginx
  • Reverse proxy and version _(e.g. nginx 1.27.2)
    • nginx
  • PHP version (e.g, 8.3):
    • docker nextcloud-fpm
  • Installation method (e.g. AlO, NCP, Bare Metal/Archive, etc.)
    • docker
  • Are you using CloudfIare, mod_security, or similar? (Yes / No)
    • no

Summary of the issue you are facing:

BD rollback

Steps to replicate it (hint: details matter!):

  1. run cron job

Log entries

BD process list + transaction

SHOW ENGINE INNODB STATUS;

=====================================
2026-04-02 09:17:59 0x7f49440f2640 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 40 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 0 srv_active, 0 srv_shutdown, 73104 srv_idle
srv_master_thread log flush and writes: 73103
----------
SEMAPHORES
----------
------------
TRANSACTIONS
------------
Trx id counter 2415236
Purge done for trx's n:o < 2415234 undo n:o < 0 state: running
History list length 47547
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 2415107, ACTIVE 102 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1128, 1 row lock(s)
MariaDB thread id 781118, OS thread handle 139934838785600, query id 36698825 10.10.0.5 nextcloud Updating
UPDATE `oc_filecache` SET `size` = '28357504013' WHERE (`fileid` = 84204249) AND (`storage` = 3239) AND ((`size` <> '28357504013') OR (`size` IS NULL))
------- TRX HAS BEEN WAITING 102337446 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 316262 n bits 320 index PRIMARY of table `nextcloud`.`oc_filecache` trx id 2415107 lock_mode X locks rec but not gap waiting
Record lock, heap no 31 PHYSICAL RECORD: n_fields 18; compact format; info bits 0
 0: len 8; hex 800000000504dad9; asc         ;;
 1: len 6; hex 00000024d44e; asc    $ N;;
 2: len 7; hex 1c000029480677; asc    )H w;;
 3: len 8; hex 8000000000000ca7; asc         ;;
 4: len 5; hex 66696c6573; asc files;;
 5: len 30; hex 343562393633333937616134306434613030363365306438356534666537; asc 45b963397aa40d4a0063e0d85e4fe7; (total 32 bytes);
 6: len 8; hex 800000000504dad7; asc         ;;
 7: len 5; hex 66696c6573; asc files;;
 8: len 8; hex 8000000000000002; asc         ;;
 9: len 8; hex 8000000000000001; asc         ;;
 10: len 8; hex 800000069a387f0b; asc      8  ;;
 11: len 8; hex 8000000069cd367c; asc     i 6|;;
 12: len 8; hex 8000000069848480; asc     i   ;;
 13: len 4; hex 80000000; asc     ;;
 14: len 8; hex 8000000000000000; asc         ;;
 15: len 13; hex 36396364333637633237623837; asc 69cd367c27b87;;
 16: len 4; hex 8000001f; asc     ;;
 17: len 0; hex ; asc ;;

------------------
---TRANSACTION 2414991, ACTIVE 217 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1128, 1 row lock(s)
MariaDB thread id 780874, OS thread handle 139952661825088, query id 36695764 10.10.0.5 nextcloud Sending data
SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 3825) AND (`path_hash` IN ('0bb427854797c407afb702a4b8565a01', '45b963397aa40d4a0063e0d85e4fe7a1', '8a4c1241a9d56d890fb16a8bb804f20b', 'a01bfa157f930f9df32f4537b9ad2548', 'b5e789dd388803da0b8cefc5f81d3273', 'd41d8cd98f00b204e9800998ecf8427e')) ORDER BY `path_hash` ASC FOR UPDATE
------- TRX HAS BEEN WAITING 216884807 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 632378 n bits 320 index fs_storage_path_hash of table `nextcloud`.`oc_filecache` trx id 2414991 lock_mode X locks rec but not gap waiting
Record lock, heap no 172 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
 0: len 8; hex 8000000000000ef1; asc         ;;
 1: len 30; hex 306262343237383534373937633430376166623730326134623835363561; asc 0bb427854797c407afb702a4b8565a; (total 32 bytes);
 2: len 8; hex 8000000005c30446; asc        F;;

------------------
---TRANSACTION 2414964, ACTIVE 231 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1128, 1 row lock(s)
MariaDB thread id 780819, OS thread handle 139935269455424, query id 36695317 10.10.0.5 nextcloud Updating
UPDATE `oc_jobs` SET `reserved_at` = '0', `last_checked` = 1775121248, `last_run` = 0 WHERE (`class` = 'OC\\Command\\CommandJob') AND (`argument_hash` = '4123cc11a7f2491c86af956c3830acb5bbb944e44f3282e28071b4529f65ed61')
------- TRX HAS BEEN WAITING 230769425 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 310 page no 228 n bits 320 index job_argument_hash of table `nextcloud`.`oc_jobs` trx id 2414964 lock_mode X locks rec but not gap waiting
Record lock, heap no 90 PHYSICAL RECORD: n_fields 3; compact format; info bits 32
 0: len 21; hex 4f435c436f6d6d616e645c436f6d6d616e644a6f62; asc OC\Command\CommandJob;;
 1: len 30; hex 343132336363313161376632343931633836616639353663333833306163; asc 4123cc11a7f2491c86af956c3830ac; (total 64 bytes);
 2: len 8; hex 00f0b2566d9b6001; asc    Vm ` ;;

------------------
---TRANSACTION 2414962, ACTIVE 231 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1128, 1 row lock(s)
MariaDB thread id 780818, OS thread handle 139952661517888, query id 36695293 10.10.0.5 nextcloud Sending data
SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 3825) AND (`path_hash` IN ('0bb427854797c407afb702a4b8565a01', '45b963397aa40d4a0063e0d85e4fe7a1', '8a4c1241a9d56d890fb16a8bb804f20b', 'a01bfa157f930f9df32f4537b9ad2548', 'b5e789dd388803da0b8cefc5f81d3273', 'd41d8cd98f00b204e9800998ecf8427e')) ORDER BY `path_hash` ASC FOR UPDATE
------- TRX HAS BEEN WAITING 231329066 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 632378 n bits 320 index fs_storage_path_hash of table `nextcloud`.`oc_filecache` trx id 2414962 lock_mode X locks rec but not gap waiting
Record lock, heap no 172 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
 0: len 8; hex 8000000000000ef1; asc         ;;
 1: len 30; hex 306262343237383534373937633430376166623730326134623835363561; asc 0bb427854797c407afb702a4b8565a; (total 32 bytes);
 2: len 8; hex 8000000005c30446; asc        F;;

------------------
---TRANSACTION 2414930, ACTIVE 256 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 7 lock struct(s), heap size 1128, 9 row lock(s)
MariaDB thread id 780781, OS thread handle 139935270991424, query id 36694775 10.10.0.5 nextcloud Sending data
SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 3825) AND (`path_hash` IN ('0bb427854797c407afb702a4b8565a01', '45b963397aa40d4a0063e0d85e4fe7a1', 'a01bfa157f930f9df32f4537b9ad2548', 'b5e789dd388803da0b8cefc5f81d3273', 'd41d8cd98f00b204e9800998ecf8427e')) ORDER BY `path_hash` ASC FOR UPDATE
------- TRX HAS BEEN WAITING 233572158 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 632379 n bits 320 index fs_storage_path_hash of table `nextcloud`.`oc_filecache` trx id 2414930 lock_mode X locks rec but not gap waiting
Record lock, heap no 210 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
 0: len 8; hex 8000000000000ef1; asc         ;;
 1: len 30; hex 643431643863643938663030623230346539383030393938656366383432; asc d41d8cd98f00b204e9800998ecf842; (total 32 bytes);
 2: len 8; hex 8000000005c3025d; asc        ];;

------------------
---TRANSACTION 2414782, ACTIVE 374 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1128, 1 row lock(s)
MariaDB thread id 780559, OS thread handle 139934811665984, query id 36691423 10.10.0.5 nextcloud Updating
UPDATE `oc_jobs` SET `reserved_at` = '0', `last_checked` = 1775121105, `last_run` = 0 WHERE (`class` = 'OC\\Command\\CommandJob') AND (`argument_hash` = 'd3702d8eaee4eeea97573576dff05bba292ba6be59908dbaf59b84e7fea78a99')
------- TRX HAS BEEN WAITING 374063650 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 310 page no 340 n bits 320 index job_argument_hash of table `nextcloud`.`oc_jobs` trx id 2414782 lock_mode X locks rec but not gap waiting
Record lock, heap no 83 PHYSICAL RECORD: n_fields 3; compact format; info bits 32
 0: len 21; hex 4f435c436f6d6d616e645c436f6d6d616e644a6f62; asc OC\Command\CommandJob;;
 1: len 30; hex 643337303264386561656534656565613937353733353736646666303562; asc d3702d8eaee4eeea97573576dff05b; (total 64 bytes);
 2: len 8; hex 00f0c6afae5b6001; asc      [` ;;

------------------
---TRANSACTION 2414751, ACTIVE 402 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1128, 1 row lock(s)
MariaDB thread id 780508, OS thread handle 139952668583488, query id 36690861 10.10.0.5 nextcloud Updating
UPDATE `oc_filecache` SET `size` = '28357504013' WHERE (`fileid` = 84204249) AND (`storage` = 3239) AND ((`size` <> '28357504013') OR (`size` IS NULL))
------- TRX HAS BEEN WAITING 402149152 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 316262 n bits 320 index PRIMARY of table `nextcloud`.`oc_filecache` trx id 2414751 lock_mode X locks rec but not gap waiting
Record lock, heap no 31 PHYSICAL RECORD: n_fields 18; compact format; info bits 0
 0: len 8; hex 800000000504dad9; asc         ;;
 1: len 6; hex 00000024d44e; asc    $ N;;
 2: len 7; hex 1c000029480677; asc    )H w;;
 3: len 8; hex 8000000000000ca7; asc         ;;
 4: len 5; hex 66696c6573; asc files;;
 5: len 30; hex 343562393633333937616134306434613030363365306438356534666537; asc 45b963397aa40d4a0063e0d85e4fe7; (total 32 bytes);
 6: len 8; hex 800000000504dad7; asc         ;;
 7: len 5; hex 66696c6573; asc files;;
 8: len 8; hex 8000000000000002; asc         ;;
 9: len 8; hex 8000000000000001; asc         ;;
 10: len 8; hex 800000069a387f0b; asc      8  ;;
 11: len 8; hex 8000000069cd367c; asc     i 6|;;
 12: len 8; hex 8000000069848480; asc     i   ;;
 13: len 4; hex 80000000; asc     ;;
 14: len 8; hex 8000000000000000; asc         ;;
 15: len 13; hex 36396364333637633237623837; asc 69cd367c27b87;;
 16: len 4; hex 8000001f; asc     ;;
 17: len 0; hex ; asc ;;

------------------
---TRANSACTION 2414739, ACTIVE 407 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1128, 1 row lock(s)
MariaDB thread id 780506, OS thread handle 139934810129984, query id 36690668 10.10.0.5 nextcloud Updating
UPDATE `oc_filecache` SET `size` = '155428934' WHERE (`fileid` = 29854537) AND (`storage` = 424) AND ((`size` <> '155428934') OR (`size` IS NULL))
------- TRX HAS BEEN WAITING 406819294 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 18385 n bits 320 index PRIMARY of table `nextcloud`.`oc_filecache` trx id 2414739 lock_mode X locks rec but not gap waiting
Record lock, heap no 6 PHYSICAL RECORD: n_fields 18; compact format; info bits 0
 0: len 8; hex 8000000001c78b49; asc        I;;
 1: len 6; hex 000000234ee5; asc    #N ;;
 2: len 7; hex 00000054d43bc8; asc    T ; ;;
 3: len 8; hex 80000000000001a8; asc         ;;
 4: len 14; hex 66696c65735f76657273696f6e73; asc files_versions;;
 5: len 30; hex 393639326161653530303232663435663130393836343639333962323837; asc 9692aae50022f45f1098646939b287; (total 32 bytes);
 6: len 8; hex 80000000009e0685; asc         ;;
 7: len 14; hex 66696c65735f76657273696f6e73; asc files_versions;;
 8: len 8; hex 8000000000000002; asc         ;;
 9: len 8; hex 8000000000000001; asc         ;;
 10: len 8; hex 8000000009434050; asc      C@P;;
 11: len 8; hex 8000000069cd6892; asc     i h ;;
 12: len 8; hex 8000000069cd0be4; asc     i   ;;
 13: len 4; hex 80000000; asc     ;;
 14: len 8; hex 8000000000000000; asc         ;;
 15: len 13; hex 36396364363839326235613461; asc 69cd6892b5a4a;;
 16: len 4; hex 8000001f; asc     ;;
 17: SQL NULL;

------------------
---TRANSACTION 2414686, ACTIVE 450 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 4 lock struct(s), heap size 1128, 3 row lock(s)
MariaDB thread id 780429, OS thread handle 139934839707200, query id 36689641 10.10.0.5 nextcloud Sending data
SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 424) AND (`path_hash` IN ('45b963397aa40d4a0063e0d85e4fe7a1', 'd41d8cd98f00b204e9800998ecf8427e')) ORDER BY `path_hash` ASC FOR UPDATE
------- TRX HAS BEEN WAITING 450473667 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 603698 n bits 320 index fs_storage_path_hash of table `nextcloud`.`oc_filecache` trx id 2414686 lock_mode X locks rec but not gap waiting
Record lock, heap no 150 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
 0: len 8; hex 80000000000001a8; asc         ;;
 1: len 30; hex 643431643863643938663030623230346539383030393938656366383432; asc d41d8cd98f00b204e9800998ecf842; (total 32 bytes);
 2: len 8; hex 80000000009e0685; asc         ;;

------------------
---TRANSACTION 2414716, ACTIVE 425 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1128, 1 row lock(s)
MariaDB thread id 779907, OS thread handle 139934843086400, query id 36690108 10.10.0.5 nextcloud Updating
UPDATE `oc_filecache` SET `size` = '28357504013' WHERE (`fileid` = 84204249) AND (`storage` = 3239) AND ((`size` <> '28357504013') OR (`size` IS NULL))
------- TRX HAS BEEN WAITING 425105032 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 316262 n bits 320 index PRIMARY of table `nextcloud`.`oc_filecache` trx id 2414716 lock_mode X locks rec but not gap waiting
Record lock, heap no 31 PHYSICAL RECORD: n_fields 18; compact format; info bits 0
 0: len 8; hex 800000000504dad9; asc         ;;
 1: len 6; hex 00000024d44e; asc    $ N;;
 2: len 7; hex 1c000029480677; asc    )H w;;
 3: len 8; hex 8000000000000ca7; asc         ;;
 4: len 5; hex 66696c6573; asc files;;
 5: len 30; hex 343562393633333937616134306434613030363365306438356534666537; asc 45b963397aa40d4a0063e0d85e4fe7; (total 32 bytes);
 6: len 8; hex 800000000504dad7; asc         ;;
 7: len 5; hex 66696c6573; asc files;;
 8: len 8; hex 8000000000000002; asc         ;;
 9: len 8; hex 8000000000000001; asc         ;;
 10: len 8; hex 800000069a387f0b; asc      8  ;;
 11: len 8; hex 8000000069cd367c; asc     i 6|;;
 12: len 8; hex 8000000069848480; asc     i   ;;
 13: len 4; hex 80000000; asc     ;;
 14: len 8; hex 8000000000000000; asc         ;;
 15: len 13; hex 36396364333637633237623837; asc 69cd367c27b87;;
 16: len 4; hex 8000001f; asc     ;;
 17: len 0; hex ; asc ;;

------------------
---TRANSACTION 2414955, ACTIVE 233 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1128, 1 row lock(s)
MariaDB thread id 779632, OS thread handle 139952655889984, query id 36695259 10.10.0.5 nextcloud Sending data
SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 3825) AND (`path_hash` IN ('45b963397aa40d4a0063e0d85e4fe7a1', '907dc726212f1d757dbb06bc6e5e48fd', 'a01bfa157f930f9df32f4537b9ad2548', 'b5e789dd388803da0b8cefc5f81d3273', 'd41d8cd98f00b204e9800998ecf8427e')) ORDER BY `path_hash` ASC FOR UPDATE
------- TRX HAS BEEN WAITING 232764345 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 632378 n bits 320 index fs_storage_path_hash of table `nextcloud`.`oc_filecache` trx id 2414955 lock_mode X locks rec but not gap waiting
Record lock, heap no 269 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
 0: len 8; hex 800000g0000000ef1; asc         ;;
 1: len 30; hex 343562393633333937616134306434613030363365306438356534666537; asc 45b963397aa40d4a0063e0d85e4fe7; (total 32 bytes);
 2: len 8; hex 8000000005c3025f; asc        _;;

------------------
---TRANSACTION 2414863, ACTIVE 301 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 6 lock struct(s), heap size 1128, 5 row lock(s)
MariaDB thread id 779457, OS thread handle 139952660903488, query id 36692851 10.10.0.5 nextcloud Sending data
SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 57) AND (`path_hash` IN ('3284223de1ffe5d2a4d1e58681bda01b', '45b963397aa40d4a0063e0d85e4fe7a1', 'd41d8cd98f00b204e9800998ecf8427e')) ORDER BY `path_hash` ASC FOR UPDATE
------- TRX HAS BEEN WAITING 301358543 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 567859 n bits 320 index fs_storage_path_hash of table `nextcloud`.`oc_filecache` trx id 2414863 lock_mode X locks rec but not gap waiting
Record lock, heap no 87 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
 0: len 8; hex 8000000000000039; asc        9;;
 1: len 30; hex 643431643863643938663030623230346539383030393938656366383432; asc d41d8cd98f00b204e9800998ecf842; (total 32 bytes);
 2: len 8; hex 80000000006bd9e9; asc      k  ;;

------------------
---TRANSACTION 2414780, ACTIVE 375 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1128, 1 row lock(s)
MariaDB thread id 779370, OS thread handle 139934837864000, query id 36691388 10.10.0.5 nextcloud Sending data
SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 3239) AND (`path_hash` IN ('45b963397aa40d4a0063e0d85e4fe7a1', '59b457f3aeff6e10f1f34d60690fcc14', '5f2e1f76a7fdbba43f6fbacdd8117ccb', '6659c5f228048ebce94a21f0165e00f5', '7622e3ad5c706bac2ae9a9703f405c1c', 'ab18d35aab0526c66a8511ca4eb1075a', 'd41d8cd98f00b204e9800998ecf8427e', 'f1537afbc20da181b360a8bdb3f0ae46')) ORDER BY `path_hash` ASC FOR UPDATE
------- TRX HAS BEEN WAITING 374826145 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 1177496 n bits 320 index fs_storage_path_hash of table `nextcloud`.`oc_filecache` trx id 2414780 lock_mode X locks rec but not gap waiting
Record lock, heap no 216 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
 0: len 8; hex 8000000000000ca7; asc         ;;
 1: len 30; hex 343562393633333937616134306434613030363365306438356534666537; asc 45b963397aa40d4a0063e0d85e4fe7; (total 32 bytes);
 2: len 8; hex 800000000504dad9; asc         ;;

------------------
---TRANSACTION 2414717, ACTIVE 425 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 1128, 1 row lock(s)
MariaDB thread id 779293, OS thread handle 139952653125184, query id 36690111 10.10.0.5 nextcloud Sending data
SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 3239) AND (`path_hash` IN ('45b963397aa40d4a0063e0d85e4fe7a1', '5f2e1f76a7fdbba43f6fbacdd8117ccb', '6659c5f228048ebce94a21f0165e00f5', '7622e3ad5c706bac2ae9a9703f405c1c', '8975712643d5543bca4d1becc89e0ca4', 'd41d8cd98f00b204e9800998ecf8427e', 'f1537afbc20da181b360a8bdb3f0ae46')) ORDER BY `path_hash` ASC FOR UPDATE
------- TRX HAS BEEN WAITING 425099480 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 1177496 n bits 320 index fs_storage_path_hash of table `nextcloud`.`oc_filecache` trx id 2414717 lock_mode X locks rec but not gap waiting
Record lock, heap no 216 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
 0: len 8; hex 8000000000000ca7; asc         ;;
 1: len 30; hex 343562393633333937616134306434613030363365306438356534666537; asc 45b963397aa40d4a0063e0d85e4fe7; (total 32 bytes);
 2: len 8; hex 800000000504dad9; asc         ;;

------------------
---TRANSACTION 2414175, ACTIVE 705 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 8 lock struct(s), heap size 1128, 7 row lock(s)
MariaDB thread id 778559, OS thread handle 139952787232320, query id 36681914 10.10.0.5 nextcloud Sending data
SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 3239) AND (`path_hash` IN ('45b963397aa40d4a0063e0d85e4fe7a1', '53a6e1445b73b8385cc0936a7004c09f', '7622e3ad5c706bac2ae9a9703f405c1c', 'd41d8cd98f00b204e9800998ecf8427e', 'f1537afbc20da181b360a8bdb3f0ae46')) ORDER BY `path_hash` ASC FOR UPDATE
------- TRX HAS BEEN WAITING 425113980 us FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 265 page no 629221 n bits 320 index fs_storage_path_hash of table `nextcloud`.`oc_filecache` trx id 2414175 lock_mode X locks rec but not gap waiting
Record lock, heap no 44 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
 0: len 8; hex 8000000000000ca7; asc         ;;
 1: len 30; hex 643431643863643938663030623230346539383030393938656366383432; asc d41d8cd98f00b204e9800998ecf842; (total 32 bytes);
 2: len 8; hex 800000000504dad7; asc         ;;

------------------
---TRANSACTION 2313957, ACTIVE 52605 sec rollback
ROLLING BACK 19628 lock struct(s), heap size 2334840, 725852 row lock(s), undo log entries 1115049
MariaDB thread id 380250, OS thread handle 139952663975488, query id 34463391 10.10.0.5 nextcloud Rollback
--------
....

Configuration

Nextcloud

The output of occ config:list system or similar is best, but, if not possible, the contents of your config.php file from /path/to/nextcloud is fine (make sure to remove any identifiable information!):

{
    "system": {
        "allow_user_to_change_display_name": false,
        "default_language": "fr",
        "enable_avatars": false,
        "forcessl": true,
        "knowledgebaseenabled": true,
        "knowledgebaseurl": "xxxx",
        "ldapIgnoreNamingRules": false,
        "ldap": {
            "ldapConnectionTimeout": 10,
            "ldapNetworkTimeout": 10
        },
        "loglevel": 2,
        "log_type": "file",
        "logfile": "\/var\/www\/html\/data\/nextcloud.log",
        "log_authfailip": true,
        "logtimezone": "Europe\/Paris",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "csrf.disabled": true,
        "share_folder": "\/Shared",
        "trashbin_retention_obligation": "30, auto",
        "activity_expire_days": "365",
        "session_keepalive": true,
        "remember_login_cookie_lifetime": 2592000,
        "filelocking.enabled": false,
        "maintenance_window_start": 1,
        "allowed_admin_ranges": xxxxx
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "upgrade.disable-web": true,
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "cloud.irit.fr",
            "cloudr10.irit.fr"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "33.0.1.2",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "overwriteprotocol": "https",
        "overwrite.cli.url": "https:\/\/cloud.irit.fr",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "lost_password_link": "disabled",
        "profile.enabled": false,
        "maintenance": false,
        "config_preset": 7,
        "app_install_overwrite": [
            "files_rightclick"
        ],
        "htaccess.RewriteBase": "\/",
        "mail_sendmailmode": "smtp",
        "mail_smtpstreamoptions": {
            "ssl": {
                "allow_self_signed": true,
                "verify_peer": false,
                "verify_peer_name": false
            }
        }
    },
    apps":xxxx
}

Job list

docker compose exec --user www-data app php occ background-job:list -v -l 25000
+-------------------+-------------------------------------------------------------------+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|  id                 |  class                                                              |  last_run                   | xxx |
+-------------------+-------------------------------------------------------------------+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|  4                  |  OCA\Activity\BackgroundJob\EmailNotification                       |  2026-04-01T18:39:00+00:00  | xxx |
|  7                  |  OCA\Activity\BackgroundJob\ExpireActivities                        |  2026-04-01T18:39:01+00:00  | xxx |
|  11                 |  OCA\Files\BackgroundJob\CleanupFileLocks                           |  2026-04-01T18:39:45+00:00  | xxx |
|  12                 |  OCA\Files_Trashbin\BackgroundJob\ExpireTrash                       |  2026-04-01T18:04:28+00:00  | xxx |
|  13                 |  OCA\Files_Versions\BackgroundJob\ExpireVersions                    |  2026-04-01T18:04:28+00:00  | xxx |
|  14                 |  OCA\Files\BackgroundJob\ScanFiles                                  |  2026-04-01T18:39:45+00:00  | xxx |
|  15                 |  OCA\Files\BackgroundJob\DeleteOrphanedItems                        |  2026-04-01T18:03:47+00:00  | xxx |
|  17                 |  OCA\Federation\SyncJob                                             |  2026-04-01T18:39:12+00:00  | xxx |
|  22                 |  OCA\Files_Sharing\DeleteOrphanedSharesJob                          |  2026-04-01T18:04:28+00:00  | xxx |
|  23                 |  OCA\Files_Sharing\ExpireSharesJob                                  |  2026-04-01T18:04:28+00:00  | xxx |
|  24                 |  OCA\User_LDAP\Jobs\UpdateGroups                                    |  2026-04-01T18:39:12+00:00  | xxx |
|  25                 |  OCA\User_LDAP\Jobs\CleanUp                                         |  2026-04-01T18:39:44+00:00  | xxx |
|  3093374            |  OCA\Onlyoffice\Cron\EditorsCheck                                   |  2026-04-01T18:07:10+00:00  | xxx |
|  5300084            |  OC\Authentication\Token\TokenCleanupJob                            |  1970-01-01T00:00:00+00:00  | xxx |
|  5300085            |  OC\Log\Rotate                                                      |  1970-01-01T00:00:00+00:00  | xxx |
|  5300086            |  OC\Preview\BackgroundCleanupJob                                    |  1970-01-01T00:00:00+00:00  | xxx |
|  5300116            |  OCA\User_LDAP\Jobs\Sync                                            |  2026-04-01T18:39:44+00:00  | xxx |
|  5300117            |  OCA\Files\BackgroundJob\CleanupDirectEditingTokens                 |  2026-04-01T18:03:47+00:00  | xxx |
|  5300118            |  OCA\Files\BackgroundJob\DeleteExpiredOpenLocalEditor               |  2026-04-01T18:03:47+00:00  | xxx |
|  5300119            |  OCA\Activity\BackgroundJob\DigestMail                              |  2026-04-01T18:39:44+00:00  | xxx |
|  5300124            |  OCA\DAV\BackgroundJob\CleanupDirectLinksJob                        |  2026-04-01T18:04:28+00:00  | xxx |
|  5300125            |  OCA\DAV\BackgroundJob\UpdateCalendarResourcesRoomsBackgroundJob    |  2026-04-01T18:06:25+00:00  | xxx |
|  5300126            |  OCA\DAV\BackgroundJob\CleanupInvitationTokenJob                    |  2026-04-01T18:04:28+00:00  | xxx |
|  5300127            |  OCA\DAV\BackgroundJob\EventReminderJob                             |  2026-04-01T18:04:28+00:00  | xxx |
|  5300128            |  OCA\DAV\BackgroundJob\CalendarRetentionJob                         |  2026-04-01T18:04:28+00:00  | xxx |
|  5300129            |  OCA\Files_Sharing\BackgroundJob\FederatedSharesDiscoverJob         |  2026-04-01T18:04:28+00:00  | xxx |
|  5300130            |  OCA\Notifications\BackgroundJob\GenerateUserSettings               |  2026-04-01T18:04:29+00:00  | xxx |
|  5300131            |  OCA\Notifications\BackgroundJob\SendNotificationMails              |  2026-04-01T18:04:30+00:00  | xxx |
|  5300132            |  OCA\ServerInfo\Jobs\UpdateStorageStats                             |  2026-04-01T18:04:31+00:00  | xxx |
|  5300134            |  OCA\WorkflowEngine\BackgroundJobs\Rotate                           |  2026-04-01T18:04:30+00:00  | xxx |
|  5300139            |  OCA\Text\Cron\Cleanup                                              |  2026-04-01T18:04:47+00:00  | xxx |
|  5300977            |  OC\Core\BackgroundJobs\BackgroundCleanupUpdaterBackupsJob          |  1970-01-01T00:00:00+00:00  | xxx |
|  5300978            |  OC\Core\BackgroundJobs\CleanupLoginFlowV2                          |  1970-01-01T00:00:00+00:00  | xxx |
|  5300979            |  OC\Security\Bruteforce\CleanupJob                                  |  1970-01-01T00:00:00+00:00  | xxx |
|  5300981            |  OC\Core\BackgroundJobs\LookupServerSendCheckBackgroundJob          |  1970-01-01T00:00:00+00:00  | xxx |
|  5300985            |  OCA\OAuth2\BackgroundJob\CleanupExpiredAuthorizationCode           |  2026-03-31T16:14:34+00:00  | xxx |
|  5300987            |  OCA\DAV\BackgroundJob\PruneOutdatedSyncTokensJob                   |  2026-04-01T18:04:28+00:00  | xxx |
|  5300988            |  OC\TextProcessing\RemoveOldTasksBackgroundJob                      |  1970-01-01T00:00:00+00:00  | xxx |
|  5300989            |  OCA\FilesReminders\BackgroundJob\CleanUpReminders                  |  2026-04-01T18:04:48+00:00  | xxx |
|  5300990            |  OCA\FilesReminders\BackgroundJob\ScheduledNotifications            |  2026-04-01T18:04:48+00:00  | xxx |
|  5300991            |  OC\User\BackgroundJobs\CleanupDeletedUsers                         |  1970-01-01T00:00:00+00:00  | xxx |
|  5300992            |  OC\TextToImage\RemoveOldTasksBackgroundJob                         |  1970-01-01T00:00:00+00:00  | xxx |
|  5300993            |  OC\Core\BackgroundJobs\GenerateMetadataJob                         |  1970-01-01T00:00:00+00:00  | xxx |
|  5301001            |  OCA\Activity\BackgroundJob\RemoveFormerActivitySettings            |  2026-04-01T18:39:44+00:00  | xxx |
|  5301031            |  OCA\UpdateNotification\BackgroundJob\UpdateAvailableNotifications  |  2026-04-01T18:04:48+00:00  | xxx |
|  5301036            |  OC\Repair\NC29\SanitizeAccountPropertiesJob                        |  1970-01-01T00:00:00+00:00  | xxx |
|  5301089            |  OC\TaskProcessing\RemoveOldTasksBackgroundJob                      |  1970-01-01T00:00:00+00:00  | xxx |
|  5301102            |  OCA\Files_Sharing\SharesReminderJob                                |  2026-04-01T18:04:30+00:00  | xxx |
|  5301146            |  OCA\DAV\BackgroundJob\FederatedCalendarPeriodicSyncJob             |  2026-04-01T18:04:28+00:00  | xxx |
|  67322677247979520  |  OC\Core\BackgroundJobs\PreviewMigrationJob                         |  1970-01-01T00:00:00+00:00  | xxx |
|  67330589462732801  |  OCA\TwoFactorNextcloudNotification\BackgroundJob\CleanupTokens     |  2026-04-01T18:07:10+00:00  | xxx |
|  67349348562870273  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
xxxxx
|  67832264861114369  |  OC\FilesMetadata\Job\UpdateSingleMetadata                          |  1970-01-01T00:00:00+00:00  | xxx |
|  67832771260407809  |  OC\FilesMetadata\Job\UpdateSingleMetadata                          |  1970-01-01T00:00:00+00:00  | xxx |
|  67848049033699329  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67848050187132929  |  OC\FilesMetadata\Job\UpdateSingleMetadata                          |  1970-01-01T00:00:00+00:00  | xxx |
|  67848061624999937  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67848064108027905  |  OC\FilesMetadata\Job\UpdateSingleMetadata                          |  1970-01-01T00:00:00+00:00  | xxx |
|  67848284606783489  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67848304047382529  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67849061744205825  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67849066580238337  |  OC\FilesMetadata\Job\UpdateSingleMetadata                          |  1970-01-01T00:00:00+00:00  | xxx |
|  67850108839616513  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67866286735712257  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67867597652844545  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67871567377031169  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67873006681808897  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67873006744723457  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67874323978477569  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67875821684744193  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67876959502950401  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67876966662627329  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67878417954725889  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67879030432161793  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67879737101082625  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67880912953237505  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67884261572304897  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67884540376080385  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67885014332432385  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67886303309160449  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67893042129625089  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67895540361945089  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67898864175898625  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67906523138580481  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67907813944025089  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67907870189641729  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67909160953143297  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67911862470795265  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67911866866425857  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67913210901782529  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67915911974838273  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67919949017604097  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67934980006436865  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67981381767749633  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67987655540563969  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67987657901957121  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67988950708084737  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67993272284045313  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67995275039039489  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  67996322650349569  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68003406217437185  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68003409270890497  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68004717897932801  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68005934040571905  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68005937123385345  |  OC\FilesMetadata\Job\UpdateSingleMetadata                          |  1970-01-01T00:00:00+00:00  | xxx |
|  68006874223173633  |  OC\FilesMetadata\Job\UpdateSingleMetadata                          |  1970-01-01T00:00:00+00:00  | xxx |
|  68006948831453185  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68006957744349185  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68009184261267457  |  OCA\DAV\BackgroundJob\UploadCleanup                                |  1970-01-01T00:00:00+00:00  | xxx |
|  68011338703265793  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68011724210135041  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68013059810746369  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68013060926431233  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68015460663910401  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68016873053839361  |  OC\FilesMetadata\Job\UpdateSingleMetadata                          |  1970-01-01T00:00:00+00:00  | xxx |
|  68018544261685249  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68018546195259393  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68021919913041921  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68022909348986881  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68023367517978625  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68025550359609345  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68025598355030017  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68025835723276289  |  OC\FilesMetadata\Job\UpdateSingleMetadata                          |  1970-01-01T00:00:00+00:00  | xxx |
|  68026887545667585  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68026887545667586  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68026906340343809  |  OC\Command\CommandJob                                              |  1970-01-01T00:00:00+00:00  | xxx |
|  68032060336562177  |  OC\FilesMetadata\Job\UpdateSingleMetadata                          |  1970-01-01T00:00:00+00:00  | xxx |
|  68035578707501057  |  OC\FilesMetadata\Job\UpdateSingleMetadata                          |  1970-01-01T00:00:00+00:00  | xxx |
+-------------------+-------------------------------------------------------------------+---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2

Try 33.0.2…

3

Welcome!

Can you confirm there are no warnings or errors under Admin settings->Overview, particularly database/index related?

Also, any clues in the Nextcloud log?

You may need to lower the log level to 1 (INFO level) temporarily to see a bit more about what’s going on with the jobs.

Just to make sure there isn’t a misunderstanding: When you say “stuck”, you’re not just referring to the worker not exiting, correct? Because the worker mode for cron (versus running cron.php directly every 5 minutes) is meant to be a persistent process.

Lastly, what specific MySQL/MariaDB version?

4

Thank you for welcome !

Everything looks good in the Overview page.

image
image1147×821 56.5 KB

I temporarily disabled file locking to avoid blocking my users (I have about 900 accounts on this instance).

The logs are not very helpful — there are a lot of false positives at INFO level but I active now .

From yesterday evening until this morning, the number of waiting jobs remained exactly the same, as if the process had been killed in between.
(The firewall cuts inactive connections, so I’m not sure what happened. I didn’t have tmux running at the time, only on my bastion server.)

When I tried to restart the job, I got lock errors again, similar to yesterday, with transactions stuck in the database.
This afternoon, once the rollback finally finished, I manually relaunched a few jobs, but I don’t know which one is causing the crash.

Sorry, it’s a bit unclear, but I don’t really understand what’s happening. I don’t know which job is blocking it, but I’ll restart everything this weekend to avoid generating users

What I do know is that when a transaction is in ROLLBACK, the whole instance becomes unusable.

As a workaround, I have blocked access to the synchronization clients via Nginx.

I have mariadb:10.11 and redis:8.4
Thank’s

5

Is the undo log entries on the rollback transaction dropping periodically when you re-run SHOW ENGINE INNODB STATUS periodically?

Any idea what happened ~14.6 hours ago? Was that about when you did the migration and/or upgrade(s)?

Assuming the undo log entries counter is dropping (even if very slowly), there may be other contributing factors on the db tuning side (i.e. innodb_buffer_pool_size that exasperated things).

I wouldn’t stress about the other jobs getting backed up; they will clear out when the blocker (the big rollback) finishes.

This afternoon, once the rollback finally finished

Wait, it finished at some point then a new big rollback came back?

What I do know is that when a transaction is in ROLLBACK , the whole instance becomes unusable.

I have mariadb:10.11 and redis:8.4

What version specifically? I ask because, for example, within th 10.11 series, 10.11.16 is the latest maintenance release. There are relevant bugs from time to time: MariaDB 10.11 Changes & Improvements | Release Notes | MariaDB Documentation

6

With NC33 that InnoDB rollback is painful to watch but the worst thing you can do is interrupt it mid-flight. Let it run to completion — killing it will leave your database in a partially rolled-back state that’s far harder to repair.

A few things worth checking while you wait:

On the InnoDB rollback:

  • Connect to MySQL/MariaDB and run SHOW ENGINE INNODB STATUSG — look for the “TRANSACTIONS” section to see rollback progress
  • If your tables are large, this can take hours; the rollback has to undo every uncommitted change row by row
  • Bump innodb_lock_wait_timeout to 300 temporarily if you’re seeing lock timeout errors: SET GLOBAL innodb_lock_wait_timeout = 300;

After the rollback completes:
Run these in order:

php occ maintenance:mode --on
php occ maintenance:repair
php occ db:add-missing-indices
php occ db:convert-filecache-bigint
php occ maintenance:mode --off

On the cron jobs being stuck:
The stuck cron jobs are almost certainly a symptom of the upgrade transaction not completing cleanly. Once maintenance:repair finishes, flush the job queue:

php occ background:job --force-run

Or from the web UI: Admin → Basic settings → Background jobs → click “Run now”.

If you’re on a large instance and cron was mid-run during the upgrade, some jobs may have their “last run” timestamp in the future — you can reset them with:

php occ background:job:list

Check for any jobs showing a next_run in the past that haven’t fired. Disabling and re-enabling the app they belong to usually clears it.

What database engine are you on — MariaDB or MySQL, and what version? That affects whether the rollback should complete on its own or needs a manual crash recovery.

7

Yes, the rollback eventually completes after a few hours.
However, every time I restart the cron jobs (with background-job:worker), one of them causes the instance to crash again, putting MariaDB back into a long ROLLING BACK transaction.

I suspect it may be due to too much concurrent activity between desktop client synchronizations and background jobs.

I’m using MariaDB 10.11.16 and have increased innodb_lock_wait_timeout to 600 seconds and i set innodb_flush_log_at_trx_commit to 1 to try to speed up the process.

I have already executed occ maintenance:repair and all db:* commands after each version upgrade, with no errors reported.

Unfortunately, I cannot enable maintenance mode because the service is under heavy load, particularly for collaborative document editing.
As this is a research institution, the platform must remain accessible to our users at all times.

cat config/mariadb/my.cnf/my.cnf
[mysqld]

# === InnoDB  ===
# Règle générale : 70-80% de la mémoire allouée au conteneur
innodb_buffer_pool_size = 16G

innodb_log_file_size      = 2G        # ou 25-50% du buffer pool
innodb_flush_log_at_trx_commit = 1   # 2 = bon compromis performance/sécurité
innodb_flush_method       = O_DIRECT
innodb_file_per_table     = 1
innodb_large_prefix=1
innodb_buffer_pool_instances=2
innodb_log_buffer_size=128M

innodb_read_io_threads=4         # doc. owncloud : 4
innodb_write_io_threads=4        # doc. owncloud : 4²/io_cac

innodb_purge_batch_size=500
innodb_purge_threads=8


# === Autres optimisations importantes ===
aria_pagecache_buffer_size  = 128M     # Aria = moteur pour tables temporaires
key_buffer_size             = 64M

# Tables temporaires en mémoire
tmp_table_size              = 64M
max_heap_table_size         = 64M

# Connections
thread_handling             = pool-of-threads   # Meilleur pour beaucoup de connexions simultanées

# Autres bonnes pratiques
table_open_cache            = 4000
open_files_limit            = 65535
innodb_io_capacity          = 4000
innodb_io_capacity_max      = 8000
innodb_flush_neighbors      = 0


# Connections et cache
max_connections                 = 400
table_open_cache                = 4000
open_files_limit                = 65535
max_allowed_packet           = 512M



# === Timeouts augmentés ===
innodb_lock_wait_timeout     = 600     # Temps max qu'une requête attend un verrou (défaut 50s) → réduit les erreurs 1205
wait_timeout                 = 600     # Temps max d'inactivité d'une connexion (10 minutes)
interactive_timeout          = 600
net_read_timeout             = 300
net_write_timeout            = 300
8

Edit: Problem solved After running all the jobs manually! Thanks

One last question: Is it possible to run occ maintenance:repair without putting Nextcloud into maintenance mode?