Nextcloud 20.0 stuck in a lock state

Nextcloud version (eg, 20.0.5): 20.0.12.1
Operating system and version (eg, Ubuntu 20.04): unknown
Docker image: nextcloud:20-apache
Apache or nginx version (eg, Apache 2.4.25): apache
PHP version (eg, 7.4): php 7.4

The issue you are facing:

My production server asked to update Calendar app so I did it, then the server stopped working as expected, apache server wasn’t responding anymore. I restarted the containers, it came back, I did the update again, same thing. I updated the shasum of the Docker image to upgrade from 20.0.11.1 to 20.0.12.1. Then the logs says :

Conf remoteip disabled.
To activate the new configuration, you need to run:
  service apache2 reload
Configuring Redis as session handler
Initializing nextcloud 20.0.12.1 ...
Upgrading nextcloud from 20.0.11.1 ...
Initializing finished
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Setting log level to debug

So I exec into the container and execute occ, but the program is not responding at all, it’s stuck in stdin… I can’t run it as www-data either.

Note I’m deploying using Docker and Kubernetes.

Is this the first time you’ve seen this error? (Y/N): yes

Steps to replicate it:

  1. Deploy NC 20.0.11.1
  2. Update any app or upgrade to 20.0.12.1

The output of your Nextcloud log in Admin > Logging:

{"reqId":"Cvymno9EV3f5gXjxXWrq","level":3,"time":"2021-08-20T14:12:47+00:00","remoteAddr":"10.244.0.242","user":"sundown","app":"index","method":"GET","url":"/index.phpapps/update/calendar","message":{"Exception":"Exception","Message":"Could not acquire an exclusive lock on the config file /var/www/html/config/config.php","Code":0,"Trace":[{"file":"/var/www/html/lib/private/Config.php","line":134,"function":"writeData","class":"OC\\Config","type":"->","args":[]},{"file":"/var/www/html/lib/private/SystemConfig.php","line":111,"function":"setValue","class":"OC\\Config","type":"->","args":["maintenance",true]},{"file":"/var/www/html/lib/private/AllConfig.php","line":116,"function":"setValue","class":"OC\\SystemConfig","type":"->","args":["maintenance",true]},{"file":"/var/www/html/apps/settings/lib/Controller/AppSettingsController.php","line":530,"function":"setSystemValue","class":"OC\\AllConfig","type":"->","args":["maintenance",true]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":169,"function":"updateApp","class":"OCA\\Settings\\Controller\\AppSettingsController","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":100,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Settings\\Controller\\AppSettingsController"},"updateApp"]},{"file":"/var/www/html/lib/private/AppFramework/App.php","line":152,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Settings\\Controller\\AppSettingsController"},"updateApp"]},{"file":"/var/www/html/lib/private/Route/Router.php","line":309,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["OCA\\Settings\\Controller\\AppSettingsController","updateApp",{"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},{"appId":"*** sensitive parameter replaced ***","_route":"settings.AppSettings.updateApp"}]},{"file":"/var/www/html/lib/base.php","line":1008,"function":"match","class":"OC\\Route\\Router","type":"->","args":["/settings/apps/update/calendar"]},{"file":"/var/www/html/index.php","line":37,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"/var/www/html/lib/private/Config.php","Line":257,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0","version":"20.0.11.1"}
{"reqId":"awg6W5Ok1aoMven89B7x","level":1,"time":"2021-08-20T14:21:55+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"/occ","message":"\\OC\\Updater::setDebugLogLevel: Set log level to debug","userAgent":"--","version":"20.0.11.1"}

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => 'nextcloud-redis-master',
    'password' => '*******************',
    'port' => 6379,
  ),
  'passwordsalt' => '*************',
  'secret' => '**************',
  'trusted_domains' => 
  array (
    0 => 'localhost',
    1 => '********',
  ),
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'pgsql',
  'version' => '20.0.11.1',
  'overwrite.cli.url' => 'http://localhost',
  'dbname' => 'nextcloud',
  'dbhost' => 'nextcloud-postgresql',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'nextcloud',
  'dbpassword' => '**************',
  'installed' => true,
  'instanceid' => '***********',
  'loglevel' => 1,
  'maintenance' => false,
  'twofactor_enforced' => 'true',
  'twofactor_enforced_groups' => 
  array (
  ),
  'twofactor_enforced_excluded_groups' => 
  array (
  ),
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'overwriteprotocol' => 'https',
  'overwritewebroot' => '/',
  'trusted_proxies' => 
  array (
    0 => '10.106.0.0/20',
  ),
  'theme' => '',
);

The output of your Apache/nginx/system log in /var/log/____: log files are empty

My instance is back online!

This is what I’ve done:

  • Restart container, exec into it
  • Set maintenance mode on using occ
  • Update all apps with occ app:update --all
  • Run occ maintenance:repair
  • Run occ upgrade
  • Disable maintenance mode