Internal Server Error for no apparent reason and without error messages

My Nextcloud seems to break every so often for no apparent reason. I’m running it on Ubuntu Server in Docker with MariaDB. Everything is deployed with Ansible. It somehow broke today when I was deploying a completely unrelated service and Ansible logs don’t show any changes related to Nextcloud.

Basically all information I have is the Internal Server Error page of Nextcloud. None of the log files show any information.

What I tried so far:

  • Delete all Nextcloud directories and restart. This “works” but for obvious reasons I’d rather not
  • chmod all directories to 777. This does nothing
  • Restore the nextcloud directories from an older ZFS snapshot (when it was still running). This also doesn’t work

The DB seems to be up, I can connect to it from the Nextcloud container and if I reset nextcloud it connects normally.

I’m really lost on where to start debugging this. Any pointers would be appreciated

Nextcloud version (eg, 20.0.5): 25.0.2-apache
Operating system and version (eg, Ubuntu 20.04): Ubuntu 22.04 with Docker
Apache or nginx version (eg, Apache 2.4.25): whatever is in the Docker Image
PHP version (eg, 7.4): see above

The issue you are facing:

Is this the first time you’ve seen this error? No

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'overwritehost' => 'nextcloud.',
  'overwriteprotocol' => 'https',
  'instanceid' => 'oczveewjn3kk',
  'passwordsalt' => '',
  'secret' => ',
  'trusted_domains' => 
  array (
    0 => 'nextcloud.',
  ),
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'mysql',
  'version' => '25.0.2.3',
  'overwrite.cli.url' => 'https://nextcloud.',
  'dbname' => 'nextcloud',
  'dbhost' => 'nextcloud-mysql',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud-user',
  'dbpassword' => '',
  'installed' => true,
  'app_install_overwrite' => 
  array (
    0 => 'files_reader',
    1 => 'ehr',
  ),

The output of your Apache/nginx/system log in /var/log/____:

empty

Output errors in nextcloud.log in /var/www/ or as admin user in top right menu, filtering for errors. Use a pastebin service if necessary.

empty

docker logs just shows:

[Sat Dec 17 16:35:35.052268 2022] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.54 (Debian) PHP/8.1.13 configured -- resuming normal operations
[Sat Dec 17 16:35:35.052320 2022] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
172.5.0.4 - - [17/Dec/2022:16:35:41 +0000] "GET / HTTP/1.1" 500 689 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0"