No logs, Internal error after Server restart, DB errors

Hello everyone,

about 2 days ago, I restarted my server.
Nextcloud has been working great until then.

After the restart, my monitoring tool reported that replies from Nextcloud had a high average response time and returned 500. Manually requesting the page returns the following:

Internal Server Error

The server encountered an internal error and was unable to complete your request.
Please contact the server administrator if this error reappears multiple times, please include the technical details below in your report.
More details can be found in the server log.

In the Docker logs, I cannot find any useful information, besides that it returns 500 continiously.

AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.1.151. Set the 'ServerName' directive globally to suppress this message
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.1.151. Set the 'ServerName' directive globally to suppress this message
[Sun May 21 18:31:33.412900 2023] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.56 (Debian) PHP/8.1.19 configured -- resuming normal operations
[Sun May 21 18:31:33.412930 2023] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
192.168.1.150 - - [19/May/2023:22:50:09 +0200] "GET /core/preview?fileId=129224&c=c939ac1bd2b93a6d2fac249bb425b593&x=250&y=250&forceIcon=0&a=1 HTTP/1.1" 200 9065 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:22:50:08 +0200] "GET /core/preview?fileId=129268&c=46d90e85dc88b99dc58874f7ef90ef37&x=250&y=250&forceIcon=0&a=1 HTTP/1.1" 200 9495 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:22:50:07 +0200] "GET /core/preview?fileId=129276&c=5ce1239a6d736e1a5e54caefa1a5f4d8&x=250&y=250&forceIcon=0&a=1 HTTP/1.1" 200 10149 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:22:50:09 +0200] "GET /core/preview?fileId=129238&c=11fbef05c0e9e0514fd50fef3127160d&x=250&y=250&forceIcon=0&a=1 HTTP/1.1" 200 7790 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:22:50:10 +0200] "GET /core/preview?fileId=129162&c=cb803f5fd1b8ef0a585ae27c0179069f&x=250&y=250&forceIcon=0&a=1 HTTP/1.1" 200 9486 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:22:50:09 +0200] "GET /core/preview?fileId=129180&c=db1cf4766e762d28aa511135fdc2ef49&x=250&y=250&forceIcon=0&a=1 HTTP/1.1" 200 9161 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:22:50:11 +0200] "GET /core/preview?fileId=129647&c=8e8db41f54bf4bd4b89f6fbfceb7f161&x=250&y=250&forceIcon=0&a=1 HTTP/1.1" 200 9338 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:23:33:55 +0200] "GET /apps/files/?dir=/Bilder/Conventions/Fantasy%20Basel%202023&fileid=129069 HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:23:34:11 +0200] "GET /favicon.ico HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:23:34:37 +0200] "GET /apps/files/?dir=/Bilder/Conventions/Fantasy%20Basel%202023&fileid=129069 HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:23:34:52 +0200] "GET /favicon.ico HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - admin [19/May/2023:23:37:40 +0200] "GET /status.php HTTP/1.1" 500 410 "-" "Mozilla/5.0 (Windows) mirall/3.8.2stable-Win64 (build 20230516) (Nextcloud, windows-10.0.22621 ClientArchitecture: x86_64 OsArchitecture: x86_64)"
192.168.1.150 - - [19/May/2023:23:40:02 +0200] "GET /apps/files/?dir=/Bilder/Conventions/Fantasy%20Basel%202023&fileid=129069 HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:23:40:16 +0200] "GET /favicon.ico HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:23:40:22 +0200] "GET / HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:23:40:34 +0200] "GET /favicon.ico HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:23:43:15 +0200] "GET / HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.6 - - [19/May/2023:23:43:23 +0200] "GET / HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [19/May/2023:23:43:31 +0200] "GET /favicon.ico HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.6 - - [19/May/2023:23:43:37 +0200] "GET /favicon.ico HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - admin [19/May/2023:23:46:59 +0200] "GET /status.php HTTP/1.1" 500 410 "-" "Mozilla/5.0 (Windows) mirall/3.8.2stable-Win64 (build 20230516) (Nextcloud, windows-10.0.22621 ClientArchitecture: x86_64 OsArchitecture: x86_64)"
192.168.1.150 - admin [19/May/2023:23:56:17 +0200] "GET /status.php HTTP/1.1" 500 410 "-" "Mozilla/5.0 (Windows) mirall/3.8.2stable-Win64 (build 20230516) (Nextcloud, windows-10.0.22621 ClientArchitecture: x86_64 OsArchitecture: x86_64)"
192.168.1.150 - - [20/May/2023:00:00:36 +0200] "GET / HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [20/May/2023:00:00:52 +0200] "GET /favicon.ico HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [20/May/2023:00:00:57 +0200] "GET / HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [20/May/2023:00:01:10 +0200] "GET /favicon.ico HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - admin [20/May/2023:00:05:35 +0200] "GET /status.php HTTP/1.1" 500 410 "-" "Mozilla/5.0 (Windows) mirall/3.8.2stable-Win64 (build 20230516) (Nextcloud, windows-10.0.22621 ClientArchitecture: x86_64 OsArchitecture: x86_64)"
192.168.1.150 - - [20/May/2023:00:10:51 +0200] "GET / HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [20/May/2023:00:10:51 +0200] "GET /favicon.ico HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"
192.168.1.150 - - [20/May/2023:00:10:54 +0200] "GET / HTTP/1.1" 500 702 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36"

Hence, I tried to check in the log file. However, it always stays empty and there are no lines in it.
I tried deleting it to recreate it (which it did), but I still get no log entries.

After reading through the documentation, I verified that the database is indeed reachable, user account is valid an can log in and there are no further problems (my Desktop client can query data without problems).

After using the console to enable maintenance mode, I got this error:

# php occ maintenance:mode --on
An unhandled exception has been thrown:
Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2002] No route to host in /var/www/html/lib/private/DB/Connection.php:142
Stack trace:
#0 /var/www/html/3rdparty/doctrine/dbal/src/Connection.php(1531): OC\DB\Connection->connect()
#1 /var/www/html/3rdparty/doctrine/dbal/src/Connection.php(1029): Doctrine\DBAL\Connection->getWrappedConnection()
#2 /var/www/html/lib/private/DB/Connection.php(264): Doctrine\DBAL\Connection->executeQuery('SELECT * FROM `...', Array, Array, NULL)
#3 /var/www/html/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php(345): OC\DB\Connection->executeQuery('SELECT * FROM `...', Array, Array)
#4 /var/www/html/lib/private/DB/QueryBuilder/QueryBuilder.php(280): Doctrine\DBAL\Query\QueryBuilder->execute()
#5 /var/www/html/lib/private/AppConfig.php(418): OC\DB\QueryBuilder\QueryBuilder->execute()
#6 /var/www/html/lib/private/AppConfig.php(184): OC\AppConfig->loadConfigValues()
#7 /var/www/html/lib/private/AppConfig.php(374): OC\AppConfig->getApps()
#8 /var/www/html/lib/private/legacy/OC_App.php(976): OC\AppConfig->getValues(false, 'installed_versi...')
#9 /var/www/html/lib/private/Server.php(729): OC_App::getAppVersions()
#10 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(171): OC\Server->OC\{closure}(Object(OC\Server))
#11 /var/www/html/3rdparty/pimple/pimple/src/Pimple/Container.php(122): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#12 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(138): Pimple\Container->offsetGet('OC\\Memcache\\Fac...')
#13 /var/www/html/lib/private/ServerContainer.php(171): OC\AppFramework\Utility\SimpleContainer->query('OC\\Memcache\\Fac...', true)
#14 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(65): OC\ServerContainer->query('OC\\Memcache\\Fac...')
#15 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(193): OC\AppFramework\Utility\SimpleContainer->get('OC\\Memcache\\Fac...')
#16 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(171): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(OC\Server))
#17 /var/www/html/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#18 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(138): Pimple\Container->offsetGet('OCP\\ICacheFacto...')
#19 /var/www/html/lib/private/ServerContainer.php(171): OC\AppFramework\Utility\SimpleContainer->query('OCP\\ICacheFacto...', true)
#20 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(65): OC\ServerContainer->query('OCP\\ICacheFacto...')
#21 /var/www/html/lib/private/Server.php(1110): OC\AppFramework\Utility\SimpleContainer->get('OCP\\ICacheFacto...')
#22 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(171): OC\Server->OC\{closure}(Object(OC\Server))
#23 /var/www/html/3rdparty/pimple/pimple/src/Pimple/Container.php(122): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#24 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(138): Pimple\Container->offsetGet('OCP\\Lock\\ILocki...')
#25 /var/www/html/lib/private/ServerContainer.php(171): OC\AppFramework\Utility\SimpleContainer->query('OCP\\Lock\\ILocki...', true)
#26 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(65): OC\ServerContainer->query('OCP\\Lock\\ILocki...')
#27 /var/www/html/lib/private/Server.php(2065): OC\AppFramework\Utility\SimpleContainer->get('OCP\\Lock\\ILocki...')
#28 /var/www/html/lib/private/Files/View.php(119): OC\Server->getLockingProvider()
#29 /var/www/html/lib/private/Server.php(464): OC\Files\View->__construct()
#30 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(171): OC\Server->OC\{closure}(Object(OC\Server))
#31 /var/www/html/3rdparty/pimple/pimple/src/Pimple/Container.php(122): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#32 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(138): Pimple\Container->offsetGet('OC\\Files\\Node\\H...')
#33 /var/www/html/lib/private/ServerContainer.php(171): OC\AppFramework\Utility\SimpleContainer->query('OC\\Files\\Node\\H...', true)
#34 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(65): OC\ServerContainer->query('OC\\Files\\Node\\H...')
#35 /var/www/html/lib/private/Server.php(1467): OC\AppFramework\Utility\SimpleContainer->get('OC\\Files\\Node\\H...')
#36 /var/www/html/lib/base.php(623): OC\Server->boot()
#37 /var/www/html/lib/base.php(1165): OC::init()
#38 /var/www/html/console.php(48): require_once('/var/www/html/l...')
#39 /var/www/html/occ(11): require_once('/var/www/html/c...')
#40 {main}# 

It makes me think, that the database connection is not working.

Here’s my config:

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'instanceid' => 'XXXX',
  'passwordsalt' => 'XXXX',
  'secret' => 'XXXXX',
  'trusted_domains' => 
  array (
    0 => 'cloud.XXXXXXXXXXX.com',
    1 => 'XXXXXXX',
  ),
  'overwriteprotocol' => 'https',
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'mysql',
  'version' => '26.0.1.1',
  'overwrite.cli.url' => 'XXXX',
  'dbname' => 'nextcloud',
  'dbhost' => 'XXXX',
  'dbport' => '3306',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'oc_admin',
  'dbpassword' => 'XXXX',
  'installed' => true,
  'loglevel' => 2,
  'maintenance' => false,
  'mail_from_address' => 'XXXX',
  'mail_smtpmode' => 'XXXX',
  'mail_sendmailmode' => 'XXXX',
  'mail_domain' => 'XXXX',
  'mail_smtpsecure' => 'XXXX',
  'mail_smtpauthtype' => 'XXXX',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'XXX',
  'mail_smtpport' => 'XXX',
  'mail_smtpname' => 'XXX',
  'mail_smtppassword' => 'XXXXX',
  'theme' => '',
  'app_install_overwrite' => 
  array (
    0 => 'drawio',
    1 => 'unsplash',
    2 => 'news',
    3 => 'files_mindmap',
    4 => 'extract',
  ),
);

What are the next debug steps, that I could take?
Database connection seems to work fine…

Can you clarify what, specifically, you did to verify your database connection is working?

Also, check the Docker logs for your database container rather than your web server container. That may give some clues.

Can you post your Docker Compose file?

  • My Unraid server is running the following containers:
    • nextcloud
      • running on br0 (custom network).
    • nextcloud-cronjob
      • running in bride mode
      • no exposed ports
    • mysql
      • running in bridge mode
      • exposed TCP 3306
      • mounted files at /var/lib/mysql

After manually installing iputils-ping in the nextcloud container and doing some network checks using “ping ”, it seems that Nextcloud can’t reach any of my other containers.
It probably was some weird issue with Unraid. Sorry for bothering then.

I have updated to a different network mode, updated my reverse proxy and I must wait until SSL & Cloudflare propagates my changes - I currently get gateway errors. Hopefully they’ll clear up. Thanks for your help.

1 Like