Internal Server Error - MYSQL?

Moin zusammen,

nachdem meine Nextcloud über lange Zeit stabil gelaufen ist, kann ich mich jetzt nicht mehr connecten. Ich vermute, dass ich mir entweder beim Reboot etwas zerstört habe oder bei einem PHP Upgrade.

Es wäre toll, wenn mir jemand dabei helfen könnte, Nextcloud wieder zum Laufen zu bringen.

Die Nextcloud läuft virtualisiert auf Ubuntu 18.04.5 LTS in der aktuellen Version (ich meine V19) auf einem PC.
Als Mysql Server nutze ich MariaDB.
PHP ist 7.4.1

Als Fehler kommt im Webbrowser:

Internal Server Error

The server encountered an internal error and was unable to complete your request.
Please contact the server administrator if this error reappears multiple times, please include the technical details below in your report.
More details can be found in the server log.

Scheinbar ist die Verbindung zu MSQL gestört. Auf den Befehl mysql -uroot -p erhalte ich den Fehler:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")

Im Nextcloud.log steht in den letzten Zeilen:

{"reqId":"xWXkj3rPkUHSsX0HfXM5","level":3,"time":"2020-10-10T21:21:58+00:00","remoteAddr":"84.155.114.65","user":"--","app":"core","method":"GET","url":"/index.php/204","message":{"Exception":"Doctrine\\DBAL\\DBALException","Message":"Failed to connect to the database: An exception occurred in driver: SQLSTATE[HY000] [2002] Connection refused","Code":0,"Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":889,"function":"connect","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":194,"function":"executeQuery","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":200,"function":"executeQuery","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":216,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppConfig.php","line":331,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppConfig.php","line":109,"function":"loadConfigValues","class":"OC\\AppConfig","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppConfig.php","line":287,"function":"getApps","class":"OC\\AppConfig","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_App.php","line":883,"function":"getValues","class":"OC\\AppConfig","type":"->"},{"file":"/var/www/nextcloud/lib/private/Server.php","line":665,"function":"getAppVersions","class":"OC_App","type":"::"},{"file":"/var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php","line":118,"function":"OC\\{closure}","class":"OC\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/ServerContainer.php","line":124,"function":"offsetGet","class":"Pimple\\Container","type":"->"},{"file":"/var/www/nextcloud/lib/private/Server.php","line":1704,"function":"query","class":"OC\\ServerContainer","type":"->"},{"file":"/var/www/nextcloud/lib/private/Server.php","line":856,"function":"getMemCacheFactory","class":"OC\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php","line":118,"function":"OC\\{closure}","class":"OC\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/ServerContainer.php","line":124,"function":"offsetGet","class":"Pimple\\Container","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":163,"function":"query","class":"OC\\ServerContainer","type":"->"},{"file":"/var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php","line":114,"function":"OC\\AppFramework\\Utility\\{closure}","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/ServerContainer.php","line":124,"function":"offsetGet","class":"Pimple\\Container","type":"->"},{"file":"/var/www/nextcloud/lib/private/Server.php","line":1894,"function":"query","class":"OC\\ServerContainer","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_App.php","line":347,"function":"getAppManager","class":"OC\\Server","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_App.php","line":114,"function":"getEnabledApps","class":"OC_App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":648,"function":"loadApps","class":"OC_App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1095,"function":"init","class":"OC","type":"::"},{"file":"/var/www/nextcloud/index.php","line":35,"args":["/var/www/nextcloud/lib/base.php"],"function":"require_once"}],"File":"/var/www/nextcloud/lib/private/DB/Connection.php","Line":67,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Android) Nextcloud-android/3.13.1","version":"19.0.3.1"}
{"reqId":"xWXkj3rPkUHSsX0HfXM5","level":3,"time":"2020-10-10T21:21:58+00:00","remoteAddr":"84.155.114.65","user":"--","app":"core","method":"GET","url":"/index.php/204","message":{"Exception":"Doctrine\\DBAL\\DBALException","Message":"Failed to connect to the database: An exception occurred in driver: SQLSTATE[HY000] [2002] Connection refused","Code":0,"Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":889,"function":"connect","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":194,"function":"executeQuery","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":200,"function":"executeQuery","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":216,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppConfig.php","line":331,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppConfig.php","line":109,"function":"loadConfigValues","class":"OC\\AppConfig","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppConfig.php","line":287,"function":"getApps","class":"OC\\AppConfig","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_App.php","line":883,"function":"getValues","class":"OC\\AppConfig","type":"->"},{"file":"/var/www/nextcloud/lib/private/TemplateLayout.php","line":164,"function":"getAppVersions","class":"OC_App","type":"::"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_Template.php","line":185,"function":"__construct","class":"OC\\TemplateLayout","type":"->"},{"file":"/var/www/nextcloud/lib/private/Template/Base.php","line":132,"function":"fetchPage","class":"OC_Template","type":"->"},{"file":"/var/www/nextcloud/lib/private/legacy/OC_Template.php","line":334,"function":"printPage","class":"OC\\Template\\Base","type":"->"},{"file":"/var/www/nextcloud/index.php","line":63,"function":"printExceptionErrorPage","class":"OC_Template","type":"::"}],"File":"/var/www/nextcloud/lib/private/DB/Connection.php","Line":67,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Android) Nextcloud-android/3.13.1","version":"19.0.3.1"}```

In der config.php steht:
```<?php
$CONFIG = array (
  'instanceid' => '***',
  'passwordsalt' => '***',
  'secret' => '***',
  'trusted_domains' => 
  array (
    0 => '***t.de',
    1 => '10.168.1.26',
    2 => 'localhost',
  ),
  'datadirectory' => '/var/www/nextcloud/data',
  'dbtype' => 'mysql',
  'version' => '19.0.3.1',
  'overwrite.cli.url' => 'http://cloud.***.de',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => '***',
  'dbpassword' => '***',
  'installed' => true,
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 2,
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'mysql.utf8mb4' => true,
  'mail_from_address' => 'info',
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'mail_domain' => '***.de',
  'mail_smtphost' => 'smtp.1und1.de',
  'mail_smtpsecure' => 'ssl',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtpauth' => 1,
  'mail_smtpport' => '587',
  'mail_smtpname' => 'info@***.de',
  'mail_smtppassword' => '***',
  'updater.release.channel' => 'stable',
);```

Der Fehlermeldung nach zu urteilen kann keine Verbindung zur Datenbank hergestellt werden. Schau bitte ob der Datenbankdienst aktiv ist (z.B. im terminal mit “ps axu |grep mariadbd” ).

Was steht in der nextcloud config/config.php unter “dbhost” ?

Befehl: ps axu |grep mariadbd
Ergebnis: mattias 33147 0.0 0.0 14776 972 pts/0 S+ 23:43 0:00 grep --color=auto mariadbd

In der config.php steht unter dbhost: ‘dbhost’ => ‘localhost’,

Auf den Befehl journalctl -u mariadb | tail -20

    Oct 10 23:39:35 IoT_Server systemd[1]: mariadb.service: Failed with result 'signal'.
    Oct 10 23:39:40 IoT_Server systemd[1]: mariadb.service: Service hold-off time over, scheduling restart.
    Oct 10 23:39:40 IoT_Server systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 261.
    Oct 10 23:39:40 IoT_Server systemd[1]: Stopped MariaDB 10.1.44 database server.
    Oct 10 23:39:40 IoT_Server systemd[1]: Starting MariaDB 10.1.44 database server...
    Oct 10 23:39:41 IoT_Server mysqld[14801]: 2020-10-10 23:39:41 140283045592192 [Note] /usr/sbin/mysqld (mysqld 10.1.44-MariaDB-0ubuntu0.18.04.1) starting as process 14801 ...
    Oct 10 23:39:42 IoT_Server /etc/mysql/debian-start[15085]: Upgrading MySQL tables if necessary.
    Oct 10 23:39:42 IoT_Server systemd[1]: Started MariaDB 10.1.44 database server.
    Oct 10 23:39:42 IoT_Server systemd[1]: mariadb.service: Main process exited, code=dumped, status=6/ABRT
    Oct 10 23:39:42 IoT_Server systemd[1]: mariadb.service: Failed with result 'core-dump'.
    Oct 10 23:39:48 IoT_Server systemd[1]: mariadb.service: Service hold-off time over, scheduling restart.
    Oct 10 23:39:48 IoT_Server systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 262.
    Oct 10 23:39:48 IoT_Server systemd[1]: Stopped MariaDB 10.1.44 database server.
    Oct 10 23:39:48 IoT_Server systemd[1]: Starting MariaDB 10.1.44 database server...
    Oct 10 23:39:49 IoT_Server mysqld[15362]: 2020-10-10 23:39:49 139870631705728 [Note] /usr/sbin/mysqld (mysqld 10.1.44-MariaDB-0ubuntu0.18.04.1) starting as process 15362 ...
    Oct 10 23:39:50 IoT_Server /etc/mysql/debian-start[15575]: Upgrading MySQL tables if necessary.
    Oct 10 23:39:50 IoT_Server systemd[1]: Started MariaDB 10.1.44 database server.
    Oct 10 23:39:50 IoT_Server systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
    Oct 10 23:39:50 IoT_Server systemd[1]: mariadb.service: Failed with result 'signal'.```

Der Datenbankserver startet nicht, in den logs dazu scheint die eigentliche Ursache zu fehlen. Eventuell wird diese noch Mal gelogt wenn du manuell den dienst stopst kurz wartest und wieder startest. Es sieht aus als würde der dienst in einer neustart schleife hängen beim Einlesen der Datenbankdateien. Mögliche Ursachen: Datenbankdateien korrupt oder RAM voll.
Genaueres kann man wohl erst sagen wenn ein besserer Fehler im Log auftaucht.

Ich vermute auch, dass ich mir die Datenbank zerschossen habe.
Gibt es eine Möglichkeit aussagefähigere Logs zu erstellen?

Oct 11 00:02:24 IoT_Server systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
Oct 11 00:02:24 IoT_Server systemd[1]: mariadb.service: Failed with result 'signal'.
Oct 11 00:02:29 IoT_Server systemd[1]: mariadb.service: Service hold-off time over, scheduling restart.
Oct 11 00:02:29 IoT_Server systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 428.
Oct 11 00:02:29 IoT_Server systemd[1]: Stopped MariaDB 10.1.44 database server.
Oct 11 00:02:29 IoT_Server systemd[1]: Starting MariaDB 10.1.44 database server...
Oct 11 00:02:30 IoT_Server mysqld[116972]: 2020-10-11  0:02:30 139666364132480 [Note] /usr/sbin/mysqld (mysqld 10.1.44-MariaDB-0ubuntu0.18.04.1) starting as process 116972 ...
Oct 11 00:02:30 IoT_Server /etc/mysql/debian-start[117226]: Upgrading MySQL tables if necessary.
Oct 11 00:02:30 IoT_Server systemd[1]: Started MariaDB 10.1.44 database server.
Oct 11 00:02:31 IoT_Server systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
Oct 11 00:02:31 IoT_Server systemd[1]: mariadb.service: Failed with result 'signal'.
Oct 11 00:02:36 IoT_Server systemd[1]: mariadb.service: Service hold-off time over, scheduling restart.
Oct 11 00:02:36 IoT_Server systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 429.
Oct 11 00:02:36 IoT_Server systemd[1]: Stopped MariaDB 10.1.44 database server.
Oct 11 00:02:36 IoT_Server systemd[1]: Starting MariaDB 10.1.44 database server...
Oct 11 00:02:37 IoT_Server mysqld[117471]: 2020-10-11  0:02:37 139902626724992 [Note] /usr/sbin/mysqld (mysqld 10.1.44-MariaDB-0ubuntu0.18.04.1) starting as process 117471 ...
Oct 11 00:02:37 IoT_Server /etc/mysql/debian-start[117541]: Upgrading MySQL tables if necessary.
Oct 11 00:02:37 IoT_Server systemd[1]: Started MariaDB 10.1.44 database server.
Oct 11 00:02:38 IoT_Server systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
Oct 11 00:02:38 IoT_Server systemd[1]: mariadb.service: Failed with result 'signal'.
mattias@IoT_Server:~$ Oct 11 00:01:40 IoT_Server systemd[1]: Stopped MariaDB 10.1.44 database server.
Oct 11 00:01:40 IoT_Server systemd[1]: Starting MariaDB 10.1.44 database server...
Oct 11 00:01:41 IoT_Server mysqld[113142]: 2020-10-11  0:01:41 139867549441152 [Note] /usr/sbin/mysqld (mysqld 10.1.44-MariaDB-0ubuntu0.18.04.1) starting as process 113142 ...
Oct 11 00:01:41 IoT_Server systemd[1]: Started MariaDB 10.1.44 database server.
Oct 11 00:01:42 IoT_Server systemd[1]: mariadb.service: Main process exited, code=dumped, status=6/ABRT
Oct 11 00:01:42 IoT_Server systemd[1]: mariadb.service: Failed with result 'core-dump'.
Oct 11 00:01:47 IoT_Server systemd[1]: mariadb.service: Service hold-off time over, scheduling restart.
Oct 11 00:01:47 IoT_Server systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 422.
Oct 11 00:01:47 IoT_Server systemd[1]: Stopped MariaDB 10.1.44 database server.
Oct 11 00:01:47 IoT_Server systemd[1]: Starting MariaDB 10.1.44 database server...
Oct 11 00:01:48 IoT_Server mysqld[113697]: 2020-10-11  0:01:48 139827370716288 [Note] /usr/sbin/mysqld (mysqld 10.1.44-MariaDB-0ubuntu0.18.04.1) starting as process 113697 ...
Oct 11 00:01:48 IoT_Server /etc/mysql/debian-start[113756]: Upgrading MySQL tables if necessary.
Oct 11 00:01:48 IoT_Server systemd[1]: Started MariaDB 10.1.44 database server.
Oct 11 00:01:49 IoT_Server systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
Oct 11 00:01:49 IoT_Server systemd[1]: mariadb.service: Failed with result 'signal'.
Oct 11 00:01:54 IoT_Server systemd[1]: mariadb.service: Service hold-off time over, scheduling restart.
Oct 11 00:01:54 IoT_Server systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 423.
Oct 11 00:01:54 IoT_Server systemd[1]: Stopped MariaDB 10.1.44 database server.
Oct 11 00:01:54 IoT_Server systemd[1]: Starting MariaDB 10.1.44 database server...
Oct 11 00:01:55 IoT_Server mysqld[114346]: 2020-10-11  0:01:55 139732972969088 [Note] /usr/sbin/mysqld (mysqld 10.1.44-MariaDB-0ubuntu0.18.04.1) starting as process 114346 ...``

Gibt es noch weitere Logs wie beispielsweise “/var/log/mysql/mysql_error.log” oder Ähnliche?
Mit dem Output bisher kommen wir der Ursache leider nicht bei.
Edit: https://mariadb.com/kb/en/error-log/

Ich habe mal die letzten Zeilen eingefügt, aus denen zumindest ein Crash sichtbar wird.

key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=0
max_threads=153
thread_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 352468 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55eb1dfa270e]
/usr/sbin/mysqld(handle_fatal_signal+0x53b)[0x55eb1db66d7b]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x128a0)[0x7f54b60388a0]
2020-10-11  0:12:35 140001825590400 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.46-86.2 started; log sequence number 85788850134
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7f54b52c8f47]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7f54b52ca8b1]
2020-10-11  0:12:35 140001157314304 [Note] InnoDB: Dumping buffer pool(s) not yet started
2020-10-11  0:12:35 140001825590400 [Note] Plugin 'FEEDBACK' is disabled.
2020-10-11  0:12:35 140001825590400 [Note] Recovering after a crash using tc.log
2020-10-11  0:12:35 140001825590400 [Note] Starting crash recovery...
2020-10-11  0:12:35 140001825590400 [Note] Crash recovery finished.
2020-10-11  0:12:35 140001825590400 [Note] Server socket created on IP: '0.0.0.0'.
/usr/sbin/mysqld(+0x367be7)[0x55eb1d935be7]
/usr/sbin/mysqld(+0x8b6978)[0x55eb1de84978]
2020-10-11  0:12:35 140001825590400 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.1.44-MariaDB-0ubuntu0.18.04.1'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  Ubuntu 18.04
/usr/sbin/mysqld(+0x8b6d20)[0x55eb1de84d20]
/usr/sbin/mysqld(+0x8b847e)[0x55eb1de8647e]
/usr/sbin/mysqld(+0x8a97b6)[0x55eb1de777b6]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)[0x7f54b602d6db]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f54b53aba3f]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.```

Vielleicht kann jemand aus dem backlog einen Fehler erkennen, dafür reicht mein Wissen leider nicht aus. Alternativ: hier gibts ein paar infos wie du über die verbose option vielleicht mehr infos bekommst: https://mariadb.com/kb/en/what-to-do-if-mariadb-doesnt-start/

Danke Dir, dass Du Dich der Sache bis hierhin angenommen hast!

Gerne, viel Erfolg beim Lösen - ich hoffe du hast noch eine Snapshot oder Backup eines guten Zustands. Normalerweise wirft mariadb mit hoher verbose einstellung fürs error log was brauchbares raus. Wenn du da nicht weiterkommst hilft wohl wirklich nur zurück zum letzten funktionierenden Punkt.