NextCloud 1.52.0 - Maria DB down

luke81 · May 12, 2023, 2:14pm

Hello,
I ve an issue, the mari adb is still down even after restart of the system.
Below the report.
Any help will be helpfull.
Thanks.
Luc

NextcloudPi diagnostics


NextcloudPi version  v1.52.0
NextcloudPi image    NextCloudPi_11-27-20
OS                   Debian GNU/Linux 10. 5.10.103-v8+ (aarch64)
automount            yes
USB devices          sda 
datadir              /media/ea8c30f8-7e14-49ac-90c4-dd756041387a/var/ncdata
data in SD           no
data filesystem      btrfs
data disk usage      14G/120G
rootfs usage         2.8G/29G
swapfile             /var/swap
dbdir                /media/ea8c30f8-7e14-49ac-90c4-dd756041387a/var/ncdatabase
Nextcloud check      error
HTTPD service        up
PHP service          up
MariaDB service      down
Redis service        up
HPB service          down
Postfix service      up
Internet check       ok
Public IPv4          ***REMOVED SENSITIVE VALUE***
Public IPv6          ***REMOVED SENSITIVE VALUE***
Port 80              open
Port 443             open
IP                   ***REMOVED SENSITIVE VALUE***
Gateway              ***REMOVED SENSITIVE VALUE***
Interface            eth0
Certificates         ***REMOVED SENSITIVE VALUE***
NAT loopback         no
Uptime               2min

Nextcloud configuration


An unhandled exception has been thrown:
Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2002] No such file or directory in /var/www/nextcloud/lib/private/DB/Connection.php:87
Stack trace:
#0 /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php(1519): OC\DB\Connection->connect()
#1 /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php(1041): Doctrine\DBAL\Connection->getWrappedConnection()
#2 /var/www/nextcloud/lib/private/DB/Connection.php(237): Doctrine\DBAL\Connection->executeQuery('SELECT * FROM `...', Array, Array, NULL)
#3 /var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php(345): OC\DB\Connection->executeQuery('SELECT * FROM `...', Array, Array)
#4 /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php(287): Doctrine\DBAL\Query\QueryBuilder->execute()
#5 /var/www/nextcloud/lib/private/AppConfig.php(361): OC\DB\QueryBuilder\QueryBuilder->execute()
#6 /var/www/nextcloud/lib/private/AppConfig.php(126): OC\AppConfig->loadConfigValues()
#7 /var/www/nextcloud/lib/private/AppConfig.php(317): OC\AppConfig->getApps()
#8 /var/www/nextcloud/lib/private/legacy/OC_App.php(982): OC\AppConfig->getValues(false, 'installed_versi...')
#9 /var/www/nextcloud/lib/private/Server.php(692): OC_App::getAppVersions()
#10 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(160): OC\Server->OC\{closure}(Object(OC\Server))
#11 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#12 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(127): Pimple\Container->offsetGet('OC\\Memcache\\Fac...')
#13 /var/www/nextcloud/lib/private/ServerContainer.php(136): OC\AppFramework\Utility\SimpleContainer->query('OC\\Memcache\\Fac...', false)
#14 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query('OC\\Memcache\\Fac...')
#15 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(182): OC\AppFramework\Utility\SimpleContainer->get('OC\\Memcache\\Fac...')
#16 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(160): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(OC\Server))
#17 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(114): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#18 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(127): Pimple\Container->offsetGet('OCP\\ICacheFacto...')
#19 /var/www/nextcloud/lib/private/ServerContainer.php(136): OC\AppFramework\Utility\SimpleContainer->query('OCP\\ICacheFacto...', false)
#20 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query('OCP\\ICacheFacto...')
#21 /var/www/nextcloud/lib/private/Server.php(1079): OC\AppFramework\Utility\SimpleContainer->get('OCP\\ICacheFacto...')
#22 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(160): OC\Server->OC\{closure}(Object(OC\Server))
#23 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#24 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(127): Pimple\Container->offsetGet('OCP\\Lock\\ILocki...')
#25 /var/www/nextcloud/lib/private/ServerContainer.php(136): OC\AppFramework\Utility\SimpleContainer->query('OCP\\Lock\\ILocki...', false)
#26 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query('OCP\\Lock\\ILocki...')
#27 /var/www/nextcloud/lib/private/Server.php(2040): OC\AppFramework\Utility\SimpleContainer->get('OCP\\Lock\\ILocki...')
#28 /var/www/nextcloud/lib/private/Files/View.php(118): OC\Server->getLockingProvider()
#29 /var/www/nextcloud/lib/private/Server.php(439): OC\Files\View->__construct()
#30 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(160): OC\Server->OC\{closure}(Object(OC\Server))
#31 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}(Object(Pimple\Container))
#32 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(127): Pimple\Container->offsetGet('OC\\Files\\Node\\H...')
#33 /var/www/nextcloud/lib/private/ServerContainer.php(136): OC\AppFramework\Utility\SimpleContainer->query('OC\\Files\\Node\\H...', false)
#34 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query('OC\\Files\\Node\\H...')
#35 /var/www/nextcloud/lib/private/Server.php(1403): OC\AppFramework\Utility\SimpleContainer->get('OC\\Files\\Node\\H...')
#36 /var/www/nextcloud/lib/base.php(594): OC\Server->boot()
#37 /var/www/nextcloud/lib/base.php(1089): OC::init()
#38 /var/www/nextcloud/console.php(48): require_once('/var/www/nextcl...')
#39 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#40 {main}

bb77 · May 12, 2023, 2:42pm

Whats the output of…

sudo journalctl -b --unit=mariadb

…and:

sudo cat /var/log/mysql/mysql_error.log

To display only messages that are classified as errors, you can use:

sudo cat /var/log/mysql/mysql_error.log | grep 'ERROR'

luke81 · May 12, 2023, 2:48pm

– Logs begin at Fri 2023-05-12 16:30:39 CEST, end at Fri 2023-05-12 16:47:42 CEST. –
May 12 16:31:07 nextcloudpi systemd[1]: Starting MariaDB 10.3.38 database server…
May 12 16:31:29 nextcloudpi systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
May 12 16:31:29 nextcloudpi systemd[1]: mariadb.service: Failed with result ‘signal’.
May 12 16:31:29 nextcloudpi systemd[1]: Failed to start MariaDB 10.3.38 database server.
May 12 16:31:35 nextcloudpi systemd[1]: mariadb.service: Service RestartSec=5s expired, scheduling restart.
May 12 16:31:35 nextcloudpi systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 1.
May 12 16:31:35 nextcloudpi systemd[1]: Stopped MariaDB 10.3.38 database server.
May 12 16:31:35 nextcloudpi systemd[1]: Starting MariaDB 10.3.38 database server…
May 12 16:31:55 nextcloudpi systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
May 12 16:31:55 nextcloudpi systemd[1]: mariadb.service: Failed with result ‘signal’.
May 12 16:31:55 nextcloudpi systemd[1]: Failed to start MariaDB 10.3.38 database server.
May 12 16:32:01 nextcloudpi systemd[1]: mariadb.service: Service RestartSec=5s expired, scheduling restart.
May 12 16:32:01 nextcloudpi systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 2.
May 12 16:32:01 nextcloudpi systemd[1]: Stopped MariaDB 10.3.38 database server.
May 12 16:32:01 nextcloudpi systemd[1]: Starting MariaDB 10.3.38 database server…
May 12 16:32:22 nextcloudpi systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
May 12 16:32:22 nextcloudpi systemd[1]: mariadb.service: Failed with result ‘signal’.
May 12 16:32:22 nextcloudpi systemd[1]: Failed to start MariaDB 10.3.38 database server.
May 12 16:32:27 nextcloudpi systemd[1]: mariadb.service: Service RestartSec=5s expired, scheduling restart.
May 12 16:32:27 nextcloudpi systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 3.
May 12 16:32:27 nextcloudpi systemd[1]: Stopped MariaDB 10.3.38 database server.
May 12 16:32:27 nextcloudpi systemd[1]: Starting MariaDB 10.3.38 database server…
May 12 16:32:48 nextcloudpi systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
May 12 16:32:48 nextcloudpi systemd[1]: mariadb.service: Failed with result ‘signal’.
May 12 16:32:48 nextcloudpi systemd[1]: Failed to start MariaDB 10.3.38 database server.
May 12 16:32:53 nextcloudpi systemd[1]: mariadb.service: Service RestartSec=5s expired, scheduling restart.
May 12 16:32:53 nextcloudpi systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 4.
May 12 16:32:53 nextcloudpi systemd[1]: Stopped MariaDB 10.3.38 database server.
May 12 16:32:53 nextcloudpi systemd[1]: Starting MariaDB 10.3.38 database server…
May 12 16:33:14 nextcloudpi systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
May 12 16:33:14 nextcloudpi systemd[1]: mariadb.service: Failed with result ‘signal’.
May 12 16:33:14 nextcloudpi systemd[1]: Failed to start MariaDB 10.3.38 database server.
May 12 16:33:19 nextcloudpi systemd[1]: mariadb.service: Service RestartSec=5s expired, scheduling restart.
May 12 16:33:19 nextcloudpi systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 5.
May 12 16:33:19 nextcloudpi systemd[1]: Stopped MariaDB 10.3.38 database server.
May 12 16:33:19 nextcloudpi systemd[1]: Starting MariaDB 10.3.38 database server…
May 12 16:33:40 nextcloudpi systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
May 12 16:33:40 nextcloudpi systemd[1]: mariadb.service: Failed with result ‘signal’.
May 12 16:33:40 nextcloudpi systemd[1]: Failed to start MariaDB 10.3.38 database server.
May 12 16:33:45 nextcloudpi systemd[1]: mariadb.service: Service RestartSec=5s expired, scheduling restart.
May 12 16:33:45 nextcloudpi systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 6.
May 12 16:33:45 nextcloudpi systemd[1]: Stopped MariaDB 10.3.38 database server.
May 12 16:33:45 nextcloudpi systemd[1]: Starting MariaDB 10.3.38 database server…
May 12 16:34:06 nextcloudpi systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
May 12 16:34:06 nextcloudpi systemd[1]: mariadb.service: Failed with result ‘signal’.
May 12 16:34:06 nextcloudpi systemd[1]: Failed to start MariaDB 10.3.38 database server.
May 12 16:34:11 nextcloudpi systemd[1]: mariadb.service: Service RestartSec=5s expired, scheduling restart.
May 12 16:34:11 nextcloudpi systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 7.
May 12 16:34:11 nextcloudpi systemd[1]: Stopped MariaDB 10.3.38 database server.
May 12 16:34:11 nextcloudpi systemd[1]: Starting MariaDB 10.3.38 database server…
May 12 16:34:32 nextcloudpi systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
May 12 16:34:32 nextcloudpi systemd[1]: mariadb.service: Failed with result ‘signal’.
May 12 16:34:32 nextcloudpi systemd[1]: Failed to start MariaDB 10.3.38 database server.
lines 1-54

luke81 · May 12, 2023, 2:53pm

pi@nextcloudpi:~ $ sudo cat /var/log/mysql/mysql_error.log
cat: /var/log/mysql/mysql_error.log: No such file or directory
pi@nextcloudpi:~ $ sudo cat /var/log/mysql/mysql_error.log | grep ‘ERROR’
cat: /var/log/mysql/mysql_error.log: No such file or directory
pi@nextcloudpi:~ $

bb77 · May 12, 2023, 3:45pm

The following command should tell you the correct path to the error log

grep -R log_error /etc/mysql/*

luke81 · May 12, 2023, 4:01pm

This is the report for sudo cat /var/log/mysql/error.log
Envoi en cours : Mysql_error.pdf…

luke81 · May 12, 2023, 4:09pm

152, but was only able to read 262144
2023-05-12 8:55:21 0 [ERROR] InnoDB: Operating system error number 5 in a file operation.
2023-05-12 8:55:21 0 [ERROR] InnoDB: Error number 5 means ‘Input/output error’
2023-05-12 8:55:21 0 [ERROR] InnoDB: File (unknown): ‘read’ returned OS error 205. Cannot continue operation
230512 8:55:21 [ERROR] mysqld got signal 6 ;
2023-05-12 8:55:47 0 [ERROR] InnoDB: Tried to read 1048576 bytes at offset 2097152, but was only able to read 262144
2023-05-12 8:55:47 0 [ERROR] InnoDB: Operating system error number 5 in a file operation.
2023-05-12 8:55:47 0 [ERROR] InnoDB: Error number 5 means ‘Input/output error’
2023-05-12 8:55:47 0 [ERROR] InnoDB: File (unknown): ‘read’ returned OS error 205. Cannot continue operation
230512 8:55:47 [ERROR] mysqld got signal 6 ;
2023-05-12 8:56:14 0 [ERROR] InnoDB: Tried to read 1048576 bytes at offset 2097152, but was only able to read 262144
2023-05-12 8:56:14 0 [ERROR] InnoDB: Operating system error number 5 in a file operation.
2023-05-12 8:56:14 0 [ERROR] InnoDB: Error number 5 means ‘Input/output error’
2023-05-12 8:56:14 0 [ERROR] InnoDB: File (unknown): ‘read’ returned OS error 205. Cannot continue operation

bb77 · May 12, 2023, 4:26pm

I don’t want to call the worst case yet. But these error messages could indicate a failing disk. Are you running NextcloudPi on an actual RaspberryPi, on an SD card?

Maybe you could post the same part of the logs again, including all the messages between the [ERROR] messages

sudo cat /var/log/mysql/error.log

(without grepping the ‘ERROR’ messages)

luke81 · May 12, 2023, 4:34pm

2023-05-12 17:50:51 0 [Warning] InnoDB: 1048576 bytes should have been read. Only 262144 bytes read. Retrying for the remaining bytes.
2023-05-12 17:50:51 0 [Warning] InnoDB: Retry attempts for reading partial data failed.
2023-05-12 17:50:51 0 [ERROR] InnoDB: Tried to read 1048576 bytes at offset 2097152, but was only able to read 262144
2023-05-12 17:50:51 0 [ERROR] InnoDB: Operating system error number 5 in a file operation.
2023-05-12 17:50:51 0 [ERROR] InnoDB: Error number 5 means ‘Input/output error’
2023-05-12 17:50:51 0 [Note] InnoDB: Some operating system error numbers are described at Operating System Error Codes - MariaDB Knowledge Base
2023-05-12 17:50:51 0 [ERROR] InnoDB: File (unknown): ‘read’ returned OS error 205. Cannot continue operation
230512 17:50:51 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

luke81 · May 12, 2023, 4:35pm

In fact I don’t know if it’s running on sd card.

luke81 · May 12, 2023, 4:49pm

max_used_connections=0
max_threads=153
thread_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467441 K bytes of memory
Hope that’s ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong…
stack_bottom = 0x0 thread_stack 0x49000
/usr/sbin/mysqld(my_print_stacktrace+0x30)[0x558c66cbc8]
/usr/sbin/mysqld(handle_fatal_signal+0x510)[0x558c1d0338]
addr2line: ‘linux-vdso.so.1’: No such file
linux-vdso.so.1(__kernel_rt_sigreturn+0x0)[0x7faf7977d8]
linux/raise.c:51(__GI_raise)[0x7faf1b4714]
stdlib/abort.c:81(__GI_abort)[0x7faf1a28e8]
/usr/sbin/mysqld(+0x909a8c)[0x558c399a8c]
/usr/sbin/mysqld(+0x90c6a8)[0x558c39c6a8]
/usr/sbin/mysqld(+0x9fb3f0)[0x558c48b3f0]
/usr/sbin/mysqld(+0xa664a8)[0x558c4f64a8]
/usr/sbin/mysqld(+0xa67128)[0x558c4f7128]
/usr/sbin/mysqld(+0x4d8894)[0x558bf68894]
/usr/sbin/mysqld(+0x89cf88)[0x558c32cf88]
/usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x64)[0x558c1d2884]
/usr/sbin/mysqld(+0x5a85f8)[0x558c0385f8]
/usr/sbin/mysqld(_Z11plugin_initPiPPci+0x694)[0x558c0393b4]
/usr/sbin/mysqld(+0x4f8c90)[0x558bf88c90]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x400)[0x558bf8f628]
csu/libc-start.c:342(__libc_start_main)[0x7faf1a2d24]
/usr/sbin/mysqld(+0x4f2de0)[0x558bf82de0]
The manual page at How to Produce a Full Stack Trace for mysqld - MariaDB Knowledge Base contains
information that should help you find out what is causing the crash.
Writing a core file…
Working directory at /media/ea8c30f8-7e14-49ac-90c4-dd756041387a/var/ncdatabase
Resource Limits:
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 13885 13885 processes
Max open files 32768 32768 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 13885 13885 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Core pattern: core

Kernel version: Linux version 5.10.103-v8+ (dom@buildbot) (aarch64-linux-gnu-gcc-8 (Ubuntu/Linaro 8.4.0-3ubuntu1) 8.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #1529 SMP PREEMPT Tue Mar 8 12:26:46 GMT 2022

2023-05-12 17:49:06 0 [Note] Starting MariaDB 10.3.38-MariaDB-0+deb10u1 source revision c73985f2ce8a391582787f3e310a011c1a712bec as process 331
2023-05-12 17:49:07 0 [Warning] The parameter innodb_file_format is deprecated and has no effect. It may be removed in future releases. See InnoDB File Format - MariaDB Knowledge Base
2023-05-12 17:49:07 0 [Note] InnoDB: Using Linux native AIO
2023-05-12 17:49:07 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2023-05-12 17:49:07 0 [Note] InnoDB: Uses event mutexes
2023-05-12 17:49:07 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2023-05-12 17:49:07 0 [Note] InnoDB: Number of pools: 1
2023-05-12 17:49:07 0 [Note] InnoDB: Using generic crc32 instructions
2023-05-12 17:49:07 0 [Note] InnoDB: Initializing buffer pool, total size = 1G, instances = 1, chunk size = 128M
2023-05-12 17:49:07 0 [Note] InnoDB: Completed initialization of buffer pool
2023-05-12 17:49:07 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2023-05-12 17:49:07 0 [Warning] InnoDB: 1048576 bytes should have been read. Only 262144 bytes read. Retrying for the remaining bytes.
2023-05-12 17:49:07 0 [Warning] InnoDB: Retry attempts for reading partial data failed.
2023-05-12 17:49:07 0 [ERROR] InnoDB: Tried to read 1048576 bytes at offset 2097152, but was only able to read 262144
2023-05-12 17:49:07 0 [ERROR] InnoDB: Operating system error number 5 in a file operation.
2023-05-12 17:49:07 0 [ERROR] InnoDB: Error number 5 means ‘Input/output error’
2023-05-12 17:49:07 0 [Note] InnoDB: Some operating system error numbers are described at Operating System Error Codes - MariaDB Knowledge Base
2023-05-12 17:49:07 0 [ERROR] InnoDB: File (unknown): ‘read’ returned OS error 205. Cannot continue operation
230512 17:49:07 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

bb77 · May 12, 2023, 5:00pm

I don’t use Nextcloud-Pi l, so I don’t know, but If I had to guess, I would say it’s running on the SD-CARD. Either way, this doesn’t sound good, and external disks like HDDs or SSDs can of course fail too.

What I would do:

Backup all your files, either to another disk or to your PC!
try to boot the SD_CARD on another Raspi (if available).
if the error also occurs on the other Raspi, install NCP on a new SD-CARD, and restore your backup (if available).

Or maybe others here in the forum have some more ideas…