MariaDB Issue at Boot

I’ve succssfully run/maintained an ownCloud instance for several years. Recently, I’ve been experimenting with NextCloud on a RaspberryPi Zero W with some external storage attached.

So far so good… until recently, MariaDB is no longer starting correctlly at boot. When I check the service with ‘systemctl status mysql’ I see the following output:

● mariadb.service - MariaDB database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2018-03-22 13:28:49 UTC; 8min ago
  Process: 781 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=1/FAILURE)
  Process: 705 ExecStartPre=/bin/sleep 10 (code=exited, status=0/SUCCESS)
  Process: 435 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP_S
  Process: 397 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
  Process: 364 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS)
 Main PID: 781 (code=exited, status=1/FAILURE)
   Status: "MariaDB server is down"

Mar 22 13:28:27 nextcloudpi systemd[1]: Starting MariaDB database server...
Mar 22 13:28:49 nextcloudpi mysqld[781]: 2018-03-22 13:28:49 3062385376 [Note] /usr/sbin/mysqld (mysqld 10.1.23-MariaDB-9+deb9u1) starting as process 781 ...
Mar 22 13:28:49 nextcloudpi mysqld[781]: 2018-03-22 13:28:49 3062385376 [Warning] Can't create test file /media/InvisibleDrive/ncdatabase/nextcloudpi.lower-test
Mar 22 13:28:49 nextcloudpi mysqld[781]: [108B blob data]
Mar 22 13:28:49 nextcloudpi mysqld[781]: 2018-03-22 13:28:49 3062385376 [ERROR] Aborting
Mar 22 13:28:49 nextcloudpi systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Mar 22 13:28:49 nextcloudpi systemd[1]: Failed to start MariaDB database server.
Mar 22 13:28:49 nextcloudpi systemd[1]: mariadb.service: Unit entered failed state.
Mar 22 13:28:49 nextcloudpi systemd[1]: mariadb.service: Failed with result 'exit-code'.
lines 1-20/20 (END)

I’m able to start the service without issue from the command line:

sudo systemctl start mysql.service

systemctl status mysql
● mariadb.service - MariaDB database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2018-03-22 13:25:01 UTC; 17s ago
  Process: 17392 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
  Process: 17390 ExecStartPost=/etc/mysql/debian-start (code=exited, status=0/SUCCESS)
  Process: 17321 ExecStartPre=/bin/sleep 10 (code=exited, status=0/SUCCESS)
  Process: 17223 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP
  Process: 17219 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
  Process: 17215 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS)
 Main PID: 17352 (mysqld)
   Status: "Taking your SQL requests now..."
   CGroup: /system.slice/mariadb.service
           └─17352 /usr/sbin/mysqld

Mar 22 13:24:46 nextcloudpi systemd[1]: Starting MariaDB database server...
Mar 22 13:24:58 nextcloudpi mysqld[17352]: 2018-03-22 13:24:58 3062459104 [Note] /usr/sbin/mysqld (mysqld 10.1.23-MariaDB-9+deb9u1) starting as process 17352 ...
Mar 22 13:25:01 nextcloudpi systemd[1]: Started MariaDB database server.
lines 1-17/17 (END)

Any thoughts on what’s happening and how to remedy the issue would be very welcome. This little machine isn’t on all the time, and I’d rather not b SSHing in to start the DB manually everytime it boots.

Nextcloud version : 12.0.4.3
Operating system and version : Linux nextcloudpi 4.9.66+ (So, basically up to date Raspbian)
Apache or nginx version : Apache/2.4.25
PHP version : PHP 7.0.27-0+deb9u1

Nextcloud log:

*This looks like a huge bunch of garbage. '…/data/nextcloud.log' is what we're talking about? With all the queries and connections?

config.php file:

<?php
$CONFIG = array (
  'passwordsalt' => 'xxxxxxx',
  'secret' => 'xxxxxxx',
  'trusted_domains' => 
  array (
    0 => 'localhost',
    5 => 'nextcloudpi.local',
    1 => '192.168.1.10',
    3 => 'xxxxxxx.net',
  ),
  'datadirectory' => '/media/MyDrive/ncdata',
  'overwrite.cli.url' => 'https://xxxxxxx.net',
  'dbtype' => 'mysql',
  'version' => '12.0.4.3',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'ncadmin',
  'dbpassword' => 'xxxxxxx',
  'installed' => true,
'instanceid' => 'xxxxxxx',
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => '/var/run/redis/redis.sock',
    'port' => 0,
    'timeout' => 0.0,
    'password' => 'xxxxxxx',
  ),
  'mail_smtpmode' => 'php',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_from_address' => 'xxxxxxx',
  'mail_domain' => 'xxxxxxx',
  'overwriteprotocol' => 'https',
  'maintenance' => false,
  'loglevel' => '2',
  'log_type' => 'file',
);

The output of your Apache/nginx/system log in /var/log/____:

[Tue Mar 27 06:25:04.890198 2018] [ssl:warn] [pid 452:tid 3066305632] AH01909: localhost:4443:0 server certificate does NOT include an ID which matches the server name
[Tue Mar 27 06:25:05.000622 2018] [mpm_event:notice] [pid 452:tid 3066305632] AH00489: Apache/2.4.25 (Raspbian) OpenSSL/1.0.2l configured -- resuming normal operations
[Tue Mar 27 06:25:05.000873 2018] [core:notice] [pid 452:tid 3066305632] AH00094: Command line: '/usr/sbin/apache2'

Mysqld starts before the hdd is successfully mounted. You need to delay the start. Maybe you can define a dependency, that makes sure mysqld starts after the successful mount first.

Not an issue with NC therefore :slight_smile:

1 Like

Thanks for the swift response. Can you help me to understand where you’re seeing evidence that the HDD hasn’t yet been mounted when mysqld starts?

This issue has only recently crept up, and I’ve not made any serious changes to the system, other than recently installing emacs.