Upgraded Maria DB for Nextcloud 21, now mysql-server will not start

Nextcloud version (eg, 20.0.5): 21.0.1
Operating system and version (eg, Ubuntu 20.04): 11.2-STABLE FreeBSD 11.2-STABLE #0 r325575+4710c8b6420(HEAD): Fri Feb 14 13:59:19 UTC 2020 root@tnbuild02.tn.ixsystems.com:/freenas-releng/freenas/_BE/objs/freenas-releng/freenas/_BE/os/sys/FreeNAS.amd64 amd64
Apache or nginx version (eg, Apache 2.4.25): Apache/2.4.46 (FreeBSD)
PHP version (eg, 7.4): PHP 7.3.28

The issue you are facing: upgraded MariaDB from 10.2 to 10.5 and now unable to start mysql-server
What’s really annoying is no logs generate in /var/log/mysql/mysqlerr.log

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

  1. Upgrade to NC 20; see warning about MariaDB version requirement
  2. Upgrade MariaDB to 10.5
  3. Restart services and NC gets Internal Server Error, not producing logs, and mysql-server will not start, also not producing logs.

The output of your Nextcloud log in Admin > Logging: unable to get to admin>logging as the whole site is down due to Internal Server Error
/var/log/nextcloud.log is empty

PASTE HERE

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

  GNU nano 5.6.1                                                                                                       /usr/local/www/apache24/data/nextcloud/config/config.php
<?php
$CONFIG = array (
  'instanceid' => 'XXXXXXXXXX',
  'passwordsalt' => 'XXXXXXXXXXXXXXXXXXXXXXXXXX',
  'secret' => 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
  'trusted_domains' =>
  array (
    0 => 'cloudp01',
    1 => 'myserver.xyz',
    2 => 'myserver.123',
  ),
  'datadirectory' => '/media/files',
  'overwrite.cli.url' => 'http://cloudp01',
  'dbtype' => 'mysql',
  'version' => '21.0.1.1',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost:/tmp/mysql.sock',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'nextcloud_admin',
  'dbpassword' => 'XXXXXXXXXXXXXXXXX',
  'log_type' => 'owncloud',
  'logfile' => '/var/log/nextcloud.log',
  'loglevel' => '3',
  'logdateformat' => 'F d, Y H:i:s',
  'installed' => true,
  'maintenance' => false,
  'theme' => '',
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'tls',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_from_address' => 'alerts',
  'mail_domain' => 'mymailserver.xyz',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'smtp.office365.com',
  'mail_smtpport' => '587',
  'mail_smtpname' => 'alerts@whatever.com',
  'mail_smtppassword' => 'XXXXXXXXXXXXXXXXXXX',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'mysql.utf8mb4' => true,
  'updater.release.channel' => 'stable',
  'updater.secret' => 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
);


The output of your Apache/nginx/system log in /var/log/____:

[Fri May 07 16:47:19.800896 2021] [mpm_prefork:notice] [pid 94296] AH00169: caught SIGTERM, shutting down
[Fri May 07 16:47:46.986208 2021] [ssl:warn] [pid 97785] AH01882: Init: this version of mod_ssl was compiled against a newer library (OpenSSL 1.0.2u-freebsd  20 Dec 2019, version currently loaded is OpenSSL 1.0.2s-freebsd  28 May 2019) - may result in undefined or erroneous behavior
[Fri May 07 16:47:47.007295 2021] [ssl:warn] [pid 97786] AH01882: Init: this version of mod_ssl was compiled against a newer library (OpenSSL 1.0.2u-freebsd  20 Dec 2019, version currently loaded is OpenSSL 1.0.2s-freebsd  28 May 2019) - may result in undefined or erroneous behavior
[Fri May 07 16:47:47.007406 2021] [ssl:warn] [pid 97786] AH01873: Init: Session Cache is not configured [hint: SSLSessionCache]
[Fri May 07 16:47:47.064655 2021] [mpm_prefork:notice] [pid 97786] AH00163: Apache/2.4.46 (FreeBSD) OpenSSL/1.0.2s-freebsd PHP/7.3.28 configured -- resuming normal operations
[Fri May 07 16:47:47.064717 2021] [core:notice] [pid 97786] AH00094: Command line: '/usr/local/sbin/httpd -D NOHTTPACCEPT'

root@CLOUDP01:/var/log # service mysql-server start && service mysql-server status && cat /var/log/mysql/mysqld.err
Starting mysql.
mysql is not running.
root@CLOUDP01:/var/log # 

Well… I figured it out:

  1. MariaDB 10.5 does not like my.cnf being in /usr/local/etc
  2. MariaDB 10.5 also does not like ‘innodb_file_format_max=barracuda’
  3. When upgrading to MariaDB 10.5 my.cnf needs to be in /usr/local/etc/mysql/my.cnf
  4. You also need to check that /usr/local/etc/mysql/conf.d/server.cnf has relevant parts uncommented.
  5. NCInstall/config/config.php needs to have dbhost updated to ‘localhost:/var/run/mysql/mysql.sock’ instead of ‘localhost:/tmp/mysql.sock’

After making all of those changes, I am seeing proper logs, NC is running and my MariaDB is updated and running.

I hope if someone else has this issue on a FreeBSD Jail/FreeNAS gets answers they need from this post because it took some stubborn trial and error to figure out because literally none of that is documented anywhere for upgrading from pre NC 18.x/MariaDB 10.1