Can't import database to Docker container

Arthur_D · May 28, 2023, 2:25pm

Hi, I have been migrating my old, cobbled together Docker compose installation of Nextcloud from Ubuntu 18.04 to Ubuntu 22.04. However, following the official migration instructions leads to

sudo docker compose exec nextcloud-mariadb sh -c “mysql --user nextcloud --password nextcloud < /dmp”
Enter password:
ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ‘error: Found option without preceding group in config file: /etc/mysql/conf.d…’ at line 1

However, when I exec into the nextcloud-mariadb container and check /etc/mysql/conf.d it’s an empty directory:

sudo docker exec -it nextcloud-mariadb /bin/bash
root@aa7130f66f45:/# ls -l /etc/mysql/conf.d/
total 0

So how can I have an error in my SQL syntax from a config file that doesn’t exist? I am quite confused.

jtr · May 28, 2023, 2:49pm

This threw me for a moment than I looked closer. The part in single quotes is the literal string it’s erroring on. My guess is your /dmp contains that literal string in it (you can view it with more/less - it’s just a text file). If so, something went wrong with your SQL export (mysqldump). The /etc/msql/conf.d probably exists on your old host and mysqldump failed there with an error rather so your exported dump file just contains whatever the error was.

Arthur_D · May 28, 2023, 3:33pm

Thank you so, so much! Apparently, looking at /dmp with more (less is not installed), the first line is indeed

error: Found option without preceding group in config file: /etc/mysql/conf.d/ncp-mysql.cnf at line: 14

However, it does seem like the rest of the dump went through, starting with

– MariaDB dump 10.19 Distrib 10.5.19-MariaDB, for debian-linux-gnu (x86_64)

– Host: localhost Database: nextcloud

– Server version 10.5.19-MariaDB-1:10.5.19+maria~ubu2004

/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT /;
/!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS /;
/!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION /;
/!40101 SET NAMES utf8mb4 /;
/!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE /;
/!40103 SET TIME_ZONE=‘+00:00’ /;
/!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 /;
/!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 /;
/!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE=‘NO_AUTO_VALUE_ON_ZERO’ /;
/!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;

–
– Table structure for table oc_accounts
etc. So would it be a solution to just delete the first line with the error message? And how would I best do that? I suspect I don’t have any text editor installed in the container.

Arthur_D · May 28, 2023, 4:06pm

I’m stupid, I didn’t remember I could just edit the database before copying it to /dmp. Did that and it seemed to have worked, however I’m still trying to figure out why I get
Configuration was not read or initialized correctly, not overwriting /var/www/html/config/config.php
but maybe that’s because of permissions somehow? Idk, need to check the logs.

Edit: After exec into the nextcloud container, it seems like the folders aren’t owned properly by the container:

root@38d3cb91c9f8:/var/www/html# ls -lh
total 164K
drwxr-xr-x 45 www-data www-data 4.0K May 27 16:17 3rdparty
-rw-r–r-- 1 www-data www-data 19K May 27 16:17 AUTHORS
-rw-r–r-- 1 www-data www-data 34K May 27 16:17 COPYING
drwxr-xr-x 50 www-data www-data 4.0K May 27 16:17 apps
drwxrwxr-x 2 1000 1000 4.0K May 26 16:34 config
-rw-r–r-- 1 www-data www-data 4.0K May 27 16:17 console.php
drwxr-xr-x 24 www-data www-data 4.0K May 27 16:17 core
-rw-r–r-- 1 www-data www-data 6.2K May 27 16:17 cron.php
drwxrwxr-x 30 1000 1000 4.0K May 26 15:40 custom_apps
drwxrwxr-x 14 1000 1000 4.0K May 27 13:46 data
drwxr-xr-x 2 www-data www-data 12K May 27 16:17 dist
-rw-r–r-- 1 www-data www-data 156 May 27 16:17 index.html
-rw-r–r-- 1 www-data www-data 3.4K May 27 16:17 index.php
drwxr-xr-x 6 www-data www-data 4.0K May 27 16:17 lib
-rw-r–r-- 1 root root 0 May 28 18:01 nextcloud-init-sync.lock
-rwxr-xr-x 1 www-data www-data 283 May 27 16:17 occ
drwxr-xr-x 2 www-data www-data 4.0K May 27 16:17 ocm-provider
drwxr-xr-x 2 www-data www-data 4.0K May 27 16:17 ocs
drwxr-xr-x 2 www-data www-data 4.0K May 27 16:17 ocs-provider
-rw-r–r-- 1 www-data www-data 3.2K May 27 16:17 public.php
-rw-r–r-- 1 www-data www-data 5.5K May 27 16:17 remote.php
drwxr-xr-x 4 www-data www-data 4.0K May 27 16:17 resources
-rw-r–r-- 1 www-data www-data 26 May 27 16:17 robots.txt
-rw-r–r-- 1 www-data www-data 2.4K May 27 16:17 status.php
drwxrwxr-x 3 1000 1000 4.0K May 26 16:34 themes
-rw-r–r-- 1 www-data www-data 383 May 27 16:17 version.php

I thought docker would automatically take care of group and user shenanigans when mounting a volume?

jtr · May 28, 2023, 7:41pm

Sort of, but only if you’re using named volumes. If you’re using bind mounts, not so much.

Arthur_D · May 29, 2023, 4:38pm

Okay, so after chowning the folders to www-data, it works! However when clicking on Contacts it says: “Could not load your contacts”
Checking in “Logging” in the administration panel, it has a bunch of these errors for various apps, including Contacts:
/appinfo/app.php is not loaded when \OCP\AppFramework\Bootstrap\IBootstrap on the application class is used. Migrate everything from app.php to the Application class.

I found this unresolved GitHub issue about this error: /appinfo/app.php is not loaded when \OCP\AppFramework\Bootstrap\IBootstrap on the application class is used. Migrate everything from app.php to the Application class. · Issue #30372 · nextcloud/server · GitHub

However, just moving the app.php file did not seem to help except the error no longer showed up for Contacts… so maybe there’s something else causing Contacts to not load my… well, contacts?

jtr · May 29, 2023, 8:11pm

Can you provide the output of:

./occ config:list system
./occ app:list
`https://YOUR_NC_HOSTNAME/index.php/settings/integrity/failed

Are all your apps shown as up-to-date under Apps?

Arthur_D · May 29, 2023, 8:31pm

sudo docker compose exec -u www-data -it nextcloud php occ config:list system

    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "cloud.pspodcasting.net"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "26.0.1.1",
        "overwrite.cli.url": "http:\/\/cloud.pspodcasting.net",
        "overwriteprotocol": "https",
        "installed": true,
        "maintenance": false,
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "theme": "",
        "loglevel": 0,
        "mysql.utf8mb4": true,
        "mail_smtpdebug": false,
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "mail_smtptimeout": 10,
        "mail_smtpstreamoptions": [],
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "default_phone_region": "NO",
        "updater.secret": "***REMOVED SENSITIVE VALUE***"
    }
}

sudo docker compose exec -u www-data -it nextcloud php occ app:list

  - activity: 2.18.0
  - admin_audit: 1.16.0
  - calendar: 4.3.4
  - circles: 26.0.0
  - cloud_federation_api: 1.9.0
  - comments: 1.16.0
  - contacts: 5.2.0
  - contactsinteraction: 1.7.0
  - dashboard: 7.6.0
  - dav: 1.25.0
  - external: 5.1.0
  - federatedfilesharing: 1.16.0
  - federation: 1.16.0
  - files: 1.21.1
  - files_accesscontrol: 1.16.0
  - files_pdfviewer: 2.7.0
  - files_rightclick: 1.5.0
  - files_sharing: 1.18.0
  - files_trashbin: 1.16.0
  - files_versions: 1.19.1
  - firstrunwizard: 2.15.0
  - groupfolders: 14.0.2
  - logreader: 2.11.0
  - lookup_server_connector: 1.14.0
  - maps: 1.0.2
  - nextcloud_announcements: 1.15.0
  - notifications: 2.14.0
  - oauth2: 1.14.0
  - password_policy: 1.16.0
  - photos: 2.2.0
  - privacy: 1.10.0
  - provisioning_api: 1.16.0
  - recognize: 4.1.0
  - recommendations: 1.5.0
  - related_resources: 1.1.0-alpha1
  - richdocuments: 8.0.2
  - serverinfo: 1.16.0
  - settings: 1.8.0
  - sharebymail: 1.16.0
  - spreed: 16.0.4
  - support: 1.9.0
  - survey_client: 1.14.0
  - systemtags: 1.16.0
  - tasks: 0.15.0
  - text: 3.7.2
  - theming: 2.1.1
  - twofactor_backupcodes: 1.15.0
  - twofactor_totp: 8.0.0-alpha.0
  - updatenotification: 1.16.0
  - user_status: 1.6.0
  - viewer: 1.10.0
  - weather_status: 1.6.0
  - workflowengine: 2.8.0
Disabled:
  - apporder: 0.15.0 (installed 0.15.0)
  - bruteforcesettings: 2.6.0 (installed 2.4.0)
  - data_request: 1.13.0 (installed 1.13.0)
  - drawio: 2.1.1 (installed 2.1.1)
  - duplicatefinder: 0.0.15 (installed 0.0.15)
  - encryption: 2.14.0 (installed 2.9.0)
  - extract: 1.3.5 (installed 1.3.5)
  - files_external: 1.18.0
  - files_fulltextsearch: 26.0.0 (installed 26.0.0)
  - files_markdown: 2.4.0 (installed 2.4.0)
  - files_mindmap: 0.0.27 (installed 0.0.27)
  - fulltextsearch: 26.0.0 (installed 26.0.0)
  - fulltextsearch_elasticsearch: 26.0.0 (installed 26.0.0)
  - metadata: 0.18.0 (installed 0.18.0)
  - ransomware_protection: 1.14.0 (installed 1.14.0)
  - socialsharing_email: 2.6.0 (installed 2.6.0)
  - suspicious_login: 4.4.0
  - user_ldap: 1.16.0

https://cloud.pspodcasting.net/index.php/settings/integrity/failed

No errors have been found.

https://cloud.pspodcasting.net/index.php/settings/apps

All apps are up to date.

jtr · May 29, 2023, 10:16pm

If you’re still getting that message from any other apps something is wrong with each of those apps. Might be challenging to tackle the Contacts issue while those are still around (if nothing else because of the logs being so full of stuff). Should be able to address them in the same way, since it’s likely that some stray app.php files got left around from one of your upgrade increments or something.

Out of curiosity - how did you handle upgrading NC during this migration? Were you on NC26 prior to switching to the new setup or… ?

Anyhow, for Contacts, that’s just a generic error coming from the frontend when it goes to load your contacts from the server. There should be an associated log entry with a stack trace on the server side in nextcloud.log with all the useful information for tracking down what is going on.

Arthur_D · May 31, 2023, 6:48pm

Yeah I moved the app.php files out of the way so they wouldn’t generate extra errors.

I am not entirely sure whether I was on NC26 or NC25 before migrating, I can’t remember, sorry.

I tried to search through nextcloud.log but there is A LOT of very densely packed information, so I might miss something. However, Contacts gives exceptions:

"user":"magne","app":"index","method":"POST","url":"/index.php/contactsmenu/contacts","message":"Parameter \"guestName\" for route \"core.GuestAvatar.getAvatar\" must match \"[^/]++\" (\"\" given) to generate a corresponding URL.","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0","version":"26.0.1.1","exception":{"Exception":"Symfony\\Component\\Routing\\Exception\\InvalidParameterException","Message":"Parameter \"guestName\" for route \"core.GuestAvatar.getAvatar\" must match \"[^/]++\" (\"\" given) to generate a corresponding URL.","Code":0,"Trace":[{"file":"/var/www/html/3rdparty/symfony/routing/Generator/UrlGenerator.php","line":160,"function":"doGenerate","class":"Symfony\\Component\\Routing\\Generator\\UrlGenerator","type":"->","args":[[0,1],[["core","GuestAvatarController","getAvatar"],null],[],[["variable","/","[^/]++","size"],["variable","/","[^/]++","guestName"],["text","/avatar/guest"]],["",64],"core.GuestAvatar.getAvatar",1,[],[]]},{"file":"/var/www/html/lib/private/Route/Router.php","line":372,"function":"generate","class":"Symfony\\Component\\Routing\\Generator\\UrlGenerator","type":"->","args":["core.GuestAvatar.getAvatar",["",64],1]},{"file":"/var/www/html/lib/private/Route/CachingRouter.php","line":65,"function":"generate","class":"OC\\Route\\Router","type":"->","args":["core.GuestAvatar.getAvatar",["",64],false]},{"file":"/var/www/html/lib/private/URLGenerator.php","line":103,"function":"generate","class":"OC\\Route\\CachingRouter","type":"->","args":["core.GuestAvatar.getAvatar",["",64]]},{"file":"/var/www/html/lib/private/URLGenerator.php","line":115,"function":"linkToRoute","class":"OC\\URLGenerator","type":"->","args":["core.GuestAvatar.getAvatar",["",64]]},{"file":"/var/www/html/lib/private/Contacts/ContactsMenu/ContactsStore.php","line":293,"function":"linkToRouteAbsolute","class":"OC\\URLGenerator","type":"->","args":["core.GuestAvatar.getAvatar",["",64]]},{"file":"/var/www/html/lib/private/Contacts/ContactsMenu/ContactsStore.php","line":110,"function":"contactArrayToEntry","class":"OC\\Contacts\\ContactsMenu\\ContactsStore","type":"->","args":["*** sensitive parameters replaced ***"]},{"function":"OC\\Contacts\\ContactsMenu\\{closure}","class":"OC\\Contacts\\ContactsMenu\\ContactsStore","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/Contacts/ContactsMenu/ContactsStore.php","line":109,"function":"array_map","args":[["Closure"],[["235cc70d-e77b-442b-a571-b710371eb27a.vcf","4.0","+//IDN bitfire.at//DAVx5/4.3-gplay ez-vcard/0.11.3","235cc70d-e77b-442b-a571-b710371eb27a"

I don’t know whether these exceptions are what causes the contacts to not load. After the end of the quote there’s a list of all the contacts I had, so they are still there, just not loading correctly.