First attempt at creating a docker-compose .yml file

AIS_Todd · February 14, 2019, 5:03pm

Hi,

Host VPS system: using Plesk 17.8.11 on top of Ubuntu 18.04 with Apache 2.4.29 and PHP 7.2.15.

I am a nextcloud newbie and a docker newbie who got some good advice from Reiner_Nippes within this thread. Since then I’ve been doing mostly copy/paste stuff from various articles, and I’m pretty sure I need some more help before I’ll be able to get this nextcloud installation working. I don’t even know the directory I need to put the .yml file in before entering the docker-compose up command, so I haven’t tried anything yet.

I created a “nextcloud_network” docker network (bridge driver), and the .yml file I’m going to use looks like this so far:

> version: '3' 
> 
> services:
> 
>   db:
>     image: mariadb:latest
>     container_name: nextcloud-mariadb
>     ports:
>       - "3307:3307"
>     networks:
>       - nextcloud_network
>     volumes:
>       - db:/mnt/mariadb-data
>       - /etc/localtime:/etc/localtime:ro
>     environment:
>       - MYSQL_ROOT_PASSWORD=secret
>       - MYSQL_USER=nextcloud_db_user
>       - MYSQL_PASSWORD=secret
>       - MYSQL_DATABASE=nextcloud
>       - TRANSACTION_ISOLATION=READ-COMMITTED
>     restart: unless-stopped
>   
>   app:
>     image: nextcloud:latest
>     container_name: nextcloud
>     networks:
>       - nextcloud_network
>     depends_on:
>       - db
>     volumes:
>       - nextcloud:/var/www/html
>       - ./app/data:/mnt/nextcloud-data
>       - ./app/config:/mnt/nextcloud-config
>       - ./app/apps:/var/www/html/apps
>       - ./app/custom_apps:/var/www/html/custom_apps
>       - ./app/themes:/var/www/html/themes
>       - /etc/localtime:/etc/localtime:ro
>     environment:
>       - MYSQL_HOST=db
>     restart: unless-stopped
> 
> volumes:
>   nextcloud:
>   db:
> 
> networks:
>   nextcloud_network:

I need to use port 3307 for the MariaDB (3306 is in use for my VPS account’s internal MySQL databases and my hosting provider recommended that I not change their transaction isolation setting from REPEATABLE-READ to anything else). Is that “3307:3307” ports line what I need? Is the “TRANSACTION_ISOLATION=READ-COMMITTED” line appropriate?

I plan to run everything on https://subdomain.mydomain.com. I have existing letsencrypt certificates (installed via Plesk) for both mydomain.com and subdomain.mydomain.com, which seemed to work fine for the trial run I did with docker nextcloud:15.02 (using the SQLite db). How do I go about making this new docker-compose installation secure?

Assuming I can get this docker-compose up, how would I then go about backing up nextcloud and its database/data? I ask because my Plesk documentation contains this disclaimer: “Docker containers in Plesk cannot be migrated or backed up. However, you can back up data used by containers (see Volume Mapping below), or download snapshots.” I’m guessing that such a disclaimer applies only to Plesk-based GUI administration, so could I still follow the cli commands outlined in Nextcloud’s Server Admin Manual?

Thanks for any insight.

Reiner_Nippes · February 14, 2019, 11:37pm

no. that what expose port 3307 from inside the container to the world.
a) no one is listening to this port
b) you don’t want to expose the database to the world.
c) because nextcloud is accessing the database through the internal docker network.

you have to look into the chapter “Configuration without a cnf file”
It says:

If you would like to see a complete list of available options, just run:

$ docker run -it --rm mariadb:tag --verbose --help

I guess there is a parameter port. Set this to 3307

btw: since nextcloud accesses the db through the docker network and you don’t expose the db to the host network. you don’t need to change the port. or?

AIS_Todd · February 15, 2019, 4:30am

Thanks very much for the information. I’ll try it without any special port configuration.

I’ll look through that chapter on configuring nextcloud without a cnf file, possibly posting again in this thread if I get confused about something or other. Perhaps that should be when I get confused

Thanks again for taking the time to reply.

AIS_Todd · February 15, 2019, 9:43pm

Okay, so I got the docker-compose up -d command to work with the following revised .yml file:

> version: '3' 
> 
> services:
> 
>   db:
>     image: mariadb:latest
>     command: --transaction-isolation=READ-COMMITTED --binlog-format=ROW
>     container_name: nextcloud-mariadb
>     networks:
>       - nextcloud_network
>     volumes:
>       - db:/mnt/mariadb-data
>       - /etc/localtime:/etc/localtime:ro
>     environment:
>       - MYSQL_ROOT_PASSWORD=secret
>       - MYSQL_USER=mariadb_user
>       - MYSQL_PASSWORD=secret
>       - MYSQL_DATABASE=nextcloud
>     restart: unless-stopped
>   
>   app:
>     image: nextcloud:latest
>     container_name: nextcloud
>     networks:
>       - nextcloud_network
>     depends_on:
>       - db
>     volumes:
>       - nextcloud:/var/www
>       - ./app/data:/mnt/nextcloud-data
>       - ./app/config:/mnt/nextcloud-config
>       - ./app/apps:/var/www/nextcloud/apps
>       - ./app/custom_apps:/var/www/nextcloud/custom_apps
>       - ./app/themes:/var/www/nextcloud/themes
>       - /etc/localtime:/etc/localtime:ro
>     environment:
>       - MYSQL_HOST=db
>     restart: unless-stopped
> 
> volumes:
>   nextcloud:
>   db:
> 
> networks:
>   nextcloud_network:

I noticed two things with this setup:

The log for the MariaDB contains the following warning: “You need to use --log-bin to make --binlog-format work.” I found a reference page for a --log-bin=datadir/'log-basename'-bin command, but I’m wondering how to tweak it to do what I want. Would I need to include the /mnt/mariadb-data path somehow?
The /var/www/nextcloud, /mnt/nextcloud-data and /mnt/nextcloud-config directories are empty, but the directory containing my docker-compose.yml file got seeded with an “app” directory which contains nextcloud’s data, config, apps, custom_apps and themes directories. Is that to be expected according to the format of my .yml file, because I want the data and config directories to be under /mnt/nextcloud-data and /mnt/nextcloud-config (respectively) and I want the apps, custom_apps and themes directories to be under /var/www/nextcloud.

I copy/pasted those volumes for the nextcloud app, so I might not be doing it right. Thanks for any insight.

Reiner_Nippes · February 16, 2019, 11:52am

left side = host filesystem (or docker volume) / right side = container.

so:

AIS_Todd:

volumes:
  - nextcloud:/var/www/html
  - /mnt/nextcloud-data:/var/www/html/data/
  - /mnt/nextcloud-config:/var/www/html/config/
  - ./app/apps:/var/www/html/apps/
  - ./app/custom_apps:/var/www/html/custom_apps/
  - ./app/themes:/var/www/html/themes/
  - /etc/localtime:/etc/localtime:ro

because you use ./app/xxx on the left side. personally i would always use an absolute path (beginning with / ). not relative (beginning with ./ ) to the location where you are when you run the docker-compose command.
cd /tmp ; docker up -d /path-to/nextcloud-compose.yml would put the app folder into /tmp.

so what about:

AIS_Todd:

volumes:
  - /opt/nextcloud/www:/var/www/html
  - /opt/nextcloud/data:/var/www/html/data/
  - /opt/nextcloud/config:/var/www/html/config/
  - /opt/nextcloud/apps:/var/www/html/apps/
  - /opt/nextcloud/custom_apps:/var/www/html/custom_apps/
  - /opt/nextcloud/themes:/var/www/html/themes/
  - /etc/localtime:/etc/localtime:ro

Docker Chapter: Using a custom MySQL configuration file
here https://www.c-rieger.de/nextcloud-installation-guide-ubuntu/#c03 you’ll find a my.cnf. take the parameters you want/need. (or all). write them to /opt/nextcloud/database-config/nextcloud.cnf and add

volumes:
  - /opt/mariadb/data:/var/lib/mysql
  - /opt/mariadb/log:/var/log/mysql
  - /opt/mariadb/database-config:/etc/mysql/conf.d
  - /etc/localtime:/etc/localtime:ro

to the compose files db section.
p.s. didn’t test the snippets above. might contain typos.

AIS_Todd · February 17, 2019, 6:03am

Thanks very much for all this assistance. I can almost see the light at the end of the tunnel.

I think I have the various configuration files straightened out. Here is my current .yml file:

> version: '3' 
> 
> services:
> 
>   db:
>     image: mariadb:latest
>     container_name: nextcloud-mariadb
>     networks:
>       my_network
>     volumes:
>       - /opt/mariadb/data:/var/lib/mysql
>       - /opt/mariadb/log:/var/log/mysql
>       - /opt/mariadb/config:/etc/mysql/conf.d
>       - /etc/localtime:/etc/localtime:ro
>     environment:
>       - MYSQL_ROOT_PASSWORD=secret
>       - MYSQL_USER=mariadb_user
>       - MYSQL_PASSWORD=secret
>       - MYSQL_DATABASE=nextcloud
>     restart: unless-stopped
>   
>   app:
>     image: nextcloud:latest
>     container_name: nextcloud
>     networks:
>       - my_network
>     depends_on:
>       - db
>     volumes:
>       - /opt/nextcloud/www:/var/www/html
>       - /opt/nextcloud/data:/var/www/html/data/
>       - /opt/nextcloud/config:/var/www/html/config/
>       - /opt/nextcloud/apps:/var/www/html/apps/
>       - /opt/nextcloud/custom_apps:/var/www/html/custom_apps/
>       - /opt/nextcloud/themes:/var/www/html/themes/
>       - /etc/localtime:/etc/localtime:ro
>     environment:
>       - MYSQL_HOST=db
>     restart: unless-stopped
> 
> volumes:
>   nextcloud:
>   db:
> 
> networks:
>   my_network:

When I run docker-compose up -d /path/to/docker-compose.yml I get the following error:

ERROR: 
        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?

        Supported filenames: docker-compose.yml, docker-compose.yaml

I confirmed by way of ls that I am entering the appropriate path in the part of the command where I substituted /path/to/ for this post. I tried running the docker-compose up command both with and without the filename, getting the same error each time. Obviously, I’m doing something wrong.

Thanks for any insight.

Reiner_Nippes · February 17, 2019, 10:42am

in one of this statments a " - " before my_network is missing. but that’s not the problem.

sorry. i’m not using docker-compse frequently. (and i don’t have a installation at and.) so it was just a guess that one can call docker-compose with a file name to point at the compose-file.

use docker-compose --help to get the correct command to use a specific compose-file.

nevertheless since now you use absolute pathes you may use docker-compose up -d in the same path then your compose-file.

if you want a nice docker gui have a look at https://portainer.io
if you want to get some ideas how to put it together with your nextcloud container have a look at GitHub - norweeg/self-hosted-docker-server-templates: Just some templates to get someone started with hosting various servers in Docker a nice collection of docker compose file to create stacks like nextcloud.

AIS_Todd · February 18, 2019, 7:30am

No need to apologize, I’m the one who feels sorry for taking up so much of your time. Thanks as well for those links. When I acquire some more server resources I’ll try implementing the BASE stack from that github page you provided.

I went ahead and entered docker-compose up -d without any errors. MariaDB didn’t start, though, and the log contains this information:

> Warning: skipping '!includedir /etc/mysql/mariadb.conf.d/' directive as maximum includerecursion level was reached in file /etc/mysql/mariadb.cnf at line 19
> Warning: skipping '!include /etc/mysql/mariadb.cnf' directive as maximum includerecursion level was reached in file /etc/mysql/conf.d/nextcloud.cnf at line 77
> Warning: skipping '!includedir /etc/mysql/conf.d/' directive as maximum includerecursion level was reached in file /etc/mysql/conf.d/nextcloud.cnf at line 78
> 2019-02-17 15:51:57 0 [Note] mysqld (mysqld 10.3.12-MariaDB-1:10.3.12+maria~bionic-log) starting as process 1 ...
> 2019-02-17 15:51:57 0 [ERROR] mysqld: File '/var/log/mysql/mariadb-bin.index' not found (Errcode: 13 "Permission denied")
> 2019-02-17 15:51:57 0 [ERROR] Aborting

I’m not sure what to do. I’d also like to clear those warnings if I can, especially since some of them are relevant to nextcloud.cnf.

Thanks for any insight.

Reiner_Nippes · February 18, 2019, 8:35am

look at the end of the nextcloud.cnf file and remove the two !include lines

[isamchk]
!include /etc/mysql/mariadb.cnf
!includedir /etc/mysql/conf.d/
key_buffer = 16M

carsten provides a new my.cnf that will include files in /etc/mysql/conf.d/
you copied this statement into a file in that directory. that gives an infinite include loop.

my guess is that you didn’t change the owner of /opt/mariadb on the host. i’m not sure which uid the user mysql has. maybe chown -R 27:27 /opt/mariadb will help. (27 should be the default uid of the mysql user.)
if not: check the folder permisions with
ls -l /opt/mariadb/log
then
chmod 0777 /opt/mariadb/log
and start the container.
look which ids the files created in there have. use this id in the chown command and revert chmod 07xx /opt/mariadb/log. (replace the xx with the original values.)

AIS_Todd · February 19, 2019, 2:45am

Thanks very much for the continued advice. I’m getting closer, I can feel it.

Deleting the !include lines from nextcloud.cnf seemed to do the trick for those warnings, but I’m still having trouble when I try to get the MariaDB server up. I entered chown -R 27:27 /opt/mariadb and then entered ls -l /opt. The result for the mariadb directory was as follows:

drwxr-xr-x 5 27 sudo 4096 Feb 16 23:54 mariadb

That didn’t seem to me to be quite right, but I tried docker-compose up -d anyway and found another “permission denied” error in the MariaDB log.

I then tried something else based on a DuckDuck search I did for uid, so I’m sorry if I mucked things. I entered getent passwd from the command line and noticed this output for the mysql entry:

mysql:x:112:121:MySQL Server,,,:/nonexistent:/bin/false

Then after entering chown -R 112:121 /opt/mariadb and ls -l /opt, I saw this output for the mariadb directory:

drwxr-xr-x 5 mysql mysql 4096 Feb 16 23:54 mariadb

That seemed more like what I need, so I tried docker-compose up -d again and found another “permission denied” error in the MariaDB log.

> 2019-02-18 20:57:01 0 [ERROR] mysqld: File '/var/log/mysql/mariadb-bin.index' not found (Errcode: 13 "Permission denied")
> 2019-02-18 20:57:01 0 [ERROR] Aborting

I’m not sure what to try next. Thanks for any insight.

Reiner_Nippes · February 19, 2019, 7:27am

yes. but you need to do this inside the container. so login to the container first.

sudo docker exec -it db /bin/bash

AIS_Todd · February 19, 2019, 7:42pm

I’m sorry for continuing to be such a pain, but I have another hurdle to overcome. When I entered docker exec -it <containerName> /bin/bash I got the following response:

Error response from daemon: Container <containerID> is restarting, wait until the container is running

I waited & retried several times, each time getting the same response, and I’m not sure what to try next. Can I do some kind of docker run command on just the mariadb:latest image and then work with that container for changing necessary chown parameters? I’m guessing that such an approach wouldn’t help my docker-compose stuff.

Thanks for any insight you’re willing to offer.

AIS_Todd · March 5, 2019, 12:01am

Apologies for the thread bump, but I really would like to get Nextcloud up & running. Could someone suggest how I might overcome my newbie errors with this Docker Compose setup?

Thanks for any insight.