Redis session manager breaking nextcloud docker compose installation

lsbNextcloud · April 9, 2023, 4:19am

I’m attempting to install nextcloud docker on my Synology 923+ inside portainer.

When I create the instance without redis in my docker compose file I can bring up the nextcloud app without issue. The same is true if I add the redis portion to my compose file, but exclude the REDIS_HOST env from the nextcloud service. However, once nextcloud attempts to use redis as the session handler the nextcloud site will spin until it times out.

After many hours of troubleshooting I think I’ve found the issue, but can’t figure out a fix. I have confirmed the redis server is up. Unfortunately ping is not installed with the nextcloud docker, but using /dev/tcp I’ve established that the nextcloud container cannot ping the redis container. This the db container which is on the same network as the redis and nextcloud containers can ping without issue.

Here is my docker-compose.yml:

version: '3'

services:
  db:
    image: mariadb:10.6
    restart: always
    command: --transaction-isolation=READ-COMMITTED --log-bin=binlog --binlog-format=ROW --innodb-read-only-compressed=OFF
    volumes:
      - /volume1/docker/nextcloud/database:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD_FILE=/run/secrets/mysql_root_pw
      - MYSQL_PASSWORD_FILE=/run/secrets/mysql_pw
      - MYSQL_DATABASE_FILE=/run/secrets/mysql_db
      - MYSQL_USER_FILE=/run/secrets/mysql_user
      - MARIADB_AUTO_UPGRADE=1
      - MARIADB_DISABLE_UPGRADE_BACKUP=1
    secrets:
      - mysql_db
      - mysql_user
      - mysql_pw
      - mysql_root_pw

      
  nc-redis:
    image: redis:5
    restart: always
    volumes:
      - /volume1/docker/nextcloud/redis:/data
    expose:
      - 6379

  app:
    image: nextcloud
    restart: always
    ports:
      - 8080:80
    volumes:
      - /volume1/docker/nextcloud/config:/var/www/html
    environment:
      - MYSQL_PASSWORD_FILE=/run/secrets/mysql_pw
      - MYSQL_DATABASE_FILE=/run/secrets/mysql_db
      - MYSQL_USER_FILE=/run/secrets/mysql_user
      - MYSQL_HOST=db
      - REDIS_HOST=nc-redis
    secrets:
      - mysql_pw
      - mysql_db
      - mysql_user
    depends_on:
      - db
      - nc-redis
      
secrets:
  mysql_user:
    file: /volume1/docker/vars/mysql_user.txt
  mysql_db:
    file: /volume1/docker/vars/mysql_db.txt
  mysql_pw:
    file: /volume1/docker/vars/mysql_pw.txt
  mysql_root_pw:
    file: /volume1/docker/vars/mysql_root_pw.txt
  redis_pass:
    file: /volume1/docker/vars/redis_pw.txt
  nc_user:
    file: /volume1/docker/vars/nc_user.txt
  nc_pw:
    file: /volume1/docker/vars/nc_pw.txt

Unfortunately all the logs I’ve been able to find have been useless.

Does anyone have any ideas?

Thanks in advance.

KarlF12 · April 9, 2023, 12:20pm

You can install it in the container if you need it.

What address are you using to hit redis? If it’s in the same docker network, you can just use the container’s name, and then you also don’t have to open a port for it.

However I don’t see a docker network defined in your compose file.

Instead of expose, this should have a ports: setting like the others.

Also I think since a particular version, you need to change the redis command to either disable authentication or provide a password. You can find some examples of that on the forum.

lsbNextcloud · April 9, 2023, 1:54pm

Thanks for the response!

I would like to install ping, but the normal commands to do that seem to fail when I try.

I hit tcp://redis:6379, but I’ve also tried the in network IP assigned by portainer/docker and port 6379, both failed.

I should have mentioned previous other failed attempts sorry about that. I’m currently just using the default docker network, but I’ve also tried it using a bridge network defined in the docker-compose file.

Back to my other failed attempts, I’ve tried adding a ports: section to the nc-redis service it did not help. I’m not sure why I would need to publish the ports for in network communication though. Additionally, I’ve tried running redis with a defined password to no avail.

For good measure I just retried using this updated docker-compose with some of your suggestions:

version: '3'

services:
  db:
    image: mariadb:10.6
    restart: always
    command: --transaction-isolation=READ-COMMITTED --log-bin=binlog --binlog-format=ROW --innodb-read-only-compressed=OFF
    volumes:
      - /volume1/docker/nextcloud/database:/var/lib/mysql
    networks:
      - ncnet
    environment:
      - MYSQL_ROOT_PASSWORD_FILE=/run/secrets/mysql_root_pw
      - MYSQL_PASSWORD_FILE=/run/secrets/mysql_pw
      - MYSQL_DATABASE_FILE=/run/secrets/mysql_db
      - MYSQL_USER_FILE=/run/secrets/mysql_user
      - MARIADB_AUTO_UPGRADE=1
      - MARIADB_DISABLE_UPGRADE_BACKUP=1
    secrets:
      - mysql_db
      - mysql_user
      - mysql_pw
      - mysql_root_pw

      
  nc-redis:
    image: redis:latest
    restart: always
    volumes:
      - /volume1/docker/nextcloud/redis:/data
    networks:
      - ncnet
    ports:
      - 6379:6379
    command: redis-server --requirepass testPass 

  app:
    image: nextcloud
    restart: always
    ports:
      - 8080:80
    networks:
      - ncnet
    volumes:
      - /volume1/docker/nextcloud/config:/var/www/html
    environment:
      - MYSQL_PASSWORD_FILE=/run/secrets/mysql_pw
      - MYSQL_DATABASE_FILE=/run/secrets/mysql_db
      - MYSQL_USER_FILE=/run/secrets/mysql_user
      - MYSQL_HOST=db
      - REDIS_HOST=nc-redis
      - REDIS_PORT=6379
      - REDIS_HOST_PASSWORD=testPass
    secrets:
      - mysql_pw
      - mysql_db
      - mysql_user
    depends_on:
      - db
      - nc-redis
      
networks:
  ncnet:
    driver: bridge
      
secrets:
  mysql_user:
    file: /volume1/docker/vars/mysql_user.txt
  mysql_db:
    file: /volume1/docker/vars/mysql_db.txt
  mysql_pw:
    file: /volume1/docker/vars/mysql_pw.txt
  mysql_root_pw:
    file: /volume1/docker/vars/mysql_root_pw.txt
  redis_pass:
    file: /volume1/docker/vars/redis_pw.txt
  nc_user:
    file: /volume1/docker/vars/nc_user.txt
  nc_pw:
    file: /volume1/docker/vars/nc_pw.txt

it failed the same way.

Here are some additional details.

From the command line of my redis container:
root@fdfd6d24afff:/data# redis-cli -a testPass ping Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. PONG
root@fdfd6d24afff:/data# redis-cli -a testPass Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 127.0.0.1:6379> KEYS * (empty array)
root@fdfd6d24afff:/data# (</dev/tcp/127.0.0.1/6379) &>/dev/null && echo "OPEN" || echo "CLOSED" OPEN
root@fdfd6d24afff:/data# (</dev/tcp/nc-redis/6379) &>/dev/null && echo "OPEN" || echo "CLOSED" OPEN
root@fdfd6d24afff:/data# (</dev/tcp/db/3306) &>/dev/null && echo "OPEN" || echo "CLOSED" OPEN
root@fdfd6d24afff:/data# (</dev/tcp/app/80) &>/dev/null && echo "OPEN" || echo "CLOSED" CLOSED

The final command to ping app/80 is different from the other “CLOSED” failures in that it spins for over a minute before failing.

Some info from the command line of the nextcloud container:
root@b02730e8192d:/var/www/html# (</dev/tcp/app/80) &>/dev/null && echo "OPEN" || echo "CLOSED" OPEN
root@b02730e8192d:/var/www/html# (</dev/tcp/127.0.0.1/80) &>/dev/null && echo "OPEN" || echo "CLOSED" OPEN
root@b02730e8192d:/var/www/html# (</dev/tcp/nc-redis/6379) &>/dev/null && echo "OPEN" || echo "CLOSED" CLOSED
root@b02730e8192d:/var/www/html# (</dev/tcp/172.19.0.2/6379) &>/dev/null && echo "OPEN" || echo "CLOSED" CLOSED

(172.19.0.2 is the local IP assigned by docker/portainer for the redis container)
Similar to attempting to ping the app container from the redis container, when I attempt to ping the redis container from the app container it spins for over a minute and then fails.

KarlF12 · April 9, 2023, 2:49pm

There are a few different ways you can do this. What I personally do is define a docker network in the compose file so all the containers are in it together. Then they can address each other by container name. This does not require the ports to be open on redis or mariadb.

I don’t use the bridge driver when doing this. My docker network section is literally just this:

networks:
 nextcloud:

And then under each container:

    networks:
      - nextcloud

That puts them in the same virtual network where they can address each other by container name and docker automatically translates them to the docker network IPs.

You can use the docker network IPs directly, but I don’t like doing that because I worry they may change during a restart one day, so you’d have to assign them statically which is really just extra unnecessary work when you can use the names.

Another method is to open the ports on the host and connect to the container via docker NAT. Note that if you do this, DO NOT use a loopback address. You need to use the IP or hostname of the host server itself.

lsbNextcloud · April 9, 2023, 3:04pm

Thanks again for your quick response!

KarlF12:

I don’t use the bridge driver when doing this. My docker network section is literally just this:
networks:
 nextcloud:
And then under each container:
    networks:
      - nextcloud
That puts them in the same virtual network where they can address each other by container name and docker automatically translates them to the docker network IPs.

I just tried this, but the same problem persists.

I actually tried this above:

1root@b02730e8192d:/var/www/html# (</dev/tcp/172.19.0.2/6379)
&>/dev/null && echo “OPEN” || echo “CLOSED” CLOSED`

and it failed.

I’m not 100% sure this is even a network issue vs something blocking connections in and out of the redis container to and from the nextcloud container. I’d be more convinced it was a network only issue if I couldn’t ping any containers from my redis container, but I could hit my db service without issue as I posted above:

root@fdfd6d24afff:/data# (</dev/tcp/db/3306) &>/dev/null && echo "OPEN" || echo "CLOSED" OPEN

I’m not super familiar with this method and I’d probably break more than I fix attempting it. Also I’m not sure if this would open me up to security concerns.

KarlF12 · April 9, 2023, 3:56pm

Are you running any firewall on the server? It can affect docker networking.

You have a very unusual way of checking if the port is reachable, so I’m not sure what to expect from that. Run docker compose ps to make sure containers are running and ports are open. Also check the uptime of redis to make sure it’s not failing to start and restarting continuously.

KarlF12 · April 9, 2023, 4:28pm

Here’s an example from a guide a wrote. This was for an older version, so I don’t think it works as-is for the current one (namely due to change in default auth in redis).

As you can see, because redis was only receiving connections from another container in the same docker network, I had no port open for redis at all. It’s only needed if you’re taking connections from outside the docker network. I have no open port on MariaDB either.

The ports are still “open” per se, and you can see them in docker compose ps, but they aren’t port forwarded to the container from outside the docker network. That’s what the port spec on the container does.

lsbNextcloud · April 9, 2023, 6:36pm

I’ve got it working now!

Thanks for the advice, it looks like it was a firewall issue. Once I opened the 6379 and 80 ports on my synology firewall everything spun up correctly. I wasn’t expecting the docker network to interplay with the host firewall.

Thanks again for all your time helping me out with this!

KarlF12 · April 9, 2023, 6:51pm

Sure, glad to hear it’s working. I’ve run into that from UFW before. I think it’s because docker creates interfaces on the host and so the firewall wants to block incoming since there are no rules, and doesn’t take into account that the source is another local interface.