[SOLVED] Latest Docker image broke the installation (Redis password auth)

broken.pipe · July 18, 2020, 7:24am

Hello everyone,

after updating and restarting the container i can’t log in anymore. It displays “internal server error 500” … Unfortunately I cannot execute any OCC command, because every command leads into a Redis Exception (I am not using any password for redis). Do you have any idea?

OCC logs

root@HOME-SERVER:[config]> docker exec -u www-data -it nextcloud php /var/www/html/occ maintenance:mode --on
An unhandled exception has been thrown:
RedisException: ERR AUTH <password> called without any password configured for the default user. Are you sure your configuration is correct? in /var/www/html/lib/private/RedisFactory.php:94
Stack trace:
#0 /var/www/html/lib/private/RedisFactory.php(94): Redis->auth(false)
#1 /var/www/html/lib/private/RedisFactory.php(108): OC\RedisFactory->create()
#2 /var/www/html/lib/private/Memcache/Redis.php(43): OC\RedisFactory->getInstance()
#3 /var/www/html/lib/private/Memcache/Factory.php(135): OC\Memcache\Redis->__construct('091c2ff6469d381...')
#4 /var/www/html/lib/private/Server.php(1021): OC\Memcache\Factory->createLocking('lock')
#5 /var/www/html/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\Server->OC\{closure}(Object(OC\Server))
#6 /var/www/html/lib/private/ServerContainer.php(124): Pimple\Container->offsetGet('OCP\\Lock\\ILocki...')
#7 /var/www/html/lib/private/Server.php(1970): OC\ServerContainer->query('OCP\\Lock\\ILocki...')
#8 /var/www/html/lib/private/Files/View.php(118): OC\Server->getLockingProvider()
#9 /var/www/html/lib/private/Server.php(299): OC\Files\View->__construct()
#10 /var/www/html/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\Server->OC\{closure}(Object(OC\Server))
#11 /var/www/html/lib/private/ServerContainer.php(124): Pimple\Container->offsetGet('OCP\\Encryption\\...')
#12 /var/www/html/lib/private/Server.php(1443): OC\ServerContainer->query('OCP\\Encryption\\...')
#13 /var/www/html/lib/base.php(843): OC\Server->getEncryptionManager()
#14 /var/www/html/lib/base.php(728): OC::registerEncryptionWrapper()
#15 /var/www/html/lib/base.php(1090): OC::init()
#16 /var/www/html/console.php(49): require_once('/var/www/html/l...')
#17 /var/www/html/occ(11): require_once('/var/www/html/c...')

Startup logs of the containers:

root@HOME-SERVER:[nextcloud]> docker-compose up
Starting nextcloud-redis ... done
Starting nextcloud-db    ... done
Starting nextcloud       ... done
Starting nextcloud-cron  ... done
Attaching to nextcloud-redis, nextcloud-db, nextcloud, nextcloud-cron
nextcloud          | Configuring Redis as session handler
nextcloud          | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.12. Set the 'ServerName' directive globally to suppress this message
nextcloud-cron     | crond: crond (busybox 1.30.1) started, log level 0
nextcloud          | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.12. Set the 'ServerName' directive globally to suppress this message
nextcloud-cron     | crond: user:www-data entry:(null)
nextcloud-redis    | 1:C 18 Jul 2020 07:13:36.491 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
nextcloud-redis    | 1:C 18 Jul 2020 07:13:36.491 # Redis version=6.0.5, bits=64, commit=00000000, modified=0, pid=1, just started
nextcloud-redis    | 1:C 18 Jul 2020 07:13:36.491 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
nextcloud-redis    | 1:M 18 Jul 2020 07:13:36.493 * Running mode=standalone, port=6379.
nextcloud-db       | 2020-07-18 07:13:36+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.4+maria~focal started.
nextcloud-redis    | 1:M 18 Jul 2020 07:13:36.493 # Server initialized
nextcloud          | [Sat Jul 18 07:13:38.140645 2020] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.38 (Debian) PHP/7.4.8 configured -- resuming normal operations
nextcloud          | [Sat Jul 18 07:13:38.140753 2020] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
nextcloud-db       | 2020-07-18 07:13:37+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
nextcloud-redis    | 1:M 18 Jul 2020 07:13:36.494 * Loading RDB produced by version 6.0.5
nextcloud-redis    | 1:M 18 Jul 2020 07:13:36.494 * RDB age 127 seconds
nextcloud-redis    | 1:M 18 Jul 2020 07:13:36.494 * RDB memory usage when created 0.77 Mb
nextcloud-redis    | 1:M 18 Jul 2020 07:13:36.494 * DB loaded from disk: 0.000 seconds
nextcloud-redis    | 1:M 18 Jul 2020 07:13:36.494 * Ready to accept connections
nextcloud-db       | 2020-07-18 07:13:37+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.4+maria~focal started.
nextcloud-db       | 2020-07-18  7:13:37 0 [Note] mysqld (mysqld 10.5.4-MariaDB-1:10.5.4+maria~focal-log) starting as process 1 ...
nextcloud-cron     | 100001000010000100001000010000100001000010000100001000010000
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: Using Linux native AIO
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: Uses event mutexes
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: Number of pools: 1
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: Using SSE4.2 crc32 instructions
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: Completed initialization of buffer pool
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: 128 rollback segments are active.
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: Creating shared tablespace for temporary tables
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
nextcloud-cron     | 111111111111111111111111
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: 10.5.4 started; log sequence number 2151504124; transaction id 3920897
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] Plugin 'FEEDBACK' is disabled.
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
nextcloud-cron     | 11111111111111111111111111111111
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] InnoDB: Buffer pool(s) load completed at 200718  7:13:38
nextcloud-cron     | 111111111111
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] Server socket created on IP: '::'.
nextcloud-cron     | 1111111
nextcloud-db       | 2020-07-18  7:13:38 0 [Warning] 'proxies_priv' entry '@% root@3ceaadfe2e14' ignored in --skip-name-resolve mode.
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] Reading of all Master_info entries succeeded
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] Added new Master_info '' to hash table
nextcloud-db       | 2020-07-18  7:13:38 0 [Note] mysqld: ready for connections.
nextcloud-db       | Version: '10.5.4-MariaDB-1:10.5.4+maria~focal-log'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
nextcloud-cron     | crond: user:www-data entry:(null)
nextcloud-cron     | 100001000010000100001000010000100001000010000100001000010000
nextcloud-cron     | 111111111111111111111111
nextcloud-cron     | 11111111111111111111111111111111
nextcloud-cron     | 111111111111
nextcloud-cron     | 1111111
nextcloud-cron     | crond: wakeup dt=22
nextcloud-cron     | crond: file www-data:
nextcloud-cron     | crond:  line php -f /var/www/html/cron.php
nextcloud          | 172.18.0.4 - markus [18/Jul/2020:07:14:02 +0000] "GET /status.php HTTP/1.1" 500 424 "-" "Mozilla/5.0 (Windows) mirall/2.6.5stable-Win64 (build 20200710) (Nextcloud)"
nextcloud          | 172.18.0.4 - - [18/Jul/2020:07:14:19 +0000] "GET / HTTP/1.1" 500 2669 "-" "Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0"
nextcloud          | 172.18.0.4 - - [18/Jul/2020:07:14:19 +0000] "GET /core/css/guest.css?v=ba222ded25d957b900c03bef914333cd HTTP/1.1" 200 6058 "-" "Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0"
nextcloud          | 172.18.0.4 - - [18/Jul/2020:07:14:19 +0000] "GET /core/js/dist/main.js?v=ba222ded25d957b900c03bef914333cd HTTP/1.1" 200 425692 "-" "Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0"

devofox · July 18, 2020, 8:10am

Hi @broken.pipe,

I had the same problem as you and I’m not sure what caused the sudden auth problem with the redis server. I found a workaround (assuming you are using docker-compose):

Append “command: redis-server --requirepass YOURPASSWORD” under your redis docker-compose configuration
Go to your config.php in your Nextcloud and under redis change ‘password’ => ‘false’ to ‘password’ => ‘YOURPASSWORD’.
Recreate the containers

From now on nextcloud will use a password to auth against the redis server. In my case the auth error disappeared and I could log in again.

Hope this helps!

broken.pipe · July 18, 2020, 8:43am

Perfect! Thank you very much!! Nextcloud is up and running again

I also had to add the enviroment variable to the nextcloud container:

3x3cut0r55 · August 7, 2020, 3:25pm

I have the same issue with nextcloud 19.0.1 and redis 6.0.6-alpine as two different docker containers.
but i got a differen error screen:

redis is using requirepass
and nextcloud is set to:

‘memcache.distributed’ => ‘\OC\Memcache\Redis’,
‘memcache.locking’ => ‘\OC\Memcache\Redis’,
‘filelocking.enabled’ => ‘true’,
‘redis’ =>
array (
‘host’ => ‘redis’,
‘password’ => ‘superstrongpassword’,
‘port’ => 6379,
),

if i undo the password steps i can connect but i get the ERR AUTH warnings every second on the log tab in settings!

MLaudren · August 9, 2020, 1:22pm

Hi,

I’m facing similar issue but didn’t manage to solve it either.
Have you found a solution about this?

I looked at different places in nextcloud, but the password is correct everywhere, and I am able to access the redis DB from the redis CLI.

If disabling the password, I get like 4-5 messages per second in the log console.

If you have any idea, I happy to try anything you propose.

3x3cut0r55 · August 9, 2020, 6:53pm

no i didn’t
i really tried everything. i absolutely have no idea anymore.
i’m sure redis and nextcloud are configured correctly!
i also ask for help on reddit … but still no comment!

fareszr · August 12, 2020, 8:45pm

as @broken.pipe said you need to add - REDIS_HOST_PASSWORD=YOURPASSWORD to the nextcloud container in the docker-compose file.

if you get stuck in maintenance mode go to config.php and change maintenance to 'maintenance' => false,

3x3cut0r55 · August 13, 2020, 5:04pm

thats not gonna work for me. already did this … still internal server error

Kees · August 28, 2020, 12:11pm

I had the same issue at the beginning of July. After resolving it by setting a password for my Redis container (using Docker with compose) and setting it in the Nextcloud configuration file, I had to remove all Redis password additions to resolve this Internal Server Error after upgrading to the Nextcloud 19.0.2.2 container.

Clement · September 8, 2020, 7:41pm

Ok, it worked for me to add the password like devofox sayed, and I needed to add the “REDIS_HOST_PASSWORD=YOURPASSWORD” too.

Thank you !

@3x3cut0r55, it doesn’t work for you ?

3x3cut0r55 · September 10, 2020, 7:49am

no, with and without the ENV its still the same!
i try to upgrade to 19.0.2. maybe this will help.

@Kees what do you mean with removing all password additions?

Kees · September 14, 2020, 4:32pm

As a password was required after upgrading Redis, I:

added command: redis-server --requirepass SUPER_SECRET_PASSWORD under service redis in docker-compose.yml
changed the password entry under redis to 'password' => 'SUPER_SECRET_PASSWORD' in config.php in the Nextcloud container
added environment variable REDIS_HOST_PASSWORD = SUPER_SECRET_PASSWORD under service nextcloud in docker-compose.yml

and it worked temporarily. This ceased working after a while, but then reversal of these three changes (thus “removing all password additions”) got the services back up, until last night.

Last night, I re-applied the steps as stated above and I got HTTP 500 errors without any explanation. It was without explanation, until I tried using the OCC command.

$ sudo docker-compose exec --user=www-data app php occ help
An unhandled exception has been thrown:
RedisException: WRONGPASS invalid username-password pair in /var/www/html/lib/private/RedisFactory.php:94
Stack trace:
#0 /var/www/html/lib/private/RedisFactory.php(94): Redis->auth( '"SUPER_SE...' )
#1 /var/www/html/lib/private/RedisFactory.php(108): OC\RedisFactory->create()
#2 /var/www/html/lib/private/Memcache/Redis.php(43): OC\RedisFactory->getInstance()
#3 /var/www/html/lib/private/Memcache/Factory.php(135): OC\Memcache\Redis->__construct('e65a83d136200a1...')
#4 /var/www/html/lib/private/Server.php(1025): OC\Memcache\Factory->createLocking('lock')
#5 /var/www/html/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\Server->OC\{closure}(Object(OC\Server))
#6 /var/www/html/lib/private/ServerContainer.php(124): Pimple\Container->offsetGet('OCP\\Lock\\ILocki...')
#7 /var/www/html/lib/private/Server.php(1975): OC\ServerContainer->query('OCP\\Lock\\ILocki...')
#8 /var/www/html/lib/private/Files/View.php(118): OC\Server->getLockingProvider()
#9 /var/www/html/lib/private/Server.php(302): OC\Files\View->__construct()
#10 /var/www/html/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\Server->OC\{closure}(Object(OC\Server))
#11 /var/www/html/lib/private/ServerContainer.php(124): Pimple\Container->offsetGet('OCP\\Encryption\\...')
#12 /var/www/html/lib/private/Server.php(1448): OC\ServerContainer->query('OCP\\Encryption\\...')
#13 /var/www/html/lib/base.php(844): OC\Server->getEncryptionManager()
#14 /var/www/html/lib/base.php(729): OC::registerEncryptionWrapper()
#15 /var/www/html/lib/base.php(1095): OC::init()
#16 /var/www/html/console.php(49): require_once('/var/www/html/l...')
#17 /var/www/html/occ(11): require_once('/var/www/html/c...')
#18 {main}

There I noticed an extra " character inside the password string. That was weird.

So, I changed /var/html/lib/private/RedisFactory.php (around line 94), using var_dump($var) to see what was happening.

                        if (isset($config['password']) && $config['password'] !== '') {
                                //var_dump($config['password']); // string (23) ""SUPER_SECRET_PASSWORD""
                                $config['password'] = "SUPER_SECRET_PASSWORD";
                                //var_dump($config['password']); // string (21) "SUPER_SECRET_PASSWORD"
                                $this->instance->auth($config['password']);
                        }

This seemed to fix the errors. I have still not figured out why there is an extra pair of quotation marks (") characters. I also do not recommend storing the password like this, but this seems to work, for now. If I find some time, I will do further investigation (and, if necessary, propose changes to the Nextcloud code).

If this is your issue as well, @3x3cut0r55, I am happy to assist you if needed.

P.S.: I am not really using SUPER_SECRET_PASSWORD as a password.

3x3cut0r55 · September 15, 2020, 9:50am

After some hours of research i did it and found my mistake after i saw your comment with the unhandled exception. This message i got only if the maintenance-mode is enabled!

Short answer: my redis docker container was misconfigured and i got a “connection refused” error. after i start it like this: “redis-server --requirepass SUPER_SECRET_PASSWORD” instead of writing the password in the redis.conf it run like a charm!

i didn’t have to change the RedisFactory.php at all but i also didn’t used quotation at all!

thank you a lot!

esvalot · November 11, 2020, 12:17am

Editing the REDIS configuration is not a good solution, or not in my case. I use REDIS with another containers in the same host (GitLab and 2 more aplications) if REDIS is not exposed outside, is also secure without password.

It’s a BUG in version 20.0.1.1 of Nextcloud, you need to do something in Nextcloud to fix it.
In my case I deleted provisionally 3 lines, from 93 to 95 from lib/private/RedisFactory.php

captura

kosssi · December 2, 2020, 2:03pm

This issue fix this https://github.com/nextcloud/docker/pull/1232

bego68 · December 16, 2020, 7:54am

We have fixed the problem with the password:

add REDIS_HOST_PASSWORD to file .env

env

Add REDIS_HOST_PASSWORD to docker-composer.yml

babaz · January 17, 2021, 6:32pm

Super. Thank you very much it was a nightmare hours thrown on the rubbish before finding your comment. thanks again.

robertofabrizi · March 1, 2021, 11:21am

Is that enough to enable redis? The documentation isn’t really clear as to wether or not I should also modify the config.php, or that gets automatically modified when the container starts with the redis_host env variable set (ala trusted domains).

Thank you,
Roberto

wwe · March 2, 2021, 9:07pm

yes you only need to set REDIS_HOST_PASSWORD in redis container and provide REDIS_HOST and REDIS_HOST_PASSWORD as NC variable. the ultimate test redis is in use is the real-time logging on redis side:

#live view on redis requests (${redis_host_password} comes from docker .env file)
docker exec -ti "redis_container_name" sh -c 'redis-cli -a ${REDIS_HOST_PASSWORD} MONITOR'

any interaction with NC web-interface results in some log output.

johen · January 2, 2022, 8:34pm

Also run into this problem with nextcloud after updating to version 22.2.3.0 and adding a redis server using the nextcloud docker-compose files - beside the fix to add/force a password, adding (string) to the config/redis.config.php (‘password’ => (string) getenv(‘REDIS_HOST_PASSWORD’)) also fixed the problem without forcing a password.
This change is already done in the latest docker compose config files but as the config files are in a permanent docker volume it is not updated with docker-compose pull and up -d