Redis cluster support and configuration

installation
#1

Hi everybody,

I can’t figure out how to configure nextcloud to use a external redis cluster with master and slaves as a cache.
I have read that this is supported, but despite of very long and deep searches on the web, i couldn’t find any explanation on how to configure it.

I have a two nodes redis cluster working, with a master and a slave. If I configure the master node in the nextcloud config.php file, it works like a charm. But if the master redis node fails, nothing seems to handle the redirection on the old slave and promote it as new master.

Maybe I should use somewhere redis-sentinel to promote the slave to master, but how announce it to nextcloud ?

If someone has informations about that, or links where I can find it, thanks a lot !

Best regards.

#2

Maybe I found the beginning of a response by exploring files on the Nextcloud Server Github page.

I found there a commit called ‘redis cluster support added’, where there are modifications in the config.sample.php file talking about Redis clustering.

There is this section in that file :

/**
 * Connection details for a Redis Cluster
 *
 * Only for use with Redis Clustering, for Sentinel-based setups use the single
 * server configuration above, and perform HA on the hostname.
 *
 * Redis Cluster support requires the php module phpredis in version 3.0.0 or higher.
 *
 * Available failover modes:
 *  - \RedisCluster::FAILOVER_NONE - only send commands to master nodes (default)
 *  - \RedisCluster::FAILOVER_ERROR - failover to slaves for read commands if master is unavailable
 *  - \RedisCluster::FAILOVER_DISTRIBUTE - randomly distribute read commands across master and slaves
 */
'redis.cluster' => [
	'seeds' => [ // provide some/all of the cluster servers to bootstrap discovery, port required
		'localhost:7000',
		'localhost:7001'
	],
	'timeout' => 0.0,
	'read_timeout' => 0.0,
	'failover_mode' => \RedisCluster::FAILOVER_DISTRIBUTE
],

I will rebuild my Redis clustering setup with the new clustering capabilities of Redis 3.0 (which seems not use redis-sentinel to promote masters), and then install a nextcloud server using these settings to check if all is working, then I’ll let you know the results.

1 Like
#3

I have implemented Netflix Dynomite ( https://github.com/Netflix/dynomite ) as an active-active Redis cluster. Works nice and you don’t have to do any failover management/configuration in Nextcloud.

#4

Did you have any luck?

#5

Good day Ruud,

after understanding that a redis cluster needs at least 3 working master nodes to work - too many for my resources - I started looking for an alternative.
I found your post, and set up a dynomite cluster.
it consists of two racks of three nodes each:
server1: rack1-node1, rack2-node3
server2: rack1-node2, rack2-node1
server3: rack1-node3, rack2-node2
The three nodes of each rack cover the keyspace. The second rack replicates the first.

In Nextcloud, I thought that setting the nodes of both racks as a cluster would be enough to get redundancy and load balancing, but Nextcloud complains and does not work.
I tried to remove the three nodes of second rack from the configuration, but no luck.
It only worked when I used a single redis server configuration pointing at one of dynomite nodes/ports

If that node fails, data are still available, but Nextcloud would not be able to retrieve them.

Is this assumprion correct?

If so, how do I configure Nextcloud (or setup dynomite) to get automatic failover in case of node failure?

Regards.