Sitelet https://github.com/predis/predis/pull/1486
Skip to content

Read operations from a random Redis Cluster node - #1486

Open
disc wants to merge 8 commits into
predis:v2.xfrom
disc:read-from-cluster-replicas
Open

disc wants to merge 8 commits into
predis:v2.xfrom
disc:read-from-cluster-replicas

Conversation

@disc

@disc disc commented Oct 28, 2024 •

Copy link
Copy Markdown

Here's a proposal of scaling read operations in Redis Cluster by sending them to cluster's master or replicas nodes with a READONLY command as init command for any connection.

The option scaleReadOperations supports two modes:

  • replicas - send read operations to replicas nodes only
  • random - send read operations to master and replicas nodes
'scaleReadOperations' => 'replicas', // or 'random'

How to use:

$parameters = [
    'tcp://172.38.0.11:6379',
    'tcp://172.38.0.12:6379',
    'tcp://172.38.0.13:6379',
    'tcp://172.38.0.14:6379',
    'tcp://172.38.0.15:6379',
    'tcp://172.38.0.16:6379',
];
$options = [
    'cluster' => 'redis',
    // Enables reading from Redis Cluster replica nodes by mapping masters and replicas.
    // and sending READONLY to any cluster node upon connection
    // works only when using native Redis cluster mode (`'cluster' => 'redis'`)
    // Supports two modes:
    // `replicas` - send read operations to replicas nodes only
    // `random` - send read operations to master and replicas nodes
    'scaleReadOperations' => 'replicas',
];

$client = new Predis\Client($parameters, $options);
// Builds slots map and master <-> replicas relations map
$client->getConnection()->askSlotMap();

// GET command will be executed on replica node
$result = $client->get('my-key');

Expected commands in monitor (in case of replicas mode)

  1. READONLY to any random node (entry point from parameters)
  2. CLUSTER SLOTS to a selected connection above
  3. READONLY to a random replica node (if it hasn't been sent before)
  4. GET my-key to a selected replica connection above

@disc
disc force-pushed the read-from-cluster-replicas branch from 7ab8149 to 0dfc7f5 Compare October 28, 2024 09:46
@coveralls

coveralls commented Oct 28, 2024 •

Copy link
Copy Markdown

Coverage Status

coverage: 80.846% (+0.3%) from 80.539%
when pulling 70fc685 on disc:read-from-cluster-replicas
into 1726db0 on predis:v2.x.

@disc
disc force-pushed the read-from-cluster-replicas branch 4 times, most recently from 0a25c59 to 153589f Compare October 28, 2024 12:22
@disc
disc marked this pull request as ready for review October 28, 2024 15:00
@disc
disc requested a review from tillkruss as a code owner October 28, 2024 15:00
```
$options = [
    'cluster' => 'redis',
    // Enables reading from Redis Cluster replica nodes by mapping masters and replicas.
    // and sending READONLY to any cluster node upon connection
    // works only when using native Redis cluster mode (`'cluster' => 'redis'`)
    'readonly' => true,
];
```
@disc
disc force-pushed the read-from-cluster-replicas branch from 153589f to 13a6171 Compare October 28, 2024 19:05
@disc disc changed the title Read operations from a random Redis Cluster replica Read operations from a random Redis Cluster node Nov 2, 2024
@disc
disc force-pushed the read-from-cluster-replicas branch from 8b88d88 to ca4244b Compare November 2, 2024 19:10
@disc

disc commented Nov 2, 2024

Copy link
Copy Markdown
Author

Hello @tillkruss and @vladvildanov. I've just slightly improved PR by supporting a few modes for scaling read operations in redis-cluster. In this version, read operations can be scaled between master and replica nodes in a random mode, or just by replicas in replicas mode.

… a few modes:

  * `replicas` - send read operations to replicas nodes only
  * `random` - send read operations to master and replicas nodes
```
$options = [
    'cluster' => 'redis',
    // Enables reading from Redis Cluster replica nodes by mapping masters and replicas.
    // and sending READONLY to any cluster node upon connection
    // works only when using native Redis cluster mode (`'cluster' => 'redis'`)
    // Supports two modes:
    // `replicas` - send read operations to replicas nodes only
    // `random` - send read operations to master and replicas nodes
    'scaleReadOperations' => 'replicas',
];
```

Moved a logic related to detection of an operation type (readonly or not) into a read connection selector
@disc
disc force-pushed the read-from-cluster-replicas branch from ca4244b to 601d3b5 Compare November 2, 2024 19:17
@disc

disc commented Dec 7, 2024

Copy link
Copy Markdown
Author

Hello @tillkruss. Could you look into my changes and share your opinion about the feature?

@tillkruss

Copy link
Copy Markdown
Member

Looks like not breaking changes, which is great. I'd like @vladvildanov to review this and give his thumbs up.

@disc

disc commented Jan 15, 2025

Copy link
Copy Markdown
Author

Looks like not breaking changes, which is great. I'd like @vladvildanov to review this and give his thumbs up.

Hello @vladvildanov. Have you had a chance to look into the suggested changes?

@vladvildanov

Copy link
Copy Markdown
Contributor

@disc Sorry for late response guys, I'm on it

@vladvildanov

Copy link
Copy Markdown
Contributor

@disc @tillkruss

I see some major issues with this PR that will only worsen an existing problems in a client:

  1. Current cluster implementation are not designed to support master/slave replication

Adding new connection manager (e.g ReadConnectionSelector class) on top of existing RedisCluster, just to manage replica connections is a workaround that ignores the general issue that master/slave replication isn't supported. Load balancing is one of the features that comes up with replication, but the problem should be scoped accordingly - "we need to add a support for master/slave replication in Cluster API". And the access to master/replica nodes should be managed by Cluster object.

https://github.com/predis/predis/blob/v2.x/src/Connection/Cluster/RedisCluster.php#L287

  1. ReadonlyOperationDetector class

I disagree that we need an external class that should distinguish read and write commands. The information about the command mode should be exposed by command object itself, so the CommandInterface should be extended with something like getMode() method. In other case we need to change a code in different places when we add a new command, we already have the same detector for Replication:

https://github.com/predis/predis/blob/v2.x/src/Replication/ReplicationStrategy.php#L213

Apart of it we have to specify separately if command is supported by cluster:

https://github.com/predis/predis/blob/v2.x/src/Cluster/ClusterStrategy.php#L41

If we will add another detector, each new command that will be added needs to be appended to each of this detectors in 3 different places. So my point that we shouldn't make it even worse.

  1. ReadConnectionSelector class

As I mentioned above, a separate class that will manage connections seems like workaround, connections are already managed by RedisCluster and new object should not share this responsibilities, also it will be aligned with Replication logic.

https://github.com/predis/predis/blob/v2.x/src/Connection/Replication/MasterSlaveReplication.php#L166

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

4 participants