Skip to content

fix(index): read cluster FT.INFO on RESP3 and initialise fresh async cluster clients - #795

Merged
vishal-bala merged 7 commits into
mainfrom
fix/cluster-ft-info-resp3-async-init
Nov 19, 2026
Merged

vishal-bala merged 7 commits into
mainfrom
fix/cluster-ft-info-resp3-async-init

Conversation

@vishal-bala

@vishal-bala vishal-bala commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

When the requires_cluster suite was run against a real six-node cluster (Redis 8.4, redis-py 8.1.0), three tests failed because of two bugs in the cluster branch of _info(). CI never runs these tests, so neither bug was caught:

  • test_redis_cluster_support.py::test_search_index_cluster_info fails with KeyError: 1.
  • test_async_search_index_cluster_info and test_async_search_index_client fail with IndexError: Cannot choose from an empty sequence, and the cleanup then fails with 'NoneType' object has no attribute 'connection_kwargs'.

Anyone who passes a freshly built AsyncRedisCluster to AsyncSearchIndex hits the second bug on create(). The first bug hits any sync cluster user on redis-py 8, which uses RESP3 by default.

Changes

RESP3 replies to the cluster FT.INFO

On a cluster, SearchIndex._info() and AsyncSearchIndex._info() send a raw FT.INFO to a single node, so redis-py's search parser never sees the reply. That reply is a flat list on RESP2 and a map on RESP3, and make_dict() only accepted the list. Both versions of _info() now accept either shape. The nested sections stay maps on RESP3, and every internal consumer already handles them: convert_index_info_to_schema, the MCP schema loader, migration and the CLI.

Fresh async cluster clients

redis-py's async cluster client finds out about its nodes on its first command. Until then, get_random_node() raises IndexError and get_default_node() returns None. AsyncSearchIndex._get_client() now calls initialize() on cluster clients; when the client is already initialised, that call does nothing. This also fixes delete(), listall() and search() when one of them is the first call. _info() initialises the client as well, because from_existing() and the MCP server call it directly without going through _get_client().

Index metadata is read from the default node

_info() used to send FT.INFO to get_random_node(), which can return a replica. FT.CREATE, FT.DROPINDEX and FT.SEARCH all go to the default node, a primary. Redis replicates asynchronously, so a replica can answer "no such index" straight after create(). That is how the nightly cluster job failed on redis-py 8.1.0. Sync and async _info() now ask the default node. Both listall() variants do the same: the async one used a random node too, and on redis-py 7.x and 8.x the sync one raised RedisClusterException on a cluster, because redis-py has no routing for a keyless FT._LIST.

Tests

tests/unit/test_cluster_index_info.py runs in default CI. Its stand-in clients are built with create_autospec and raise IndexError until initialize() has been called. The tests feed both RESP2 and RESP3 replies. Two existing mocks needed an awaitable initialize.

Notes

  • Errors from cluster initialisation are not wrapped. _get_client() runs outside each method's try, so if an injected AsyncRedisCluster can't reach its startup node, exists() and the other async methods raise redis-py's own ConnectionError, not RedisSearchError. This follows the existing convention: only the Redis command is wrapped, never the step that gets a client. A client built from a URL already fails the same way, because it sends CLIENT SETINFO when it connects.
  • A stale default node after a failover is documented, not fixed. After a failover, redis-py keeps the demoted node as its default node until a keyed command gets MOVED. Until then, index writes raise ReadOnlyError and index reads come from the replica. This already applied to create(), delete() and search() on main. Catching stale reads would need a ROLE round trip on every call. The write half belongs in redis-py, whose Sentinel client already handles READONLY from a demoted master. docs/concepts/search-and-indexing.md now describes the behaviour and how to recover.

Reviewer Brief

Start here: SearchIndex._info() and AsyncSearchIndex._info() in redisvl/index/index.py. Every cluster read of index metadata now goes through one shape-agnostic FT.INFO sent to the default node. exists(), info(), create() and from_existing() all depend on it.

Where it could fail

  • AsyncSearchIndex._get_client(): it now calls initialize() on every injected AsyncRedisCluster. Check that nothing depends on a cluster client staying uninitialised until its first command.
  • _info() on a cluster: on RESP3, which redis-py 8 uses by default, info() returns attributes and index_definition as maps. A standalone client in redis-py 8's default legacy mode returns them as flat lists. Every internal consumer accepts both shapes, but an external caller of info() might rely on the list shape.

Not verified: the Linux nightly job after these commits, since it hasn't been re-dispatched. The replica lag behind the CI failure didn't reproduce locally (0 misses in 900 FT.INFO calls to replicas straight after FT.CREATE), because all six local nodes share one container. I also haven't checked Redis versions other than 8.4, or Python versions other than 3.13 and 3.14.

Beyond CI:

  • Cluster suite: against a local six-node Redis 8.4 cluster, test_redis_cluster_support.py, test_cluster_pipelining.py and test_async_cluster_connection.py gave 24 passed on each of redis-py 8.1.0, 7.4.1 and 6.4.0. That ran on 310af379, before the sync listall() change. The router test errored each time because sentence-transformers isn't installed.
  • Sync listall(): on a live cluster it raised RedisClusterException on redis-py 8.1.0 and 7.4.1 before 62164c00, and returns normally after it on all three versions.
  • Protocol script: I ran it with protocol=2, with protocol=3 and with the default. It covers info(), a from_existing() round trip, and fresh async clients where listall(), create() or delete() is the first command. It passed on all three redis-py versions, on 8e075faf.
  • Unit tests: with index.py from before 310af379, 7 of the new or changed unit tests fail. The sync listall() unit test wasn't run against the old code.

Since the last brief: d5d2bef1 adds the cluster failover section to docs/concepts/search-and-indexing.md; there is no code change. The branch was also rebased onto main after #794 merged. a5098cbf corrects that section to say create() and delete() raise RedisSearchError, with the ReadOnlyError as its cause.

Reviewed 8e075faf and 310af379, fixes in b2626140 and 62164c00 · Claude Code (claude-opus-5-5) · lenses: correctness, security, design, tests, docs, Redis behaviour (310af379: correctness, Redis behaviour)


Note

Medium Risk
Changes core cluster routing and FT.INFO parsing used by exists(), create(), and from_existing(); stale-node behavior after failover is documented but not auto-remediated.

Overview
Fixes Redis Cluster index metadata and lifecycle bugs that showed up under redis-py 8 (RESP3) and with uninitialized async cluster clients.

Cluster FT.INFO / listall(): Raw FT.INFO and FT._LIST now target the client's default node (same as FT.CREATE / FT.DROPINDEX) instead of a random node, avoiding false "index missing" and stale listings when a replica has not caught up. Replies accept both RESP2 lists and RESP3 maps before normalization.

Async cluster: AsyncSearchIndex._get_client() and AsyncSearchIndex._info() call initialize() on injected AsyncRedisCluster instances so get_default_node() is valid on first use (create(), delete(), listall(), from_existing(), etc.).

Docs: Adds guidance on index commands after a cluster failover (stale default node, read-only writes, replica lag on reads, recovery).

Tests: New default-CI unit coverage for RESP2/RESP3 info parsing, primary-only routing, fresh async clients, and cluster listall().

Reviewed by Cursor Bugbot for commit a5098cb. Bugbot is set up for automated code reviews on this repo. Configure here.

@vishal-bala vishal-bala added the auto:patch Increment the patch version when merged label Oct 30, 2026
vishal-bala added a commit that referenced this pull request Oct 30, 2026
Co-Authored-By: Claude <noreply@anthropic.com>
@vishal-bala
vishal-bala changed the base branch from main to test/cluster-pipelining-client October 30, 2026 10:57
@vishal-bala
vishal-bala added this pull request to stack #797 October 30, 2026 10:57
vishal-bala added a commit that referenced this pull request Oct 30, 2026
Co-Authored-By: Claude <noreply@anthropic.com>
@vishal-bala
vishal-bala removed this pull request from stack #797 October 30, 2026 12:39
@vishal-bala
vishal-bala added this pull request to stack #801 October 30, 2026 12:39
@vishal-bala
vishal-bala marked this pull request as ready for review November 9, 2026 14:00
Base automatically changed from test/cluster-pipelining-client to main November 11, 2026 09:29
…cluster clients

The cluster branch of SearchIndex._info() and AsyncSearchIndex._info() sends
a raw FT.INFO to one node, so redis-py's search parser never sees the reply.
On RESP3, the redis-py 8 default, that reply is a map, and make_dict() raised
KeyError: 1. Accept either shape.

The async cluster client discovers its nodes on its first command, so on a
fresh client get_random_node() raised IndexError and get_default_node()
returned None. That broke create(), and delete() then failed with
"'NoneType' object has no attribute 'connection_kwargs'". Initialise the
client in _get_client(), and in _info() for callers that bypass it.
_info() sent FT.INFO to get_random_node(), which picks from every node in
the cluster, replicas included. FT.CREATE goes to the default node, a
primary, and Redis replicates asynchronously, so a replica can answer
"no such index" straight after create(). The nightly cluster job hit this
on redis-py 8.1.0, with FT.INFO landing on the replica at port 7005.

Ask the default node instead, the node FT.CREATE, FT.DROPINDEX and
FT.SEARCH already use. Async listall() had the same random-node read and
now does the same; sync listall() already reached the default node through
redis-py's keyless routing.
The previous commit said sync listall() already reached the default node
through redis-py's keyless routing. That holds only on redis-py 6.x. On 7.x
and 8.x, FT._LIST has no routing entry, so a keyless call on a cluster
raises RedisClusterException ("No way to dispatch this command to Redis
Cluster. Missing key"). Target the default node explicitly, as the async
twin and _info() do.
@vishal-bala
vishal-bala force-pushed the fix/cluster-ft-info-resp3-async-init branch from 810f829 to c688b1d Compare November 11, 2026 09:29

@limjoobin limjoobin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one comment regarding calling the primary node after failover. The fix looks good to me otherwise!

Comment thread redisvl/index/index.py Resolved
Index commands go to redis-py's default node, which redis-py re-points
only after a keyed command is redirected or a connection fails. Until
then, writes fail with ReadOnlyError and reads come from the demoted
node. Say so, and how to recover.
Co-Authored-By: Claude <noreply@anthropic.com>
@raphaeldelio
raphaeldelio self-requested a review November 19, 2026 10:20

@raphaeldelio raphaeldelio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread docs/concepts/search-and-indexing.md Resolved
…a failover

They wrap redis-py's ReadOnlyError in RedisSearchError, so code that
catches ReadOnlyError never runs. Say what to catch.
@vishal-bala
vishal-bala merged commit f78b519 into main Nov 19, 2026
58 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto:patch Increment the patch version when merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants