Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/pre-pr-review/references/correctness.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Find changes that make the code behave wrongly for some input, state or environm
- **Inputs the fix wasn't aimed at.** For each changed condition, try `None`, an empty string, `0`, `False` and an empty collection, and compare the result with the base (the PR's target branch). A key that is present with the value `None` is not a missing key, and a truthiness test treats a genuine zero as unset. Check also which error wins when two checks fail: moving a new check ahead of an old one changes the exception callers see.
- **Redis replies.** The code has to handle RESP2 and RESP3 reply shapes, and both hash and JSON storage.
- **Result limits.** Default result counts (`num_results`, `LIMIT`, `MAXSEARCHRESULTS`) silently truncate. Check any read that is meant to return everything.
- **Cluster mode.** Multi-key operations, `SCAN` and pipelines behave differently on a cluster, and partial failures are possible. CI never runs cluster tests, so reason about these cases from the code. If you can't settle one that way, put it under "Not verified" in the Reviewer Brief.
- **Cluster mode.** Multi-key operations, `SCAN` and pipelines behave differently on a cluster, and partial failures are possible. Pull-request CI never runs cluster tests (they run only nightly), so reason about these cases from the code. If you can't settle one that way, put it under "Not verified" in the Reviewer Brief.
- **Client ownership.** Who creates, owns and closes a Redis client, and what happens on reconnect, especially for async clients reused across event loops.

## Depth
Expand Down
2 changes: 1 addition & 1 deletion .agents/skills/pre-pr-review/references/tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ A file that fails at collection on the base proves nothing either way. Check its
- **Missing critical paths.** Error paths, boundary values (empty, one, exactly the limit, one past it), and each sync or async variant and storage type the change does branch on. Every method the change decorates or rewrites should be called by some test.
- **Tests of private details.** Calls counted on a private method, a private class imported, handlers counted, exact log text or wire format. Test through the public entry point that callers use, so the test survives a restructuring.
- **Tests that certify behaviour nobody wants to promise.** A side effect of the current implementation, or a value the server rejects in practice, pinned as if it were a contract.
- **Areas CI can't reach.** `requires_cluster` tests are skipped in CI, and `requires_api_keys` tests only run with keys. If the change is in one of those areas, ask for a hermetic unit test, using mocks or a fake client, that pins the behaviour. Otherwise, make sure the Reviewer Brief lists it under "Not verified".
- **Areas CI can't reach.** `requires_cluster` tests are skipped in pull-request CI (they run only nightly), and `requires_api_keys` tests only run with keys. If the change is in one of those areas, ask for a hermetic unit test, using mocks or a fake client, that pins the behaviour. Otherwise, make sure the Reviewer Brief lists it under "Not verified".
- **Weakened assertions.** Loosened expectations, added skips or broader `pytest.raises` that make a failing test pass rather than fixing the code.
- **Dead scaffolding.** A mock return value that an `assert_not_called` makes unreachable, an unused parametrize column, a hand-written stand-in class where `types.SimpleNamespace` does the job, a skip condition copied four times where one module-level mark would do.
- **Placement.** Pure logic belongs in `tests/unit/`, and anything needing Redis in `tests/integration/`, in the file that already covers that class. Integration tests share a container, so check that new ones clean up their keys and indexes.
Expand Down
78 changes: 78 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,85 @@
run: |
make test

cluster-tests:
name: Cluster Tests - redis-py ${{ matrix.redis-py-version }}
# Nightly and on demand only, to keep the cluster's setup time off PRs.
# No other job passes `--run-cluster-tests`, so this is the only one that
# runs `requires_cluster` tests.
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
env:
HF_HOME: ${{ github.workspace }}/hf_cache
REDIS_PY_VERSION: ${{ matrix.redis-py-version }}
# Required: without it `uv run` re-syncs the venv to uv.lock and reverts
# the redis-py pin set below.
UV_NO_SYNC: "1"
strategy:
fail-fast: false
matrix:
# Cluster replies differ by redis-py version (8.x defaults to RESP3),
# so cover each major the `test` job does. The Redis image is pinned
# in tests/cluster-compose.yml.
redis-py-version: ["6.x", "7.x", "8.x"]
steps:
- name: Check out repository
uses: actions/checkout@v6

- name: Cache HuggingFace Models
uses: actions/cache@v4
with:
path: hf_cache
key: ${{ runner.os }}-hf-cache

- name: Install Python
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Install uv
uses: astral-sh/setup-uv@v10.0.0
with:
version: ${{ env.UV_VERSION }}
enable-cache: true
python-version: ${{ env.PYTHON_VERSION }} # sets UV_PYTHON
cache-dependency-glob: |
pyproject.toml
uv.lock

- name: Install dependencies
run: |
set -euo pipefail

uv sync --all-extras --frozen

# Keep in sync with the `test` job's install rules.
case "$REDIS_PY_VERSION" in
"6.x") spec="redis>=6,<7" ;;
"7.x") spec="redis>=7,<8" ;;
"8.x") spec="redis>=8.1,<9" ;;
*)
echo "::error title=Unhandled redis-py-version::Matrix value '${REDIS_PY_VERSION}' has no install rule -- add a case branch in .github/workflows/test.yml."
exit 1
;;
esac

uv pip install "$spec"
uv run python -c 'import redis; print("redis-py:", redis.__version__)'

- name: Resolve host.docker.internal on the runner
# The cluster nodes announce host.docker.internal (see
# tests/cluster-compose.yml), so MOVED redirects point the test client
# there. Linux runners do not resolve that name by default.
run: |
echo "127.0.0.1 host.docker.internal" | sudo tee -a /etc/hosts

- name: Run cluster tests
# -n 0: the cluster publishes fixed ports 7001-7006, so per-worker
# clusters under xdist would collide.
run: |
uv run python -m pytest -n 0 --log-level=CRITICAL --run-cluster-tests -m requires_cluster tests/

notebooks:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium test

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
name: Notebook Validation
runs-on: ubuntu-latest
# Ordered after the HF cache is primed, not gated on it; see the `test` job.
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,4 +172,4 @@ Cursor Bugbot reviews every PR, and humans may comment too. Handle each point as
## Honesty rules

- Never claim a check in the Reviewer Brief unless you ran it and saw its output.
- Say what you didn't check. Cluster tests and tests that need API keys don't run in CI, so name them when the change touches those paths.
- Say what you didn't check. Cluster tests don't run in pull-request CI (only nightly), and tests that need API keys don't run in CI, so name them when the change touches those paths.
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ belonging to other workers and other pull requests' CI runs. See the module
docstring in `tests/integration/test_langcache_semantic_cache_integration.py` for a
worked example.

**Note:** Tests marked `requires_cluster` only run when you pass `--run-cluster-tests`. The cluster itself is provisioned for you by the `redis_cluster_container` fixture in `tests/conftest.py`, so Docker needs to be running.
**Note:** Tests marked `requires_cluster` only run when you pass `--run-cluster-tests`. The cluster itself is provisioned for you by the `redis_cluster_container` fixture in `tests/conftest.py`, so Docker needs to be running. The nodes announce themselves as `host.docker.internal`, so the host running the tests must resolve that name too: add `127.0.0.1 host.docker.internal` to `/etc/hosts` if it doesn't already. CI runs these tests nightly, in the `cluster-tests` job of `.github/workflows/test.yml`, not on pull requests.

## Documentation

Expand Down
24 changes: 24 additions & 0 deletions tests/cluster-compose.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
# Every node announces host.docker.internal, which is the address it gossips to
# its peers and hands to clients in MOVED redirects, so the name has to resolve
# both inside the containers and on the host running the tests.
# - Inside: Docker Desktop provides the name; Linux Docker Engine (GitHub's
# ubuntu runners) needs the `extra_hosts` mapping on every container below.
# - On the host: add `127.0.0.1 host.docker.internal` to the hosts file, as the
# `cluster-tests` job in .github/workflows/test.yml does. macOS does not
# resolve the name by default either.
services:
redis-node-1:
image: "redis:8.4"
Expand All @@ -12,6 +20,8 @@ services:
- --cluster-announce-port 7001
- --cluster-announce-bus-port 17001
- --protected-mode no
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "7001:7001"
- "17001:17001"
Expand All @@ -31,6 +41,8 @@ services:
- --cluster-announce-port 7002
- --cluster-announce-bus-port 17002
- --protected-mode no
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "7002:7002"
- "17002:17002"
Expand All @@ -50,6 +62,8 @@ services:
- --cluster-announce-port 7003
- --cluster-announce-bus-port 17003
- --protected-mode no
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "7003:7003"
- "17003:17003"
Expand All @@ -69,6 +83,8 @@ services:
- --cluster-announce-port 7004
- --cluster-announce-bus-port 17004
- --protected-mode no
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "7004:7004"
- "17004:17004"
Expand All @@ -88,6 +104,8 @@ services:
- --cluster-announce-port 7005
- --cluster-announce-bus-port 17005
- --protected-mode no
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "7005:7005"
- "17005:17005"
Expand All @@ -107,6 +125,8 @@ services:
- --cluster-announce-port 7006
- --cluster-announce-bus-port 17006
- --protected-mode no
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "7006:7006"
- "17006:17006"
Expand All @@ -115,6 +135,10 @@ services:

redis-cluster-setup:
image: "redis:8.4"
# `redis-cli --cluster create` reconnects to every node at its announced
# address once the nodes have met, so this container needs the name too.
extra_hosts:
- "host.docker.internal:host-gateway"
depends_on:
- redis-node-1
- redis-node-2
Expand Down
7 changes: 4 additions & 3 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,10 @@ def redis_cluster_container(worker_id):
compose_file = os.path.join(
os.environ.get("GITHUB_WORKSPACE", pwd), "tests", "cluster-compose.yml"
)
os.environ["COMPOSE_PROJECT_NAME"] = (
project_name # For docker compose to pick it up if needed
)
# Every command below passes `-p project_name`. Don't export it as
# COMPOSE_PROJECT_NAME: `redis_container`'s testcontainers commands read
# that variable, so `redis_url` would then look for the standalone `redis`
# service in the cluster's project and fail.
# Cluster tests use a pinned Redis 8 image for consistency.
os.environ.setdefault("REDIS_IMAGE", "redis:8.4")

Expand Down
25 changes: 11 additions & 14 deletions tests/integration/test_redis_cluster_support.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
"""Tests for Redis Cluster support in RedisVL."""

import asyncio

import pytest
from redis import Redis
from redis.asyncio.cluster import RedisCluster as AsyncRedisCluster
Expand Down Expand Up @@ -408,19 +406,18 @@ async def test_embeddings_cache_aclear_multipage_cluster(
pipe.set(f"{noise_prefix}:{i}", "keep")
await pipe.execute()

# Seeded with aset, not amset: amset silently writes nothing on an
# async cluster client (it awaits the pipeline returned by the queueing
# call, which drains the queue). That is a separate bug; this test is
# about aclear, so don't let it depend on the broken path.
await asyncio.gather(
*(
cache.aset(
content=f"content-{i}",
model_name="test",
embedding=[0.1, 0.2, 0.3],
)
# Seed with one amset (one pipeline), not concurrent asets: hundreds of
# pipelines in flight exceed redis-py 8's default of 100 connections
# per node, and its pipeline retry then closes the shared client.
await cache.amset(
[
{
"content": f"content-{i}",
"model_name": "test",
"embedding": [0.1, 0.2, 0.3],
}
for i in range(CLEAR_CACHE_KEYS)
)
]
)
assert await _acount_keys(cluster_client, f"{name}:*") == CLEAR_CACHE_KEYS

Expand Down
Loading