Skip to content

fix(mcp): allow locked exists on vector fields with index_missing - #845

Open
Purv-Kabaria wants to merge 2 commits into
redis:mainfrom
Purv-Kabaria:fix/mcp-locked-exists-vector
Open

Purv-Kabaria wants to merge 2 commits into
redis:mainfrom
Purv-Kabaria:fix/mcp-locked-exists-vector

Conversation

@Purv-Kabaria

@Purv-Kabaria Purv-Kabaria commented Nov 12, 2026 •

Copy link
Copy Markdown

Motivation

A custom tool profile whose lock.filter uses exists on a vector field fails at startup, even when the field is declared with index_missing: true. Since #771, RedisVL emits INDEXMISSING on vector fields correctly, and Redis answers ismissing(@field) on them, so a locked "only documents that have an embedding" filter is a query that works. Closes #788.

Changes

_validate_locked_exists_fields in redisvl/mcp/tools/profiles.py no longer rejects vector fields outright. A vector field now goes through the same index_missing check as every other field type: exists is accepted when the field declares index_missing, and a field without it still fails at startup with the existing "not indexed with INDEXMISSING" error. The unused vector_field_names parameter is removed.

The new unit tests cover a tag field and a vector field, each with and without index_missing.

Release Notes

MCP custom tool profiles can lock an exists filter on a vector field declared with index_missing.

Reviewer Brief

Start here: redisvl/mcp/tools/profiles.py, _validate_locked_exists_fields(): vector fields now use the same index_missing guard as other fields. tests/integration/test_mcp/test_locked_exists_vector.py exercises the real SearchIndex.create path with a document with an embedding and one without.

Not verified: cluster mode; the full integration suite; live Redis versions other than 8.10.3 in this review.

Beyond CI: 598 MCP unit tests and all 12 new integration cases pass against redis:8.10.3, covering filter and KNN queries on HASH/JSON with FLAT, HNSW and SVS-VAMANA; mypy passes for 125 source files and isort/black pass.

Reviewed 9620555 ? Codex ? lenses: correctness, security, design, tests, docs, Redis behaviour; independent challenger pass


Note

Low Risk
Narrows startup validation for MCP custom tool profiles only; no auth or query-path behavior change beyond accepting previously-rejected valid configs.

Overview
MCP profile startup validation no longer blocks exists locked filters on vector fields when the field has index_missing. _validate_locked_exists_fields drops the hard vector-field rejection and applies the same INDEXMISSING rule as for tag/text fields, so profiles like “only docs with an embedding” can register when the schema supports ismissing(@embedding).

Unit tests parametrize tag and vector fields with and without index_missing. New Redis integration tests assert (-ismissing(@embedding)) works as a filter and as a KNN pre-filter across vector algorithms and hash/json storage.

Reviewed by Cursor Bugbot for commit 9620555. Bugbot is set up for automated code reviews on this repo. Configure here.

A locked exists filter on a vector field was rejected at startup even when the field declares index_missing. Since redis#771, Redis accepts INDEXMISSING on vector fields and answers ismissing() on them, so vector fields now go through the same index_missing check as every other field type.

Closes redis#788
Copilot AI balanced review requested due to automatic review settings November 12, 2026 09:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@Purv-Kabaria

Copy link
Copy Markdown
Author

Hi! Following up on the locked exists filter fix for #788. The PR includes the regression test, the 598 passing MCP tests, and the live Redis 8.10.3 check. Bugbot is green and there are no requested changes so far. Happy to clarify anything in the Reviewer Brief or adjust the approach. Thanks for taking a look when you can!



@pytest.mark.parametrize("field_name", ["category", "embedding"])
def test_validate_profile_against_schema_accepts_locked_exists_with_index_missing(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test lets the profile through on the grounds that Redis answers ismissing() on the vector field. Nothing in tests/integration checks that, though: ismissing doesn't appear there at all. If FT.CREATE ever stopped carrying INDEXMISSING for a vector field, this validator would still pass the profile, and every call would fail as a retryable backend error. That's the failure mode the validator exists to prevent.

Could you add one integration test? It would create a vector field with index_missing: True, load one document with an embedding and one without, and assert that (-ismissing(@embedding)) returns only the first, both as a filter and as a KNN prefilter. I checked that behaviour by hand on 8.2, 8.4 and 8.10, for FLAT, HNSW and SVS-VAMANA, on HASH and JSON, so the test should pass on every CI version.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Purv-Kabaria once this change is in, I can merge the PR 👍

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9620555. Added integration coverage using RedisVL's own SearchIndex.create: one document with an embedding and one without, asserting that (-ismissing(@embedding)) selects only the first, both as a filter and as a KNN prefilter. The matrix covers HASH/JSON with FLAT, HNSW and SVS-VAMANA. All 12 cases and the 598 MCP unit tests pass against redis:8.10.3; formatting and mypy pass. The Reviewer Brief is updated. Thanks for spelling out the backend failure this needs to catch.

@vishal-bala vishal-bala added the auto:patch Increment the patch version when merged label Nov 16, 2026

@chrikrah chrikrah left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Purv-Kabaria this is ready to go in. I built the index with RedisVL's own SearchIndex.create, the path #771 changed, on redis:8.2.11, older than your 8.10.3. The locked filter works there too.

# head 8f8c0fe, uv sync --frozen --extra mcp, Python 3.12
# redis:8.2.11 container, SearchIndex.create, one doc with an embedding and one without
index_missing=True:  profile accepted; '(-ismissing(@embedding))' -> 1 rows
index_missing=False: profile rejected (ValueError)
# not run: Redis 7.4 or a cluster

The probe is ten lines. Want it as an integration test?

@vishal-bala vishal-bala added auto:minor Increment the minor version when merged and removed auto:patch Increment the patch version when merged labels Nov 19, 2026

@vishal-bala vishal-bala left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per my last review, these changes are ready except for one last request - once it's addressed, we can merge this. See my comment here: #845 (comment)

Signed-off-by: Purv Kabaria <purvkabaria@gmail.com>
Copilot AI balanced review requested due to automatic review settings November 21, 2026 06:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@Purv-Kabaria

Copy link
Copy Markdown
Author

Thanks for checking the SearchIndex.create path on 8.2.11, @chrikrah. I added the integration coverage in 9620555, including both the filter and KNN-prefilter assertions on HASH/JSON for FLAT, HNSW and SVS-VAMANA. All 12 cases pass against redis:8.10.3 here. This gives CI a regression check for the behavior you probed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto:minor Increment the minor version when merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MCP: allow locked exists filters on vector fields declared with index_missing

4 participants