Repository navigation
feat(schema): support HNSW reranking in Flex databases (FDE-227) - #861
Draft
vishal-bala wants to merge 4 commits into
Draft
vishal-bala wants to merge 4 commits into
vishal-bala wants to merge 4 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #861 +/- ##
==========================================
+ Coverage 79.76% 80.01% +0.24%
==========================================
Files 121 121
Lines 14852 14854 +2
Branches 2599 2601 +2
==========================================
+ Hits 11847 11885 +38
+ Misses 2291 2282 -9
+ Partials 714 687 -27
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Creating a disk HNSW index in a Flex database requires an explicit
RERANKsetting. RedisVL had no way to express that setting in a vector field schema, so it could not generate the requiredFT.CREATEcommand. Loading an existing index also lost its reranking setting and any nondefaultEF_RUNTIMEvalue reported by Redis.Changes
Configure reranking per HNSW field
Set
rerank: trueorrerank: falsein the field’sattrsto emitRERANK TRUEorRERANK FALSE. Omittingrerankleaves the option out ofFT.CREATE, preserving the command generated for existing schemas. The sync and async index clients share this field builder.Recover HNSW settings from existing indexes
When
FT.INFOreportsRERANKorEF_RUNTIME, RedisVL carries those values into the recovered schema. Recreating the index from that schema preserves an explicitfalsesetting and a nondefault search depth.The README introduces Flex databases alongside the existing Redis deployment options. The schema reference includes a YAML example, and the field attributes guide documents
rerank.Release Notes
You can now set
rerankon HNSW vector fields when creating disk indexes in Flex databases. Loading an existing HNSW index preservesrerankandef_runtimewhen Redis reports them.AI Assistance
OpenAI Codex (GPT-6) assisted with implementation, tests, documentation, review and the PR description.
Reviewer Brief
Start here.
HNSWVectorField.as_redis_fieldinredisvl/schema/fields.pycontrols how both index clients emitRERANK. The unset case preserves the existing wire command. Explicit false must emitRERANK FALSE.Where it could fail. Check
convert_index_info_to_schemainredisvl/redis/connection.pyagainst the target server'sFT.INFOreply. RedisVL can recover the rerank setting and nondefault search depth only when the server reports them.Beyond CI. Five regression cases failed on upstream main at
4ce72d3a. Both affected test files passed on the final branch (82 tests). On ARM64 Redis Software 8.2.6-58.26 with Redis 8.8.0 and Search 8.8.8, both sync and async clients created indexes for each explicit boolean, recovered it with EF_RUNTIME 32, and returned the expected nearest neighbor over RESP2 and RESP3.Not verified. Cloud Flex, cluster mode, external-provider APIs, and load behavior. Full-tree codespell still flags the existing MessageHistory.acount API and its integration test; commit hooks passed for the review changes.
Reviewed f08959d, review refinements in 20b64b3 · Codex (GPT-6) · lenses: correctness, security, design, tests, docs, Redis behaviour
Since the review. Commit
3d4b5c75introduces Flex databases in the README and aligns terminology in the schema docs and field docstring. Commit hooks passed.