Repository navigation
fix: apply one rule to every distance threshold, and let 2 match everything - #838
Open
vishal-bala wants to merge 19 commits into
Open
vishal-bala wants to merge 19 commits into
vishal-bala wants to merge 19 commits into
Conversation
check() and acheck() searched with a VectorRangeQuery, so Redis discarded every candidate beyond distance_threshold and a miss carried no information about how close it came. They now run a KNN VectorQuery for num_results candidates and keep those at or within the threshold in Python, inclusive as the range radius was. On the FLAT index the cache always uses, both queries are exhaustive, so hit and miss decisions match. Measured on Redis 8.4.7 they agreed on 3,000 of 3,000 samples per configuration and on 200 of 200 queries within 2e-6 of the threshold. The one theoretical difference is that the client compares the distance as Redis returns it, rounded to 12 significant digits, so a distance within about 1e-12 of the threshold can now fall on the other side of it. Unfiltered lookups take 2-3% longer. With a tag filter matching 10% of entries they take 13% less time at 1,000 entries and 36% less at 10,000, because KNN compares vectors only within the filtered subset where the range query scanned the whole index. The nearest candidate's distance is now in hand on every lookup, which the planned OpenTelemetry instrumentation reports. A per-call distance_threshold is still validated as the range query validated it: TypeError for a non-number, ValueError for a negative value. An id-only row (the Redis 8.8+ expiry race) still reaches _process_cache_results and is skipped there rather than raising. Query construction and the threshold filter are shared by the sync and async paths.
The cache-wide threshold (__init__, set_threshold) accepted 0 to 2, while a per-call override (check, acheck) accepted any non-negative number, including values above 2, booleans and NaN, which turned every lookup into a silent miss. Both paths now share _validate_distance_threshold: a real number from 0 to 2 inclusive, refusing bool (TypeError), strings (TypeError) and NaN (ValueError), with set_threshold's existing ValueError message.
Cover np.float32 and the 0/2 boundaries through check and acheck as well as set_threshold, and list the new TypeError on __init__.
…olds Apply SemanticCache's distance-threshold rule to SemanticMessageHistory: the constructor, set_distance_threshold and the get_relevant override now accept a real number from 0 to 2 inclusive and refuse strings, bools, NaN and values outside the COSINE range. Move the rule into redisvl.utils.utils.validate_cosine_distance_threshold so that both classes share it.
Refuse a bad distance threshold before the constructor loads the default vectorizer, and before get_relevant returns early for top_k=0, so that every path refuses the same values.
Route.distance_threshold was checked only at construction, so update_route_thresholds and direct assignment accepted NaN, bools, strings and out-of-range numbers, which then reached the aggregation FILTER. Route now validates on assignment, and update_route_thresholds checks every value before applying any.
…iform-distance-threshold # Conflicts: # redisvl/extensions/cache/llm/semantic.py
…rapper reads main's check and acheck now run inside _telemetry.operation, which reads cache.name, and resolve refresh_ttl_on_hit.
vishal-bala
marked this pull request as ready for review
November 19, 2026 15:11
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…iform-distance-threshold
Drop the 0 and 2 accept cases, which the integration tests already run end to end, and the False case, which reaches the same bool branch as True. Reword the validator docstring's NaN sentence.
…ic-cache-uniform-distance-threshold # Conflicts: # redisvl/extensions/cache/llm/semantic.py
…tic-cache-uniform-distance-threshold
Apply the cosine threshold rule to LangCacheSemanticCache overrides (0-2 on the redis scale, 0-1 normalized), refuse bool and NaN in VectorRangeQuery.set_distance_threshold, and refuse bool and strings in Vector.max_distance before pydantic coerces them. SemanticCache now clamps a reported distance to 2 before comparing it, as norm_cosine_distance does, because float32 rounding can put an exactly opposite vector just above 2 and past the largest allowed threshold. Trim router and history tests that pass without the fix.
SemanticCache now refuses a bad threshold before it builds the default vectorizer, as SemanticMessageHistory does. Vector validates max_distance on assignment as well as construction, since it is interpolated into the query. Name the real-number rule in VectorRangeQuery's TypeError and list the new errors in LangCache's check and acheck. Test the shared rule once in test_utils.py and keep one wiring check per entry point; move the router cases into test_route_schema.py.
Redis can report an exactly opposite vector a little above 2 (measured on 8.10.3: 2.00000024 for FLOAT32). cosine_distance_bound widens a threshold of 2 past that, and SemanticCache's client-side compare, the message history's range radius, and the router's range radius and min/avg FILTER all use it. A summed router distance keeps its threshold unchanged.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
A distance threshold tells a semantic lookup how close a stored vector must be to count as a match. RedisVL takes one in six places:
SemanticCache,SemanticMessageHistoryandLangCacheSemanticCacheeach have a default for every lookup and an override for a single call, routes inSemanticRoutereach have their own, andVectorRangeQueryandMultiVectorQuery'sVectortake one directly.Each place validated its threshold its own way, and most let through values that break lookups without an error. A bool passes as a number, because Python's
Trueis the integer 1. A string from a config file passes in some places and fails in others. NaN is the worst case: every comparison with NaN is false, so a NaN threshold makes every lookup match nothing. Before this PR,SemanticCache.check(distance_threshold=float("nan")),SemanticMessageHistory(distance_threshold="0.3")androuter.update_route_thresholds({"greeting": 3})were all accepted. This PR gives every site the same rule.Changes
One rule for COSINE thresholds
A threshold must now be a real number from 0 to 2 inclusive, the range of the COSINE distance these indexes use. An
int, afloator a numpy scalar qualifies. Abool, a string, aDecimalor NaN does not.validate_cosine_distance_thresholdinredisvl/utils/utils.pyapplies the rule and returns the value as a plainfloat.SemanticCache,SemanticMessageHistoryandLangCacheSemanticCacheall call it, for the default and for an override.LangCacheSemanticCachechecks against 0 to 2 on its"redis"scale and 0 to 1 on its"normalized"scale. The constructors check before they load the default HuggingFace vectorizer, so a bad threshold fails without a model download.SemanticMessageHistory.get_relevantchecks the threshold before itstop_k == 0early return.The rule raises
TypeError: distance_threshold must be a real number, got <type>for a value of the wrong type. It raisesValueError: Distance must be between 0 and 2, got <value>for a value out of range or NaN, which is the messageSemanticCache.set_thresholdalready used.The same gaps closed at the other sites
The other three sites keep their own range and error type, because their callers already rely on them. They now refuse the same bad values.
SemanticRouterroutes keep their (0, 2] range and pydanticValidationError.Routenow validates on assignment as well as construction, andupdate_route_thresholdschecks every value before it applies any, so one bad value leaves every threshold unchanged.VectorRangeQuery.set_distance_thresholdnow refuses a bool and NaN. It has no upper bound, because it serves L2 and IP indexes as well as COSINE.Vector.max_distancenow refuses a bool and a string before pydantic coerces them to floats, and validates on assignment. The value is written into the query string.A threshold of 2 matches everything
A COSINE distance of 2 is the distance between two exactly opposite vectors, so a threshold of 2 should match every vector. Float rounding breaks that at the very edge. Measured on Redis 8.10.3, a FLOAT32 vector with 1536 dimensions reports a distance of 2.00000023842 from its own negation, and FLOAT16 reports up to 2.0006. A radius of 2, or
FILTER @distance <= 2.0, drops that vector, and a radius of 3 keeps it. Before this PR a caller could pass 3 to the cache or history override to match everything. With 2 now the maximum, 2 has to cover that case.cosine_distance_boundreturns the threshold unchanged below 2, and 3 at 2. Three comparisons use it:SemanticCache's client-side check of each candidate.SemanticMessageHistory's range radius.SemanticRouter's range radius, and itsFILTERfor theminandavgaggregation methods.A summed router distance keeps its threshold unchanged, because a sum of several reference distances can genuinely exceed 2.
Tests
tests/unit/test_utils.pytests the rule once. It accepts 0, 2,np.float32(0.5)and 1 against a maximum of 1, each returned as afloat. It refuses 2 + 1e-9, -1e-9, NaN, 1 + 1e-9 against a maximum of 1,Trueand"0.5". It also testscosine_distance_bound.tests/unit/test_semantic_cache_threshold.pyandtest_semantic_history_threshold.pycheck that each entry point applies the rule. They cover the constructor (before the vectorizer loads), the setter and each override, plus the widened bound at 2.test_route_schema.py,test_semantic_router_queries.py,test_langcache_semantic_cache.py,test_query_types.pyandtest_aggregation_types.pygained a case or two each.Notes
Numeric strings and
Decimalare refused rather than converted. The parameters are typedfloat, and converting strings would also let"nan"and"inf"throughfloat(). Most sites already refused strings for an override. A caller that passes a threshold straight from an environment variable or a config file now has to convert it withfloat().The router's rule still differs from the others in three ways, all from before this PR. It excludes 0, it accepts
Decimal(pydantic's strict float allows it), and it raisesValidationErrorrather thanTypeError. Changing any of them would break existing router configurations, so this PR only closes the paths that skipped validation.This PR should merge before the change that deprecates the implicit default vectorizer (branch
feat/deprecate-implicit-vectorizer), because both editSemanticCache.__init__.Release Notes
Distance thresholds on
SemanticCache,SemanticMessageHistory,LangCacheSemanticCache,SemanticRouterroutes,VectorRangeQueryandMultiVectorQueryvectors now raise an error for a boolean, a string or NaN (and, outside routes, aDecimal), and the cache, history and LangCache thresholds also for anything outside 0 to 2 (0 to 1 on LangCache's"normalized"scale), so convert a threshold read from config withfloat(). A COSINE threshold of 2 now matches every vector, including an exactly opposite one that rounding reports just above 2.AI Assistance
Claude Code (claude-opus-5-5): implementation, tests, pre-PR review and this description.
Reviewer Brief
Start here:
redisvl/utils/utils.py,validate_cosine_distance_thresholdandcosine_distance_bound. The cache, history and LangCache call the first for every threshold, and the cache, history and router call the second to decide what a threshold of 2 means.Where it could fail
redisvl/extensions/router/semantic.py,_distance_threshold_filter: widening the bound is only correct for an aggregation whose result can't exceed one reference's distance. Check thatminandavgget it andsumdoesn't.redisvl/extensions/router/schema.py:Routenow validates every field on assignment, not only the threshold. Code outside the library that setsroute.references = []or an emptynamenow raises. The library's own delete path edits the list in place for that reason.SemanticMessageHistory.get_relevantnow raises for a bad threshold even withtop_k=0, which used to return[].Not verified: the rounding overshoot was measured on FLAT indexes over HASH storage with RESP2. HNSW, JSON storage, RESP3 and BFLOAT16 weren't measured; the bound of 3 leaves a margin of about 1 over the largest overshoot seen. The live LangCache API (mocked here), and cluster mode, which this change doesn't touch.
Beyond CI: all 12 new or changed tests outside
test_utils.pyfail againstmain's library code. The validator's tests intest_utils.py, which can't be collected againstmain, each catch a mutation of the inclusive bound, the bool check or the float conversion. On Redis 8.10.3,VECTOR_RANGE 2andFILTER @distance <= 2.0drop a vector reported at 2.00000023842, and 3.0 keeps it.Reviewed
e434aff4, fixes ina80b6b4fand7f6d9854· Claude Code (claude-opus-5-5) · lenses: correctness, security, design, tests, docs, Redis behaviourNote
Medium Risk
Changes input validation and lookup filtering across cache, history, and router paths; callers passing booleans, strings, or overrides above 2 will see new errors, but behavior for valid thresholds is tightened rather than broadened except at threshold 2.
Overview
Introduces shared cosine distance threshold validation (
validate_cosine_distance_threshold) and rounding-aware bounds (cosine_distance_bound) inredisvl/utils/utils.py, then wires them through semantic cache, LangCache, message history, router, and query types so defaults and per-call overrides follow the same rules.Validation: Thresholds must be real numbers in [0, 2] for Redis COSINE (or [0, 1] where normalized).
bool, NaN, out-of-range values, and (for several paths) numeric strings now raiseTypeErrororValueErrorbefore Redis or vectorizers run.SemanticCachedrops its private_validate_distance_thresholdin favor of the shared helper; LangCache validates before converting to similarity;SemanticMessageHistoryandSemanticCacheconstructor/setters/check paths align; routerRoutethresholds getvalidate_assignment, andupdate_route_thresholdsvalidates all values before applying any.Threshold = 2: When the max threshold is 2 (“match everything”), comparisons and
VECTOR_RANGEradii usecosine_distance_boundso Redis-reported distances slightly above 2 (float rounding) are still included; the router widens aggregateFILTERbounds for min/avg but not for sum.Query layer:
Vector.max_distanceandset_distance_thresholdreject booleans and non-real numbers; assignment-time validation is enabled where values are interpolated into queries.Tests: New/extended unit tests cover invalid thresholds, bound widening at 2, router filters, and assignment validation.
Reviewed by Cursor Bugbot for commit 7f6d985. Bugbot is set up for automated code reviews on this repo. Configure here.