[Hazel-Datastax]

What this PR does:

This PR adds the single leg sorts to the findAndRerank sort contract: $vectorize, $vector, $lexical and field sorts such as {"age": -1}. The command now reads them and checks their syntax.

Out of scope:

• Planning and running single leg sorts (Single-leg retrieval #2602, Filter retrieval #2603). Until then, they are rejected with UNSUPPORTED_FIND_AND_RERANK_SORT.
• The rerankCandidateLimit option.

Problems I ran into when adding single leg sorts:

1. The old sort class cannot hold single leg sorts.
On main, FindAndRerankSort is one class with a leg mode and a nullable field for each value. #2647 added the single leg values to this class, and it showed three problems:

• {"$lexical": null} and {} are stored the same way. So a wrong $lexical would run as a filter leg with no sort, and the user gets no error.
• A field sort such as {"age": -1} has no field to go in.
• Each new form adds more nullable fields. Every user of the class must check the fields in the right order, and this is easy to get wrong.

So the sort is now a sum type: a sealed interface with one record for each valid form.

2. The sort was parsed in a Jackson deserializer.

• Errors are thrown while Jackson reads the request, before the command runs. On HTTP the user gets the general REQUEST_STRUCTURE_MISMATCH. On MCP the user gets "unable to convert value" or "Internal error", with no error code and no reason.
• Single leg sorts use the find sort syntax, but the find rules cannot be reused. In find, the syntax checks are not separate code. CollectionSortClauseBuilder checks the syntax, checks the lexical index of the collection and builds the find sort model, all in one method. So it needs the collection, and the deserializer does not have it.
  • Calling it from the planner means the planner checks syntax. That is not the planner's job. It also returns the find sort model, which has the same problem as our old class: nullable fields that must be checked in the right order. Its $vectorize sort can also be changed in place.
  • Copying the rules into the deserializer gives two copies that can drift apart.
  • Sharing the rules as helpers still runs them in the deserializer, so MCP still loses the errors.
• The checks for one $hybrid sort were in two classes, so you had to read both to know which sorts are valid. For example, both of these sorts have a bad $vectorize value:
  • {"$hybrid": {"$vectorize": 1}}: the deserializer rejected it with REQUEST_STRUCTURE_MISMATCH, because the value is not text.
  • {"$hybrid": {"$vectorize": ""}}: the deserializer accepted it. Later, after the option checks, the planner rejected it with MISSING_HYBRID_SORT_VALUE.
• Parsing in the deserializer loses the original input. For example, "$vectorize": null and a missing $vectorize give the same result. So the metrics features had to be recorded inside the deserializer.

Proposed design

If we treat the sort as a small query language, we can reuse a design from databases. A database handles a query in separate steps. I propose the same steps for the findAndRerank sort.

NOTE: This PR uses these steps only for the sort. But we can apply the same idea to the whole findAndRerank command (e.g. hybridLimits)

flowchart LR
  J["Jackson<br/>(HTTP and MCP)<br/>keeps the sort as sent"] --> P
  subgraph R["FindAndRerankCommandResolver: calls each step in order"]
    P["Parser (this PR)<br/>is the syntax valid?"] --> B["Binder (next PR)<br/>can it run with this<br/>collection and config?"]
    B --> PL["Planner<br/>how to run it?"]
    PL --> BU["Builder<br/>makes the operation"]
  end
  BU --> E["Execution<br/>reads, then reranks"]
  classDef next stroke-dasharray: 5 5
  class B next

Loading

How this fixes the problems above:
-FindAndRerankSortDefinition uses @JsonCreator(mode = DELEGATING), the same as SortDefinition in find. Jackson only wraps the JSON, so there is no custom deserializer.

• FindAndRerankSort is a sealed interface with one record for each valid form. A wrong $lexical is a syntax error, not an empty sort.
• The parser runs after the request is read, while the command runs. So HTTP and MCP get the same error code and message.
• The parser checks only the syntax, so it needs no collection. All the syntax checks for one sort are in one place, and find can reuse them later.
• The command keeps the sort as the user sent it, so the metrics features come from the field names.

Next PRs

If we agree on the design, I plan these PRs:

1. Add the binder. Move the collection and config checks from the planner into it. This does not change behaviour.
2. Plan single leg sorts (Single-leg retrieval #2602, Filter retrieval #2603) and add them to the schema (Update HTTP/MCP descriptions and examples. #2604). Their collection checks go into the binder. Add the rerankCandidateLimit option, which only single leg sorts use.
3. Use the same steps for hybridLimits. The parser checks the shape, for example that the object has both $vector and $lexical. The binder checks the value against the range in the config, by default 1 to 100. Its error codes change, and MCP gets them too.

Which issue(s) this PR fixes:
Fixes #

Checklist

• Changes manually tested
• Automated Tests added/updated
• Documentation added/updated
• CLA Signed: DataStax CLA

[github-actions]

📈 Unit Test Coverage Delta vs Main Branch

Metric	Value
Main Branch	56.74%
This PR	56.83%
Delta	🟢 +0.09%
✅ Coverage improved!

[github-actions]

Unit Test Coverage Report

Overall Project	56.83% -0.12%	🍏
Files changed	86.36%	🍏

File	Coverage
FindAndRerankSortParser.java	100%	🍏
FindAndRerankPlanner.java	98.8% -0.8%	🍏
FindAndRerankCommand.java	86.36%	🍏
SortClauseBuilder.java	83.97%	🍏
CollectionSortClauseBuilder.java	83.43%	🍏
FindAndRerankSortDefinition.java	80% -20%	🍏
SortExpression2.java	71.96% -28.04%	🍏
FindAndRerankSort.java	55% -45%	❌
FindAndRerankCommandResolver.java	54.72% -16.98%	❌
CollectionCommandTools.java	0%	🍏

[github-actions]

📉 Integration Test Coverage Delta vs Main Branch (dse69-it)

Metric	Value
Main Branch	72.98%
This PR	72.82%
Delta	🔴 -0.16%
⚠️ Coverage decreased

[github-actions]

Integration Test Coverage Report (dse69-it)

Overall Project	72.82% -0.5%	🍏
Files changed	42.5%	❌

File	Coverage
CollectionCommandTools.java	100%	🍏
FindAndRerankCommandResolver.java	92.86%	🍏
FindAndRerankPlanner.java	87.2% -3.8%	🍏
FindAndRerankCommand.java	81.82% -0.91%	🍏
FindAndRerankSortDefinition.java	73.33% -26.67%	🍏
SortClauseBuilder.java	55.73%	🍏
CollectionSortClauseBuilder.java	55.25%	🍏
FindAndRerankSortParser.java	38.58% -61.42%	❌
FindAndRerankSort.java	18.13% -81.88%	❌
SortExpression2.java	0%	❌

[github-actions]

📉 Integration Test Coverage Delta vs Main Branch (hcd-it)

Metric	Value
Main Branch	74.33%
This PR	74.19%
Delta	🔴 -0.13%
⚠️ Coverage decreased

[github-actions]

Integration Test Coverage Report (hcd-it)

Overall Project	74.19% -0.44%	🍏
Files changed	49.11%	❌

File	Coverage
CollectionCommandTools.java	100%	🍏
FindAndRerankPlanner.java	98% -0.8%	🍏
FindAndRerankCommandResolver.java	92.86%	🍏
FindAndRerankCommand.java	82.73%	🍏
FindAndRerankSortDefinition.java	80% -20%	🍏
CollectionSortClauseBuilder.java	71.27%	🍏
SortClauseBuilder.java	55.73%	🍏
FindAndRerankSortParser.java	45.21% -54.79%	❌
FindAndRerankSort.java	25.62% -74.38%	❌
SortExpression2.java	0%	❌