Single-leg retrieval #2602

What to do

• Planner. $vectorize, $vector and $lexical sorts become a plan with exactly one leg. Its read limit is rerankCandidateLimit, from the option or the config default.
• Sort check at resolve time. Build and check the sort with CollectionSortClauseBuilder. Add one rule that find doesn't have: $lexical must not be blank.
• Collection checks per leg. Today these live in checkSortSupported ([FindAndRerankOperationBuilder.java#L172-L202](

  data-api/src/main/java/io/stargate/sgv2/jsonapi/service/resolver/FindAndRerankOperationBuilder.java

  Lines 172 to 202 in 02d0423

  	private void checkSortSupported() {
  	if (isVectorSort() || isVectorizeSort()) {
  	if (!commandContext.schemaObject().vectorConfig().vectorEnabled()) {
  	throw SortException.Code.UNSUPPORTED_VECTOR_SORT_FOR_COLLECTION.get(
  	errVars(commandContext.schemaObject()));
  	}
  	}
  	if (isVectorizeSort()) {
  	// the service definition is on the $vectorize field, not $vector
  	var vectorizeEnabled =
  	commandContext
  	.schemaObject()
  	.vectorConfig()
  	.getColumnDefinition(VECTOR_EMBEDDING_TEXT_FIELD)
  	.map(VectorColumnDefinition::vectorizeDefinition)
  	.isPresent();
  	if (!vectorizeEnabled) {
  	throw SortException.Code.UNSUPPORTED_VECTORIZE_SORT_FOR_COLLECTION.get(
  	errVars(commandContext.schemaObject()));
  	}
  	}
  	// The index is only required if the user explicitly asked for lexical,
  	// they could also have used $hybrid and it expanded into the lexical sort.
  	if (isLexicalSort() && isExplicitLexicalSort()) {
  	throwIfNoLexicalIndex();
  	}
  	}

  )):
  • vector legs need vector enabled;
  • $vectorize also needs a vectorize service;
  • $lexical needs a lexical index. A single $lexical leg without an index returns an error, unlike $hybrid: "text", which skips lexical silently.
• Rerank query and passage. Today these are decided without knowing the mode ([RerankingQuery.java#L48-L65](

  data-api/src/main/java/io/stargate/sgv2/jsonapi/service/operation/reranking/RerankingQuery.java

  Lines 48 to 65 in 02d0423

  	public static RerankingQuery create(FindAndRerankCommand command) {
  	Objects.requireNonNull(command, "command must not be null");
  	var rerankQuery =
  	getOrDefault(command.options(), FindAndRerankCommand.Options::rerankQuery, "");
  	if (rerankQuery != null && !rerankQuery.isBlank()) {
  	return new RerankingQuery(rerankQuery, Source.OPTIONS);
  	}
  	var vectorizeQuery = command.sortClause().vectorizeSort();
  	// will never be blank, but double-checking for safety
  	if (vectorizeQuery != null && !vectorizeQuery.isBlank()) {
  	return new RerankingQuery(vectorizeQuery, Source.VECTORIZE);
  	}
  	throw RequestException.Code.MISSING_RERANK_QUERY_TEXT.get();
  	}

  ), [FindAndRerankOperationBuilder.java#L453-L471](

  data-api/src/main/java/io/stargate/sgv2/jsonapi/service/resolver/FindAndRerankOperationBuilder.java

  Lines 453 to 471 in 02d0423

  	private PathMatchLocator passageLocator() {
  	var rerankOn = getOrDefault(command.options(), FindAndRerankCommand.Options::rerankOn, null);
  	var isRerankOn = rerankOn != null && !rerankOn.isBlank();
  	String finalRerankField;
  	if (isVectorizeSort()) {
  	// use the vectorize field, unless the user has overridden
  	finalRerankField = isRerankOn ? rerankOn : VECTOR_EMBEDDING_TEXT_FIELD;
  	} else if (isRerankOn) {
  	// user has to provide a field to rerank on
  	finalRerankField = rerankOn;
  	} else {
  	throw RequestException.Code.MISSING_RERANK_ON.get();
  	}
  	return PathMatchLocator.forPath(finalRerankField);
  	}

  )). New rules:
  • $vectorize: default to the vectorize text and the $vectorize field;
  • $vector: rerankQuery and rerankOn are required;
  • $lexical: default rerankQuery to the lexical search text and rerankOn to the $lexical field, following Aaron’s design;
  • user-supplied rerankQuery and rerankOn always win.
• Keep the check order so that {"findAndRerank": {}} still returns MISSING_RERANK_QUERY_TEXT. The IT for this only runs on HCD, so pin it with a unit test as well.
• ITs:
  • $vector, both as an array and as $binary;
  • $vectorize, checking the defaults;
  • $lexical, on HCD only;
  • the read count follows rerankCandidateLimit (seed more than 50 documents);
  • error cases.