Filter retrieval #2603

What to do

• FilterLeg: an inner find with the user's filter, the optional field sort, and a read limit of rerankCandidateLimit.
  • It needs neither a vector nor a lexical index. It works on collections without vectors, as long as the collection has reranking enabled or the request has a rerank override.
  • rerankQuery and rerankOn are required.
  • Build and validate the field sort at resolve time with CollectionSortClauseBuilder and SortClause.validate. That way a bad path fails early instead of inside the read task.
  • Cost note: a field sort is an in-memory sort, the same as find. It reads up to maxDocumentSortCount matching rows (default 10,000, [OperationsConfig.java#L98-L99](

    data-api/src/main/java/io/stargate/sgv2/jsonapi/config/OperationsConfig.java

    Lines 98 to 99 in 02d0423

    	@WithDefault("10000")
    	int maxDocumentSortCount();

    )) and fails above that.
• Ranking and results. This part has no dependencies and can start any time.
  • Add RankSource.FILTER ([Rank.java#L23-L28](

    data-api/src/main/java/io/stargate/sgv2/jsonapi/service/operation/reranking/Rank.java

    Lines 23 to 28 in 02d0423

    	public enum RankSource {
    	/** Rank came from a vector similarity sort */
    	VECTOR,
    	/** Rank came from a BM25 sort */
    	BM25;
    	}

    )) so filter documents get an RRF score from their read order.
    • Today only VECTOR and BM25 documents get a rank ([ScoredDocument.java#L94-L124](

      data-api/src/main/java/io/stargate/sgv2/jsonapi/service/operation/reranking/ScoredDocument.java

      Lines 94 to 124 in 02d0423

      	if (rankSource == Rank.RankSource.VECTOR) {
      	if (similarityField == null) {
      	// this can happen when the scores are not going to be returned, OK we only need the rank
      	// for RRF
      	vectorScores = DocumentScores.fromVectorRead(rank);
      	} else if (similarityField instanceof NumericNode numberNode) {
      	vectorScores = DocumentScores.fromVectorRead(numberNode.floatValue(), rank);
      	} else {
      	throw new IllegalArgumentException(
      	"%s document field is not a number or is missing type=%s _id=%s"
      	.formatted(
      	VECTOR_FUNCTION_SIMILARITY_FIELD,
      	nodeTypeAsString(similarityField),
      	documentId.asText()));
      	}
      	} else {
      	if (similarityField != null) {
      	throw new IllegalArgumentException(
      	"rankSource is %s but the document has similarity score id=%s has %s=%s"
      	.formatted(
      	rankSource, documentId, VECTOR_FUNCTION_SIMILARITY_FIELD, similarityField));
      	}
      	vectorScores = DocumentScores.EMPTY;
      	}
      	// if we dont have a vector score, then use the rank as the bm25 rank (it is one or the other)
      	// if we have a vector score, then the BM25 is Empty
      	var bm25Scores =
      	rankSource == Rank.RankSource.BM25
      	? DocumentScores.fromBm25Read(rank)
      	: DocumentScores.EMPTY;

      )).
    • When rerank scores tie, the order falls back to _id ([DocumentScores.java#L122-L131](

      data-api/src/main/java/io/stargate/sgv2/jsonapi/service/operation/reranking/DocumentScores.java

      Lines 122 to 131 in 02d0423

      	public int compareTo(DocumentScores other) {
      	Objects.requireNonNull(other, "Other must not be null");
      	var rerankCompare = rerankScore.compareTo(other.rerankScore);
      	if (rerankCompare != 0) {
      	return rerankCompare;
      	}
      	return rrfScore.compareTo(other.rrfScore);
      	}

      ), [ScoredDocument.java#L197-L203](

      data-api/src/main/java/io/stargate/sgv2/jsonapi/service/operation/reranking/ScoredDocument.java

      Lines 197 to 203 in 02d0423

      	public int compareTo(ScoredDocument other) {
      	var scoresCompare = scores.compareTo(other.scores);
      	if (scoresCompare != 0) {
      	return scoresCompare;
      	}
      	return ID_NODE_COMPARATOR.compare(this.id, other.id);
      	}

      )).
    • Ties will be common, because documents with the same field value get the same rerank score.
  • Drop a document whose rerankOn value is an array or an object. Today that throws, which returns a 500 ([ScoredDocument.java#L141-L144](

    data-api/src/main/java/io/stargate/sgv2/jsonapi/service/operation/reranking/ScoredDocument.java

    Lines 141 to 144 in 02d0423

    	default ->
    	throw new IllegalArgumentException(
    	"Passage field %s is present but not null or a valueNode _id=%s , passageField=%s"
    	.formatted(passageLocator.path(), documentId, passageNode));

    )). Missing and null values are already dropped ([ScoredDocumentMerger.java#L47-L49](

    data-api/src/main/java/io/stargate/sgv2/jsonapi/service/operation/reranking/ScoredDocumentMerger.java

    Lines 47 to 49 in 02d0423

    	if (scoredDocument.passage().isEmpty()) {
    	droppedDocumentsCount++;
    	return;

    )).
  • With includeScores, return $vector: null instead of the -2.147E9 placeholder ([Score.java#L19-L20](

    data-api/src/main/java/io/stargate/sgv2/jsonapi/service/operation/reranking/Score.java

    Lines 19 to 20 in 02d0423

    	public static final Score EMPTY_VECTOR_SCORE =
    	new Score(ScoreSource.VECTOR, false, Integer.MIN_VALUE);

    ), [DocumentScoresDesc.java#L28](

    data-api/src/main/java/io/stargate/sgv2/jsonapi/service/operation/reranking/DocumentScoresDesc.java

    Line 28 in 02d0423

    @JsonProperty(DocumentConstants.Fields.VECTOR_EMBEDDING_FIELD) float vector,

    )).
  • Keep dropping documents without rerankOn after the read, as the design doc says ("same as current"). So results can be fewer than limit.
• ITs:
  • sorts {} and {"age": -1};
  • more than 50 candidates read without truncation, both sorted and unsorted;
  • documents dropped after the read;
  • limit larger than rerankCandidateLimit;
  • a projection that excludes rerankOn;
  • an unindexed sort path;
  • OVERLOADED_SORT_ROW_LIMIT. ITs set the max sort count to 100 ([DseTestResource.java#L102-L104](

    data-api/src/test/java/io/stargate/sgv2/jsonapi/testresource/DseTestResource.java

    Lines 102 to 104 in 02d0423

    	public Long getMaxDocumentSortCount() {
    	return 100L;
    	}

    )), so this case needs its own collection with more than 100 documents.