Skip to content

Add integration test runs a real rerank #2600

Closed

Description

@Hazel-Datastax

Current behavior

Why fix it

  • We can't test the core of the feature: result order, how many candidates each leg reads, and the default rerank query and passage.

Implementation and acceptance criteria

Add a separate successful-reranking test PR before the two #2599 implementation PRs. Use class-scoped, dynamic-port WireMock URLs with the production Nvidia HTTP client. Scores must depend on passage content rather than concurrent batch arrival, and provider responses must include rankings[].logit.

Exercise existing $hybrid requests and the local embedding test provider. Cover default and explicit queries/passages, final ordering, projection, result limits, optional scores, hybrid deduplication, batching, and zero provider calls when no usable candidates remain. Extend DataApiResponseValidator.wasSuccessful() and replace the MCP missing-field workaround with successful reranking of existing documents.

Seed 60 unique passages, assert every passage reaches the HTTP reranker exactly once, and give a candidate beyond position 50 the highest score. Add database-free HTTP client coverage for serialization, headers, batch indices, response decoding, and usage. Packaged integration tests run in the existing DSE/HCD CI matrix, with lexical cases gated to HCD.

Public single-leg request syntax is outside this PR.

Draft PRs

PR #2605 adds the successful HTTP/MCP coverage. It is the first draft in the stack followed by #2606 and #2607.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions