Skip to content

fix(coverage-evidence): materialize uv.lock-only repos so their offline coverage can run - #668

Open
seonghobae wants to merge 1 commit into
mainfrom
claude/cwlab-pr-audit-governance-1hdcp5
Open

fix(coverage-evidence): materialize uv.lock-only repos so their offline coverage can run#668
seonghobae wants to merge 1 commit into
mainfrom
claude/cwlab-pr-audit-governance-1hdcp5

Conversation

@seonghobae

@seonghobae seonghobae commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Rebased onto main at 9762ebd (post-#678/#674) and narrowed to a single component — the uv.lock materializer, which is the only part of the original PR not now superseded (see the disposition table on this PR; #678 landed the Zen reviewer that superseded the model-widening, and #674 superseded the fuzz-lock exclusion). This branch was force-pushed from 4 commits to 1.

Problem

The central coverage-evidence sandbox installs the hash-pinned Python locks the materializer extracts from the PR base commit, then runs each repo's pytest tests offline. Repositories managed entirely by uv expose only a uv.lock, so content-based discovery (from #657) found nothing — no dependencies install, the suite fails at import, coverage-evidence reports failure, and OpenCode posts REQUEST_CHANGES on every PR in that repo even when the repo's own CI is green.

Confirmed still-unfixed on current main: scripts/ci/materialize_base_python_requirements.py only discovers already-hash-pinned requirement files; it has no uv export path. Live impact: keyverse#32/#34/#37, newsdom-api#464, pg-llm-batch#17 are all blocked on this exact coverage-evidence: failure for their uv-managed services/account_unification / package trees.

Fix

When a base uv.lock is discovered, reconstruct it and its sibling pyproject.toml from the exact validated base SHA in an isolated temp dir and run uv export --frozen --no-emit-project --no-editable to produce a hash-pinned closure the #661 preflight installer consumes like any other lock. --frozen forbids lock mutation and network resolution; both inputs are read only from the base commit; it is best-effort (any failure skips the uv.lock exactly as before, so it cannot break an existing pipeline).

Verification (this exact head, in a clean venv with the CI lock)

  • coverage run -m pytest tests746 passed
  • coverage report (fail_under=100, source scripts/ci) → TOTAL 100%, exit 0
  • interrogate (fail-under=100, excl tests) → PASSED 100.0%, exit 0
  • Materializer touches only scripts/ci/materialize_base_python_requirements.py (+82/-5) and tests/test_materialize_base_python_requirements.py (+130); no contract-pinned workflow/prose files are changed, so there is zero drift against the fix(governance): finish sweep with Zen review fallback #678 candidate-string/contract updates now on main.

Scope / follow-up

Native-extension repos needing a maturin build of a compiled module (e.g. fast-mlsirm's _core) remain a separate, harder gap and are intentionally not addressed here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HdCssGnNMhKHNu3TXFstWH


Generated by Claude Code

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 16 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: bae5277a-1548-4b42-9a07-fdd583fa068d

📥 Commits

Reviewing files that changed from the base of the PR and between 9762ebd and eace3df.

📒 Files selected for processing (2)
  • scripts/ci/materialize_base_python_requirements.py
  • tests/test_materialize_base_python_requirements.py

Comment @coderabbitai help to get the list of available commands.

@seonghobae seonghobae changed the title fix(review): materialize uv.lock-only repos so their offline coverage can run fix(review): materialize uv.lock-only repos + widen free reviewer model pool Jul 30, 2026
@seonghobae seonghobae changed the title fix(review): materialize uv.lock-only repos + widen free reviewer model pool fix(review): uv.lock materialization + free-model pool widening + exclude fuzz-only locks from coverage-evidence Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Correction: drop the model-pool widening from this PR — it is counterproductive given #624

I filed the "widen the free reviewer model pool" part of this PR (append gpt-5-mini, o4-mini, mistral-ai/mistral-medium-2505, meta/llama-4-maverick-17b-128e-instruct-fp8, meta/llama-4-scout-17b-16e-instruct to OPENCODE_MODEL_CANDIDATES) before re-checking the source-of-truth issues. That was a mistake:

Recommendation: drop commit c547fbf (the model-pool widening + its test_opencode_agent_contract.py changes) from this branch and let #671 handle the provider migration. The other two components of this PR are unaffected and still valid on their own merits:

  1. uv.lock materializer — makes coverage-evidence run for uv-only repos (newsdom-api, pg-llm-batch).
  2. fuzz-only lock exclusion — stops the transient atheris wheel-availability failure from failing coverage-evidence on unrelated PRs.

I could not push the branch revision myself this session (the environment's Bash is classifier-blocked, so no git), hence flagging it here rather than force-pushing the drop. Whoever picks this up: rebase to keep only the materializer + fuzz-lock-exclusion commits, or split them into a fresh PR, and route the pool fix through #671.

Also worth a look while here: this PR's merge is refused with 405 "New changes require approval from someone other than the last pusher", which is the require_last_push_approval: true behavior that #454 (closed completed) set out to remove on ruleset 18156473 — that requirement appears to have regressed or was re-added.


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

Dedup update: two of this PR's three components are now superseded — reduce to the uv.lock materializer only

Re-checking the open PR set shows the provider-outage (#624) is being fixed correctly elsewhere, which supersedes most of this PR:

#668 component status superseded by
1. uv.lock materializer keep — unique nothing else materializes uv-only repos (uv.lock → hash-pinned closure); #674 does not cover this
2. model-pool widening (5 × github-models/*) drop — counterproductive (GitHub Models retired today per #624) #672 (NVIDIA NIM), #675 (keyless OmniRoute), #678 (OpenCode Zen opencode/gpt-5.6-terra) — the live providers
3. fuzz-lock exclusion at materialization drop — superseded #674 defers yanked/interpreter-incompatible locks at the installer (DEFERABLE_PREFLIGHT_FAILURES), which is more general and correctly handles that the sandbox reads the lock from the base SHA — a limitation my materialization-scoping shares

Recommendation: rebase this branch down to component 1 (uv.lock materializer) only, or split it into a fresh uv.lock-only PR, and let #672/#674/#675/#678 own the provider + preflight-resilience fixes. I can't do the rebase from this session (environment Bash is classifier-blocked), hence flagging for triage.

Note for the pool fix: #678's approach (bot-authored PR so a non-pusher OpenCode App review can satisfy ruleset 18156473's non-pusher-approval rule) is the correct mechanism to land a pipeline-fix past the very deadlock this PR hit with 405 — that path is not available to a human pusher, so #678 (not #668) is the one to drive to merge.


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

Status update — the pipeline deadlock this PR was fighting is now resolved upstream.

#678 merged to main at 9762ebd (2026-07-30T21:27Z). It adds the paid OpenCode Zen gpt-5.6-terra reviewer as the lead pool model (keyed via OPENCODE_ZEN_API_KEY) and activates the */15 org sweep's auto-review + auto-merge. That restores opencode-review approval evidence org-wide after the GitHub Models retirement (#624), which is the root cause this PR's component 2 (model-widening) was trying to work around.

Disposition of this PR's three components against current main:

Component Status now
1. uv.lock materializer (uv-only repos' coverage-evidence) Unique / still needed — not addressed by #678 or #674; keeps newsdom-api#464, pg-llm-batch#17 unblockable.
2. Free-model pool widening (5 github-models/*) Superseded by #678. Those github-models candidates are sunset (#624); #678 leads with Zen instead.
3. Fuzz-only lock exclusion from coverage-evidence Superseded by #674 (defers yanked/interpreter-incompatible base locks at the installer, and correctly handles the base-SHA read that a repo-side pin bump can't self-heal).

Because #678 rewrote the exact OPENCODE_MODEL_CANDIDATES string and the test_opencode_agent_contract.py candidate list this PR also pins, this PR's contract test will now fail against main once it is updated onto the new base. Recommended action: rebase down to component 1 only (the uv.lock materializer) and drop components 2 and 3. That rework needs a local coverage run -m pytest tests + interrogate pass to re-verify the 100% gates, which I cannot execute in the current environment — so I am not pushing an unverified rebase. Flagging clearly rather than leaving a silently-broken PR for the sweep to merge.


Generated by Claude Code

… can run

The central coverage-evidence sandbox installs the hash-pinned Python locks the
materializer extracts from the PR base commit, then runs each repo's
`pytest tests` offline. Repositories managed entirely by uv expose only a
`uv.lock` (no `requirements*.txt` / `requirements.lock`), so the content-based
discovery from #657 finds nothing, no dependencies install, and the suite fails
at import (`ModuleNotFoundError`). coverage-evidence then reports `failure` and
OpenCode posts a deterministic REQUEST_CHANGES, permanently blocking every
uv-managed repo's PR queue even when the repo's own CI is green. Observed on
newsdom-api#464 and pg-llm-batch#17.

Fix: when a base `uv.lock` is discovered, reconstruct it and its sibling
`pyproject.toml` from the exact validated base SHA in an isolated temp dir and
run `uv export --frozen --no-emit-project` to produce a fully hash-pinned
dependency closure the existing #661 installer consumes like any other lock.
`--frozen` forbids lock mutation and network resolution, and both inputs are
read only from the base commit, so no PR-mutable content reaches uv and the
trust boundary is unchanged. It is best-effort: if `uv` is absent, the base
`pyproject.toml` is missing, the export fails, or its output is not fully
hash-pinned, the lock is skipped exactly as before, so this can never break an
otherwise-working build. Flat-layout and requirements-based repos are
unaffected.

Verification (against the real newsdom-api base tree): `uv.lock` ->
`requirements-000.txt` with 225 `--hash=` pins. Full suite: 710 passed;
scripts/ci line coverage 100% (materializer 114/114); interrogate 100%.

Scope: this covers uv.lock-only repos on runners where `uv` is available
(GitHub's ubuntu-latest ships it). Native-extension repos that need a maturin
build of a compiled module (e.g. fast-mlsirm's `_core`) remain a separate,
harder gap and are not addressed here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HdCssGnNMhKHNu3TXFstWH
@seonghobae
seonghobae force-pushed the claude/cwlab-pr-audit-governance-1hdcp5 branch from e8649ad to eace3df Compare July 30, 2026 21:40
@seonghobae seonghobae changed the title fix(review): uv.lock materialization + free-model pool widening + exclude fuzz-only locks from coverage-evidence fix(coverage-evidence): materialize uv.lock-only repos so their offline coverage can run Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

opencode-review failed at head eace3df, but the failure is not in this PR's diff — it is the org-wide model-pool outage, still unresolved.

From the failing run (30584655800) status step env:

OPENCODE_MODEL_POOL_OUTCOME: exhausted
COVERAGE_EVIDENCE_RESULT:    success

This is not fixable from this PR. The remaining blocker is org-side provisioning of a working reviewer provider — most likely the OPENCODE_ZEN_API_KEY org secret that #678's provider block reads via OPENCODE_API_KEY, or an equivalent (NVIDIA_NIM_API_KEY for #672, OMNIROUTE_API_BASE_URL for #675). Once one authenticates, re-running opencode-review on this exact head should produce approval evidence and the sweep can merge it. Flagging with logs rather than pushing a no-op change to an already-green diff.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants