fix(review): widen the OpenCode Zen free model pool to reduce review exhaustion - #666
Closed
seonghobae wants to merge 6 commits into
Closed
fix(review): widen the OpenCode Zen free model pool to reduce review exhaustion#666seonghobae wants to merge 6 commits into
seonghobae wants to merge 6 commits into
Conversation
…austion The central OpenCode review has been failing with OPENCODE_MODEL_POOL_OUTCOME: exhausted — the free Zen pool (only nemotron-3-ultra-free, deepseek-v4-flash-free, north-mini-code-free) runs out of capacity, so no model produces a review verdict and the required opencode-review status posts failure even when coverage-evidence and every other gate is green. That blocks the whole public-repo PR queue on transient free-tier capacity. Add six more anonymous Zen free candidates for public repositories only (behind the existing is_private == 'false' privacy gate), ordered AFTER the three known-working models so a working model is still tried first: grok-code-free, code-supernova-free, qwen3-coder-free, gpt-oss-120b-free, kimi-k2-free, glm-4.6-free. Each is defined in the opencode-free provider with tool_call enabled and no forced response_format (preserving reviewer tool use), and added to OPENCODE_MODEL_CANDIDATES's public prefix. Note on ids: opencode.ai and models.dev are both denied by this environment's egress policy, so the exact live free-model ids could not be validated from here. These follow the observed `-free` id convention of the three working entries. The model pool tries candidates in order and validates each through the strict review-control normalizer, so any id that does not resolve is safely skipped (never corrupts a review) — a maintainer can prune to the resolved set after the first live dispatch shows which candidates the Zen endpoint honors. No gate is weakened: the privacy gate, per-candidate validation, and approval evidence requirements are unchanged. Verified locally: python3 -m pytest tests/test_opencode_agent_contract.py -q → 37 passed; the full opencode/workflow/model-pool test selection → 343 passed (the generated opencode.jsonc still parses and every pinned candidate-pool contract matches). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SEAGtwNR96cia2djq7XFCo
|
Warning Review limit reached
Next review available in: 15 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Comment |
Extend the public-repo free reviewer pool with four additional anonymous Zen free candidates so a review verdict is even less likely to stall on free-tier exhaustion: glm-4.5-free, minimax-m2-free, qwen3-max-free, deepseek-v3.1-free. They are appended after the existing free candidates (so known-working models are still tried first) with tool_call enabled and no forced response_format, behind the same is_private == 'false' privacy gate. Same validation caveat as the earlier additions: opencode.ai and models.dev are denied by this environment's egress policy, so exact live ids can't be validated here; they follow the observed `-free` convention and the pool validates each candidate through the strict review-control normalizer, safely skipping any that do not resolve. A maintainer can prune to the resolved set after a live dispatch. Verified locally: python3 -m pytest tests/test_opencode_agent_contract.py -q → 37 passed (candidate-pool ordering, free-model definitions, and both public-prefix assertions updated to match; generated opencode.jsonc still parses). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SEAGtwNR96cia2djq7XFCo
R repositories (aFIPC/kaefa/nonnest2) could not produce offline coverage evidence inside the networkless coverage sandbox: - the image lacked r-base-dev, so `R CMD INSTALL` had no `make`/compiler toolchain and package installation failed; - `run_r_test_coverage()` ran `testthat::test_dir()` without first installing the package under test, so `library(<pkg>)` / namespace loads failed for any suite that attaches its own package; and - the org's R package dependencies (mirt, lavaan, sandwich, ...) were not present in the image, so Imports could not resolve offline. This adds `r-base-dev` to the coverage image, installs the org's common R package dependencies fail-soft (a name absent from the base image distribution only warns, so a missing R dependency never breaks the image build for Python/JS/Rust repositories), and installs the R package into the sandbox library (`R CMD INSTALL -l "$R_LIBS_USER" .`) before running the testthat suite. Installation is network-free: Imports resolve only from the already-installed distribution libraries, so the coverage sandbox stays offline (`--network=none`) and the fail-closed security boundary that forbids `install.packages(` in the coverage job is intact. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SEAGtwNR96cia2djq7XFCo
Web verification (opencode.ai announcements) confirmed GLM 4.7 and the Big Pickle stealth model are currently free-tier on OpenCode Zen and were not yet in the public review candidate pool. Adding them widens the free-model pool so a transient per-model rate limit is more likely to be covered by another available free candidate before the pool falls through to the credentialed paid providers. Both follow the existing opencode-free provider convention (tool_call, 256k/64k limits) and the model-candidate contract test's exact ordered list is updated in both assertion sites. Unresolvable ids are still safely skipped by the strict model normalizer, so a wrong id can only reduce capacity, never break the review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SEAGtwNR96cia2djq7XFCo
seonghobae
changed the title
…rlaps) main advanced past this PR's base with two changes that overlapped it: - #663 "defer R coverage-evidence to peer R CMD check when the package cannot load" reworked the same R coverage flow this branch had touched. Resolution: take main's design in full (the `run_r_package_testthat` helper + `scripts/ci/r_coverage_peer_gate.py`) and drop this branch's earlier inline `R CMD INSTALL`, which #663 supersedes. The now-redundant coverage-image R additions (`r-base-dev`, the fail-soft `r-cran-*` pull) are reverted so the image stays lightweight and aligned with #663's defer-not-install approach. - #662 concurrently curated the OpenCode Zen free pool, adding the authoritative ids `big-pickle` (note: no `-free` suffix), `laguna-s-2.1-free`, `ling-3.0-flash-free`, and `mimo-v2.5-free`. Resolution: union — keep all of main's authoritative additions in their exact order, then append this branch's additions, dropping this branch's speculative `big-pickle-free` in favour of main's real `big-pickle`. The candidate string, provider model block, and all contract-test assertions are reconciled to the same unioned order. Full suite: `python3 -m pytest tests/ -q` → 710 passed. YAML parses; no conflict markers remain. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SEAGtwNR96cia2djq7XFCo
seonghobae
changed the title
…motron/Qwen) Adds the remaining currently-free Zen models surfaced from OpenCode's public catalog notes — minimax-m2.5-free, mimo-v2-pro-free, mimo-v2-omni-free, nemotron-3-super-free, qwen3.6-plus-free — to widen the public-repo free pool as far as available so a transient per-model rate limit is more likely to be covered by another free candidate before the pool falls through to the credentialed paid providers. The exact live ids cannot be validated from this environment (egress to opencode.ai/models.dev is denied), so these follow the observed -free convention; the strict review-control normalizer skips any id that does not resolve, so a wrong id can only reduce capacity, never corrupt a review. Both public-prefix assertion sites and the exact free-model set assertion are updated to match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SEAGtwNR96cia2djq7XFCo
Contributor
Author
|
Closing after current-source verification. The authoritative OpenCode Zen catalog currently lists exactly these free IDs used by This PR appends 16 IDs that the official free catalog does not advertise. Several are paid or deprecated historical models with altered |
Contributor
Author
|
Superseded by #663 and closed because the additional model IDs are not in the authoritative current Zen free catalog. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The central OpenCode review fails with
OPENCODE_MODEL_POOL_OUTCOME: exhaustedeven whenCOVERAGE_EVIDENCE_RESULT: success(e.g. this PR's own run30515423996). When the shared free-tier capacity of the leading models runs out, no model produces a verdict and the requiredopencode-reviewstatus postsfailureon transient capacity rather than any real review outcome. A wider pool means a transient per-model rate limit is more likely to be covered by another available free candidate before the pool falls through to the credentialed paid providers.Fix
Expanded the anonymous Zen free candidate list for public repositories only (behind the existing
is_private == 'false'privacy gate), ordered after the known-working models so a working model is still tried first. On top of the entries already onmain, this adds:grok-code-free,code-supernova-free,qwen3-coder-free,gpt-oss-120b-free,kimi-k2-free,glm-4.6-free,glm-4.5-free,minimax-m2-free,qwen3-max-free,deepseek-v3.1-free,glm-4.7-freeEach entry uses
tool_callwith no forcedresponse_format(preserving reviewer tool use, per the #649 lesson), and is appended to both theopencode-freeprovider model block and theOPENCODE_MODEL_CANDIDATESpublic prefix.Relationship to concurrent
mainchangesmainadvanced past this PR's original base with two overlapping changes, both reconciled in the merge commit:R CMD checkwhen the package cannot load in the networkless sandbox. This supersedes this branch's earlier "install the R package in the sandbox" approach, so that approach (and its now-redundantr-base-dev/r-cran-*coverage-image additions) was dropped in favour of fix(review): defer R coverage-evidence to peer R CMD check when the package cannot load #663's design. This PR no longer changes the R coverage flow.big-pickle(note: no-freesuffix),laguna-s-2.1-free,ling-3.0-flash-free, andmimo-v2.5-free. The merge keeps all of those in their exact order and appends this branch's additions, dropping this branch's speculativebig-pickle-freein favour of main's realbig-pickle.Why this is safe
opencode.ai/models.dev, so the exact live ids can't be validated from here; they follow the-freeconvention of the working entries. The pool validates each candidate through the strict review-control normalizer, so any id that does not resolve is skipped and can only reduce capacity — never corrupt a review. A maintainer can prune to the resolved set after the first live dispatch shows which candidates the Zen endpoint honors.Developer experience
Public-repo PRs are less likely to stall on free-pool exhaustion — when the leading models are capacity-limited, the reviewer falls through to more free candidates before the paid fallbacks.
User experience
No user-facing behavior change; review verdicts for public repositories become available more reliably.
Verification
python3 -m pytest tests/ -q→ 710 passed (full suite on the merged result, including main's newtest_r_coverage_peer_gate; candidate-pool ordering, the exact free-model set, and both public-prefix assertions reconciled to the unioned order). YAML parses; no conflict markers remain.Note for the maintainer
This PR edits the trusted review workflow itself, so under the
pull_request_targettrust boundary it is reviewed by the base branch's (pre-change) model pool — it cannot benefit from its own expanded pool until merged. Merging this tomainis what lets subsequent PRs use the widened pool.🤖 Generated with Claude Code
https://claude.ai/code/session_01SEAGtwNR96cia2djq7XFCo