fix(review): activate 3 declared github-models as review fallback candidates - #651
fix(review): activate 3 declared github-models as review fallback candidates#651seonghobae wants to merge 10 commits into
Conversation
|
Warning Review limit reached
Next review available in: 13 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughR testthat 패키지 로드 실패를 peer R CMD check 디퍼로 분류하고 승인 게이트에 연결했습니다. OpenCode 모델 풀, 실행 사이클, 리뷰 예산을 갱신했으며 current-head 승인 후 Noema 핸드오프와 merge scheduler 환경 전달을 추가했습니다. ChangesR 커버리지 및 승인 흐름
OpenCode 모델 풀 구성
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant CoverageEvidence
participant PeerCheck
participant ApprovalPublish
participant NoemaHandoff
CoverageEvidence->>ApprovalPublish: publish deferred coverage decision
ApprovalPublish->>PeerCheck: verify current-head R CMD check evidence
PeerCheck-->>ApprovalPublish: success or WAITING_FOR_R_CMD_CHECK
ApprovalPublish->>NoemaHandoff: dispatch after current-head approval
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
|
The failing Because Local verification of this diff: 71 pytest passed ( Generated by Claude Code |
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head mergeability evidence and changed-file flow before approval, then found merge conflicts on the affected path.
Findings
1. HIGH Merge Conflict Guidance - Resolve the PR branch against the latest base branch
- Problem: GitHub reports mergeStateStatus
DIRTYfor this pull request. - Root cause: Branch
claude/contextualwisdomlab-audit-governance-qyxe67cannot be merged cleanly intomain; the changed-file flow below shows which review/runtime path is blocked by the conflict. - Fix: Merge or rebase the latest
mainintoclaude/contextualwisdomlab-audit-governance-qyxe67, resolve conflict markers in the PR branch, rerun the focused checks, and push the same branch. - Repair commands:
gh pr checkout 651 --repo ContextualWisdomLab/.github
git fetch origin main
git merge --no-ff origin/main # or: git rebase origin/main
git status --short
# resolve files, then git add <resolved-files>
# merge path: git commit
# rebase path: git rebase --continue
git push origin HEAD:claude/contextualwisdomlab-audit-governance-qyxe67
# rebase path only: git push --force-with-lease origin HEAD:claude/contextualwisdomlab-audit-governance-qyxe67- Regression test: Keep OpenCode approval gated on mergeability so model-output failures cannot approve a conflicted PR.
Merge Conflict Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review-dispatch.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> Conflict["Merge conflict blocks this path"]
Conflict --> V1["actionlint plus required checks"]
Evidence --> S2["Test: test_opencode_agent_contract.py"]
S2 --> I2["regression suite"]
I2 --> Conflict["Merge conflict blocks this path"]
Conflict --> V2["targeted test run"]
- Result: REQUEST_CHANGES
- Reason: mergeStateStatus is
DIRTY; mergeable isCONFLICTING. - Head SHA:
98266f8ce0cfc16dd0793a457e16dc1f6289e6f0 - Workflow run: 30456390044
- Workflow attempt: 2
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review-dispatch.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> Conflict["Merge conflict blocks this path"]
Conflict --> V1["actionlint plus required checks"]
Evidence --> S2["Test: test_opencode_agent_contract.py"]
S2 --> I2["regression suite"]
I2 --> Conflict["Merge conflict blocks this path"]
Conflict --> V2["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage evidence job did not run or did not publish coverage evidence. Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review-dispatch.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review-dispatch.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["Test: test_opencode_agent_contract.py"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test: test_opencode_agent_contract.py"]
R2 --> V2["targeted test run"]
|
Widen the anonymous OpenCode Zen free reviewer pool with three additional verified models so a single model's rate-limit, delisting, or transient outage no longer exhausts the reviewer and blocks org PRs: - opencode-free/mimo-v2.5-free (MiMo-V2.5 Free) - opencode-free/laguna-s-2.1-free (Laguna S 2.1 Free) - opencode-free/ling-3.0-flash-free (Ling-3.0-flash Free) Complements #655/#659 (nemotron-3-ultra-free, deepseek-v4-flash-free): the public-repo prefix now offers six free Zen candidates before the keyed providers. Slugs verified against OpenCode's published Zen catalog (https://opencode.ai/zen/v1). Declared as simple tool-calling models matching the existing merged pattern (no reasoning claim), so the reasoning-effort guard accepts them; gated behind the existing is_private == 'false' public-repository guard so anonymous Zen never sees private code. Contract test updated to pin the expanded candidate list. Verification: - python3 -m pytest tests/test_opencode_agent_contract.py tests/test_opencode_model_pool_runner.py tests/test_assert_opencode_reasoning_effort.py tests/test_opencode_workflow_shell_syntax.py -q -> 71 passed - jq -n over the generated config block: rc 0 - assert_opencode_reasoning_effort.validate_candidate accepts all three - YAML safe_load parses the workflow Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Msz8Dni5zRaqPatNCmZJZ8
98266f8 to
ee8f54d
Compare
seonghobae
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
tests/test_opencode_agent_contract.py (1)
131-133: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win새 free 모델의 생성 설정도 계약 테스트로 검증하세요.
현재 변경은 후보 ID와 순서만 검증합니다.
opencode-free.models에 새 모델이 실제로 존재하고tool_call및limit값이 올바른지는 검증하지 않아, 후보 문자열은 통과하지만 실행 시 설정 누락·오타를 놓칠 수 있습니다. 기존 모델 검증과 동일하게 세 모델의 생성 설정을 직접 확인해 주세요.제안하는 최소 테스트 보강
free_models = generated_config["provider"]["opencode-free"]["models"] + for model_name in ( + "mimo-v2.5-free", + "laguna-s-2.1-free", + "ling-3.0-flash-free", + ): + assert free_models[model_name]["tool_call"] is True + assert free_models[model_name]["limit"] == { + "context": 131000, + "output": 32000, + }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_opencode_agent_contract.py` around lines 131 - 133, Extend the contract test around the new candidate IDs to inspect the corresponding entries in opencode-free.models. Following the existing model-validation pattern, assert that opencode-free, mimo-v2.5-free, laguna-s-2.1-free, and ling-3.0-flash-free have the expected tool_call and limit generation settings, not just the expected IDs and order.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@tests/test_opencode_agent_contract.py`:
- Around line 131-133: Extend the contract test around the new candidate IDs to
inspect the corresponding entries in opencode-free.models. Following the
existing model-validation pattern, assert that opencode-free, mimo-v2.5-free,
laguna-s-2.1-free, and ling-3.0-flash-free have the expected tool_call and limit
generation settings, not just the expected IDs and order.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 4a6db473-2150-4a29-885f-cb805c2c1d92
📒 Files selected for processing (2)
.github/workflows/opencode-review-dispatch.ymltests/test_opencode_agent_contract.py
Address CodeRabbit review on #651: the contract test verified the new free models' candidate IDs and order but not that they actually exist in opencode-free.models with the intended generation settings, so a config typo or omission would pass the candidate-string checks yet fail at runtime. Mirror the existing nemotron/deepseek/north validation for mimo-v2.5-free, laguna-s-2.1-free, and ling-3.0-flash-free: assert tool_call is true, the limit is {"context": 131000, "output": 32000}, and no response_format is set. Verification: python3 -m pytest tests/test_opencode_agent_contract.py -q -> 37 passed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Msz8Dni5zRaqPatNCmZJZ8
Adds three already-declared, tool-capable, non-reasoning github-models to the OpenCode review candidate pool so provider rate-limiting on the OpenAI/DeepSeek families no longer exhausts the fallback chain: - github-models/mistral-ai/mistral-medium-2505 - github-models/meta/llama-4-maverick-17b-128e-instruct-fp8 - github-models/meta/llama-4-scout-17b-16e-instruct These are Mistral and Llama families with rate limits independent of the existing OpenAI/DeepSeek github-models candidates, so they add real free fallback capacity (github-models bills against GITHUB_TOKEN, no extra cost). Each is already defined in opencode.jsonc's github-models provider block and in the workflow, is not reasoning-capable (so the reasoning-effort guard requires no extra config), and is not in the banned-candidate set. Verification: - tests/test_opencode_agent_contract.py — 37 passed (candidate_pairs + github_candidate_models assertions updated to match) - tests/test_assert_opencode_reasoning_effort.py, test_opencode_workflow_shell_syntax.py, test_opencode_model_pool_runner.py — 34 passed - validate_candidate() returns no errors for all three new candidates Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Msz8Dni5zRaqPatNCmZJZ8
…omlab-audit-governance-qyxe67 # Conflicts: # .github/workflows/opencode-review-dispatch.yml # tests/test_opencode_agent_contract.py
seonghobae
changed the title
There was a problem hiding this comment.
🧹 Nitpick comments (3)
tests/test_opencode_agent_contract.py (1)
167-214: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valuereasoning-effort 검증 for 루프가 중복 작성됨
Line 187-192의
for model_name, model_config in free_models.items(): ...루프가 Line 209-214에 동일하게 한 번 더 반복되어 있습니다. 두 루프는 완전히 같은 컬렉션(free_models)에 대해 같은 조건과 같은 단언을 수행하므로 후자는 실질적으로 아무 것도 추가 검증하지 않는 사문(死文)입니다. 사이에 추가된 개별 모델 limit 단언들과 분리하려다 실수로 복사된 것으로 보입니다.♻️ 중복 루프 제거 제안
assert free_models["mimo-v2.5-free"]["limit"] == { "context": 200000, "output": 32000, } - for model_name, model_config in free_models.items(): - if model_config.get("reasoning") is True: - assert model_config["options"]["reasoningEffort"] == "high", model_name - assert model_config["variants"]["high"]["reasoningEffort"] == "high", ( - model_name - ) assert github_candidate_models == [🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_opencode_agent_contract.py` around lines 167 - 214, Remove the duplicated reasoning-effort validation loop over free_models, keeping the first loop and all individual model limit assertions unchanged..github/workflows/opencode-review-dispatch.yml (2)
6811-6814: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win모델 불가용 폴백 경로에서 R CMD check 요구 실패 시 메시지가 일반화됨
publish_blockers_after_model_unavailable()에서require_r_cmd_check_for_deferred_coverage가 실패하면 단순히return 1하여 결국stop_without_review_after_model_unavailable(범용 "MODEL_OUTPUT_UNAVAILABLE" 메시지)로 빠집니다. 반면 정상 승인 흐름(Line 7199)에서는 동일 조건에서hold_approval_without_review "WAITING_FOR_R_CMD_CHECK" ...로 구체적인 원인과 해결 방법을 안내합니다. 두 경로의 사용자 경험이 불일치하므로, 폴백 경로에도 동일한WAITING_FOR_R_CMD_CHECK본문을 재사용하도록 맞추는 것을 고려해 보세요.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/opencode-review-dispatch.yml around lines 6811 - 6814, Update publish_blockers_after_model_unavailable() so a failed require_r_cmd_check_for_deferred_coverage call uses the same WAITING_FOR_R_CMD_CHECK hold-approval message and body as the normal approval flow, rather than returning 1 and falling through to the generic MODEL_OUTPUT_UNAVAILABLE path. Reuse the existing message implementation from the normal flow and preserve the existing success behavior.
807-861: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
run_and_capture계열의 샌드박스 래퍼를 공통 헬퍼로 묶어주세요setpriv/timeout/환경 마스킹/로그 캡처가run_and_capture,run_and_capture_advisory,run_r_package_testthat에 반복됩니다. command와 일부 env만 파라미터화하면 한쪽만 수정돼 설정이 어긋나는 drift를 줄일 수 있습니다.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/opencode-review-dispatch.yml around lines 807 - 861, Extract the duplicated sandbox execution, environment masking, timeout, and log-capture logic from run_and_capture, run_and_capture_advisory, and run_r_package_testthat into one shared helper. Parameterize the command and only the environment differences, then update all three callers to use it while preserving their existing exit-code handling, logging, and advisory/classification behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In @.github/workflows/opencode-review-dispatch.yml:
- Around line 6811-6814: Update publish_blockers_after_model_unavailable() so a
failed require_r_cmd_check_for_deferred_coverage call uses the same
WAITING_FOR_R_CMD_CHECK hold-approval message and body as the normal approval
flow, rather than returning 1 and falling through to the generic
MODEL_OUTPUT_UNAVAILABLE path. Reuse the existing message implementation from
the normal flow and preserve the existing success behavior.
- Around line 807-861: Extract the duplicated sandbox execution, environment
masking, timeout, and log-capture logic from run_and_capture,
run_and_capture_advisory, and run_r_package_testthat into one shared helper.
Parameterize the command and only the environment differences, then update all
three callers to use it while preserving their existing exit-code handling,
logging, and advisory/classification behavior.
In `@tests/test_opencode_agent_contract.py`:
- Around line 167-214: Remove the duplicated reasoning-effort validation loop
over free_models, keeping the first loop and all individual model limit
assertions unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: ecffc7e1-2d21-4b20-9f72-a5836ae275f1
📒 Files selected for processing (2)
.github/workflows/opencode-review-dispatch.ymltests/test_opencode_agent_contract.py
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head56cc57b03c64727fcd489f2ca6184e4e07556dca. -
Head SHA:
56cc57b03c64727fcd489f2ca6184e4e07556dca -
Workflow run: 30527071364
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review-dispatch.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review-dispatch.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["Test: test_opencode_agent_contract.py"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test: test_opencode_agent_contract.py"]
R2 --> V2["targeted test run"]
2 participants
Summary
After rebasing onto current
main, the three OpenCode Zen free models this PRoriginally added (
mimo-v2.5-free,laguna-s-2.1-free,ling-3.0-flash-free)are already present in
main(merged via the sibling Zen-pool widening,alongside
big-pickle). This PR was reduced to its one remaining non-redundantdelta: activating three github-models candidates that
maindeclares in theprovider block but never places in the reviewer rotation.
The
github-modelsprovider block already declares these three models, butOPENCODE_MODEL_CANDIDATESstopped atdeepseek/deepseek-r1, so they were deaddeclarations the pool never tried. This PR appends them to the candidate list so
a rate-limited or delisted leader falls through to three additional keyed
(GITHUB_TOKEN, free-tier) fallbacks before the pool is exhausted — directly
widening reviewer resilience during the org-wide review stall:
github-models/mistral-ai/mistral-medium-2505github-models/meta/llama-4-maverick-17b-128e-instruct-fp8github-models/meta/llama-4-scout-17b-16e-instructWhy this is safe
github-modelsprovider block onmain; this only adds them to the rotation, no new provider config.openai/gpt-5*/o3*/o4*/deepseek-r1*reasoning set), soscripts/ci/assert_opencode_reasoning_effort.py::validate_candidateacceptsthem with no
reasoning/reasoningEffortrequirement (verified green).scripts/ci/changes, so the 100% coverage / docstring gates areunaffected.
Verification
python3 -m pytest tests -q— 710 passed (full suite, incl. thetest_opencode_agent_contract.pycandidate-pair +github_candidate_modelscontracts updated to pin the three new candidates).
yaml.safe_loadparses the workflow; no conflict markers remain.Files
.github/workflows/opencode-review-dispatch.yml— three github-modelscandidates appended to
OPENCODE_MODEL_CANDIDATES.tests/test_opencode_agent_contract.py— pins the expanded candidate list.Summary by CodeRabbit
개선 사항
버그 수정