fix(ci): prefetch trusted pnpm store for coverage - #642
Conversation
📝 WalkthroughWalkthrough검증된 base 커밋의 pnpm 입력을 materialize하고, trusted coverage 이미지에서 오프라인 pnpm store를 구성하도록 변경했다. PR head 설치와 JavaScript coverage 탐색·threshold 조건을 조정했으며, materializer와 워크플로 계약 테스트를 추가했다. Changes오프라인 JavaScript coverage 증거
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant BaseCommit as base commit
participant Materializer as materialize_base_javascript_packages.py
participant CoverageImage as trusted coverage image
participant PRHead as PR head
participant CoverageGate as javascript_coverage_gate.py
BaseCommit->>Materializer: pnpm-lock.yaml와 관련 입력 읽기
Materializer->>CoverageImage: materialized base inputs와 manifest 전달
CoverageImage->>CoverageImage: pnpm fetch로 /opt/pnpm-store 생성
PRHead->>CoverageImage: 오프라인 pnpm install 실행
CoverageImage->>CoverageGate: COVERAGE_SOURCE_WORKDIR 기반 증거 전달
CoverageGate->>CoverageGate: JS/TS coverage 실행 여부에 따라 threshold 검사
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
scripts/ci/materialize_base_javascript_packages.py (1)
62-129: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win
package.json을 두 번git show로 가져옴 (중복 서브프로세스 호출).Line 85-87에서
package.json내용을 파싱하기 위해 이미_git(... "show" ...)로 가져왔는데, Line 107-115의base_inputs딕셔너리 컴프리헨션에서"package.json"이PNPM_BASE_INPUT_NAMES에 포함되어 있고regular_paths에도 존재하므로 동일한 blob을 다시git show로 가져옵니다. 프로젝트당 불필요한 서브프로세스 호출이 하나 추가됩니다.♻️ 이미 가져온 바이트를 재사용하도록 수정
try: - package_data: Any = json.loads( - _git(repo_root, "show", f"{base_sha}:{package_path}").decode("utf-8") - ) + package_bytes = _git(repo_root, "show", f"{base_sha}:{package_path}") + package_data: Any = json.loads(package_bytes.decode("utf-8")) except (UnicodeDecodeError, json.JSONDecodeError) as exc: raise ValueError( f"trusted base package manifest {package_path} is invalid JSON: {exc}" ) from exc @@ base_inputs = { input_name: _git( repo_root, "show", f"{base_sha}:{project_root / input_name}", ) for input_name in PNPM_BASE_INPUT_NAMES - if str(project_root / input_name) in regular_paths + if input_name != "package.json" and str(project_root / input_name) in regular_paths } + base_inputs["package.json"] = package_bytes base_inputs["pnpm-lock.yaml"] = lock_content🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/ci/materialize_base_javascript_packages.py` around lines 62 - 129, Update base_pnpm_projects to retain the bytes read for package.json while parsing it, then reuse those bytes when constructing base_inputs instead of invoking _git again through the PNPM_BASE_INPUT_NAMES comprehension. Preserve the existing validation and include package.json in base_inputs with the same content.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/opencode-review-dispatch.yml:
- Around line 597-616: Update the trusted base package prefetch loop around the
package_manager validation so unsupported pnpm versions emit a diagnostic and
skip that project’s pnpm fetch instead of exiting the Docker build. Preserve
prefetching for projects using pnpm@11.5.3 and continue processing the remaining
manifest entries.
In `@scripts/ci/materialize_base_javascript_packages.py`:
- Around line 34-59: _regular_base_paths의 유효한 형식이지만 필터에서 제외되는 엔트리를 검증하는 테스트를
추가하세요. tests/test_materialize_base_javascript_packages.py에서 mode=120000 symlink와
mode=160000 gitlink 엔트리를 포함한 git ls-tree 출력을 구성하고, object_type == "blob" 및
mode.startswith("100") 조건에 따라 해당 경로들이 결과에서 제외되는지 확인하세요.
---
Nitpick comments:
In `@scripts/ci/materialize_base_javascript_packages.py`:
- Around line 62-129: Update base_pnpm_projects to retain the bytes read for
package.json while parsing it, then reuse those bytes when constructing
base_inputs instead of invoking _git again through the PNPM_BASE_INPUT_NAMES
comprehension. Preserve the existing validation and include package.json in
base_inputs with the same content.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: a3c49c6d-5114-413d-b0f9-8c15d50fd186
📒 Files selected for processing (5)
.github/workflows/opencode-review-dispatch.ymlscripts/ci/materialize_base_javascript_packages.pyscripts/ci/test_strix_quick_gate.shtests/test_materialize_base_javascript_packages.pytests/test_opencode_agent_contract.py
| COPY base-javascript-packages /tmp/base-javascript-packages | ||
| RUN set -eu; \ | ||
| mkdir -p /opt/pnpm-store; \ | ||
| jq -r '.[] | [.directory, .package_manager] | @tsv' \ | ||
| /tmp/base-javascript-packages/manifest.json \ | ||
| | while IFS="$(printf '\t')" read -r project_dir package_manager; do \ | ||
| [ -n "$project_dir" ] || continue; \ | ||
| if [ "$package_manager" != "pnpm@11.5.3" ]; then \ | ||
| printf 'Unsupported trusted base package manager: %s\n' "$package_manager" >&2; \ | ||
| exit 1; \ | ||
| fi; \ | ||
| cd "/tmp/base-javascript-packages/${project_dir}"; \ | ||
| pnpm fetch \ | ||
| --frozen-lockfile \ | ||
| --ignore-scripts \ | ||
| --store-dir /opt/pnpm-store; \ | ||
| done; \ | ||
| chmod -R a+rX /opt/pnpm-store; \ | ||
| rm -rf /tmp/base-javascript-packages | ||
| COPY requirements-opencode-review-ci-hashes.txt /tmp/requirements-opencode-review-ci-hashes.txt |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
pnpm 버전이 정확히 11.5.3이 아니면 전체 coverage-tool 이미지 빌드가 실패해, 모든 언어의 coverage-evidence가 함께 막힙니다.
if [ "$package_manager" != "pnpm@11.5.3" ]; then ... exit 1; fi은 base 커밋의 어떤 pnpm-lock.yaml/package.json이든 packageManager가 pnpm@11.5.3이 아니면 즉시 exit 1합니다. 이 블록은 트러스티드 네트워크 Docker 이미지 빌드(docker build) 단계에서 실행되며, 이 이미지 빌드는 Python/R/Rust 등 언어에 관계없이 모든 PR에서 공통으로 수행됩니다. 기존에는 pnpm 버전 불일치가 sandbox 내부의 JS 전용 ensure_corepack_runner()에서만 감지되어 해당 언어의 coverage만 실패시켰지만(다른 언어 coverage는 계속 진행), 이번 변경으로 인해 image build 단계에서 하드 실패하면서 PR의 coverage-evidence 전체(Python/Rust/R 포함)가 차단됩니다. repository_dispatch로 여러 대상 저장소를 리뷰하는 구조상, pnpm 11.5.3 이외 버전을 쓰는 대상 저장소가 하나라도 있으면 해당 저장소의 모든 PR이 이 시점에서 막힐 수 있습니다.
빌드를 중단시키는 대신, 지원되지 않는 pnpm 버전의 프로젝트는 로그만 남기고 prefetch를 건너뛰도록(그래서 이후 JS 단계에서만 개별적으로 실패하도록) 완화하는 것을 고려해주세요.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/opencode-review-dispatch.yml around lines 597 - 616,
Update the trusted base package prefetch loop around the package_manager
validation so unsupported pnpm versions emit a diagnostic and skip that
project’s pnpm fetch instead of exiting the Docker build. Preserve prefetching
for projects using pnpm@11.5.3 and continue processing the remaining manifest
entries.
| def _regular_base_paths(repo_root: pathlib.Path, base_sha: str) -> set[str]: | ||
| """Return regular blob paths from the exact validated base commit.""" | ||
| entries = _git(repo_root, "ls-tree", "-r", "-z", "--full-tree", base_sha) | ||
| paths: set[str] = set() | ||
| for raw_entry in entries.split(b"\0"): | ||
| if not raw_entry: | ||
| continue | ||
| metadata, separator, raw_path = raw_entry.partition(b"\t") | ||
| if not separator: | ||
| raise RuntimeError("git ls-tree returned a malformed entry") | ||
| fields = metadata.split() | ||
| if len(fields) != 3: | ||
| raise RuntimeError("git ls-tree returned malformed metadata") | ||
| mode, object_type, _object_id = ( | ||
| field.decode("ascii", errors="strict") for field in fields | ||
| ) | ||
| path = raw_path.decode("utf-8", errors="surrogateescape") | ||
| candidate = pathlib.PurePosixPath(path) | ||
| if ( | ||
| object_type == "blob" | ||
| and mode.startswith("100") | ||
| and not candidate.is_absolute() | ||
| and ".." not in candidate.parts | ||
| ): | ||
| paths.add(path) | ||
| return paths |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check whether the negative-filter branch (non-blob / non-100* mode) is exercised by any test.
rg -n "120000|160000|object_type|symlink|submodule" tests/test_materialize_base_javascript_packages.pyRepository: ContextualWisdomLab/.github
Length of output: 428
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the implementation around the branch in question.
sed -n '1,140p' scripts/ci/materialize_base_javascript_packages.py
printf '\n--- TEST FILE ---\n'
# Show the relevant test sections, including any symlink/submodule/tree-entry cases.
sed -n '1,340p' tests/test_materialize_base_javascript_packages.pyRepository: ContextualWisdomLab/.github
Length of output: 16622
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find all references to the helper and tree-entry filtering.
rg -n "_regular_base_paths|ls-tree|120000|160000|submodule|symlink|blob" scripts/ci tests
printf '\n--- TREE OF RELEVANT TESTS ---\n'
fd -a "test_materialize_base_javascript_packages.py|materialize_base_javascript_packages.py" scripts tests
printf '\n--- TARGETED TEST SECTION ---\n'
sed -n '70,170p' tests/test_materialize_base_javascript_packages.pyRepository: ContextualWisdomLab/.github
Length of output: 21008
_regular_base_paths의 필터 제외 경로 테스트를 추가하세요. tests/test_materialize_base_javascript_packages.py는 malformed entry/metadata만 다루고 있어서, mode=120000 symlink나 mode=160000 gitlink처럼 형식은 맞지만 object_type == "blob" and mode.startswith("100")에서 제외되는 엔트리를 직접 커버하지 않습니다.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/ci/materialize_base_javascript_packages.py` around lines 34 - 59,
_regular_base_paths의 유효한 형식이지만 필터에서 제외되는 엔트리를 검증하는 테스트를 추가하세요.
tests/test_materialize_base_javascript_packages.py에서 mode=120000 symlink와
mode=160000 gitlink 엔트리를 포함한 git ls-tree 출력을 구성하고, object_type == "blob" 및
mode.startswith("100") 조건에 따라 해당 경로들이 결과에서 제외되는지 확인하세요.
Source: Coding guidelines
|
Controlled governance bootstrap audit for exact head
The repository ruleset requires two approvals but no independent model reviewer is available. I will temporarily disable only ruleset |
1 participant
Summary
pnpm install --offlinein the PR-head sandboxFailure evidence
--network=noneand ended withEAI_AGAIN/ timeoutERR_PNPM_PATCH_FILE_PATH_MISSING; the corrected materializer includes the base workspace and patch inputsValidation
python3 -m pytest -q --cov=scripts/ci --cov-report=term-missing: 674 passed, 5,703 statements, 100%python3 -m interrogate -v scripts/ci: 100%ruff check .actionlint -shellcheck=pnpm install --offline: 489 packages, no downloadsSummary by CodeRabbit
개선 사항
테스트