ci(cli): validate npm tarball across platforms - #3185
Conversation
📝 WalkthroughSummaryThis PR adds a read-only release gate for the npm CLI artifact. It builds one immutable tarball and tests that exact artifact on Linux, macOS arm64, and Windows x64 with Node.js 22.19 and 24. Offline smoke tests run outside the repository with an empty npm cache. They cover both CLI bins, Eval assets, PTY and native locks, TUI startup, Runtime Host lifecycle, and a filesystem-backed model turn. The release path also enforces limits for compressed size, unpacked size, and file count. Source of truthThe PR extends the existing release packaging and validation path. It does not create a parallel packaging path. The workflow reuses one packaged artifact for all platform and runtime checks. Scope and complexityThe solution is coherent with the release-gate objective. The smoke-test complexity supports cross-platform process control, cleanup, timeouts, isolated environments, bounded output, and installed-product validation. The artifact policy tests provide focused regression coverage for release growth. No safe deletion or simplification is evident from the supplied changes. Removing the added controls or checks would weaken artifact, runtime, or cross-platform coverage. Complexity deltaThe PR adds one read-only workflow, one immutable-artifact handoff, a platform and Node.js version matrix, artifact metric limits, and extensive installed-product test states. It adds exported policy constants and validation functions, but it does not add a new packaging authority or user-facing runtime API. The PR adds workflow branches for platform-specific process termination and cleanup. It also adds test-maintenance burden through PTY, TUI, Runtime Host, mock-provider, timeout, and filesystem-worker scenarios. The single-artifact design removes the risk of validating separately rebuilt packages across jobs. The focused policy test centralizes artifact ceilings. Total maintenance complexity increases, but the increase is justified by the cross-platform release coverage and the absence of a parallel packaging path. Risks and validationThe workflow can block releases when artifact metrics exceed the configured ceilings. Cross-platform process and TUI tests can be sensitive to timing, environment differences, and cleanup failures. The offline mock-provider test can fail if the installed artifact omits required runtime assets. Reported local validation passed tarball packaging, installed-product smoke tests, release policy tests, lint, format, CLI typecheck, notice checks, and actionlint. The aggregate Review-relevant risksThe PR changes release governance by adding a read-only cross-platform artifact gate. Material governance or release changes require independent human review under repository policy. The PR changes packaged-artifact validation and can prevent releases that exceed reviewed size or file-count limits. Material release behavior changes require independent human review under repository policy. The workflow downloads and executes the packaged CLI in isolated environments. This affects release and security review surfaces. Material security or release changes require independent human review under repository policy. The packaging path validates third-party notices. Material licensing or attribution effects require independent human review under repository policy. No direct user-facing API or runtime behavior change was identified in the current diff. The person performing the merge must review the final diff. A maintainer makes the final determination. WalkthroughThe PR adds CLI release artifact limits, package validation, comprehensive installed-package smoke tests, and a GitHub Actions workflow that tests immutable artifacts across Linux, macOS, and Windows. ChangesCLI Release Validation
Estimated code review effort: 4 (Complex) | ~45 minutes Merge Risk: 🔵 Low · up to This PR adds cross-platform validation of the packaged CLI, but the validation path still has bounded risks from unpinned npm transitive dependencies, Windows cleanup and path handling, and a brittle TUI assertion that could cause nondeterministic failures or long CI timeouts. It is mergeable with explicit owner awareness and follow-up on these reliability and reproducibility issues. Sequence Diagram(s)sequenceDiagram
participant GitHubActions
participant ReleasePackager
participant ArtifactStore
participant SmokeVerifier
participant RuntimeHost
participant MockProvider
GitHubActions->>ReleasePackager: create and validate CLI tarball
ReleasePackager->>ArtifactStore: upload package and metadata
GitHubActions->>ArtifactStore: download immutable artifact
GitHubActions->>SmokeVerifier: run platform smoke test
SmokeVerifier->>RuntimeHost: install and execute CLI
RuntimeHost->>MockProvider: discover model and process requests
RuntimeHost-->>SmokeVerifier: return runtime and filesystem results
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
440d137 to
74d4d39
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
scripts/smoke-release-cli-package.mjs (1)
316-324: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueCheck
provider.errorbefore the sentinel assertion.A provider-side failure returns HTTP 500, so the CLI output lacks
RESPONSE_SENTINEL. Line 319 then throws the genericExpected output to contain "MAKA_RELEASE_SMOKE_OK"error, and the precise provider error at line 323 never surfaces. Report the root cause first.♻️ Proposed reordering
if (result.exitCode !== 0) { throw new Error(`Controlled maka run exited with ${result.exitCode}: ${result.stderr}`); } + if (provider.error) throw provider.error; assertOutput(result.stdout, RESPONSE_SENTINEL); if (!provider.sawReadTool || !provider.sawFileSentinel) { throw new Error('Controlled maka run did not execute the installed filesystem worker'); } - if (provider.error) throw provider.error; behaviorCompleted = true;
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 665d1b16-c876-4def-be02-fa26bac39752
📒 Files selected for processing (6)
.github/workflows/cli-package-validation.ymlpackage.jsonscripts/release-cli-artifact-policy.mjsscripts/release-cli-artifact-policy.test.mjsscripts/release-cli-package.mjsscripts/smoke-release-cli-package.mjs
Included review availability: Your plan includes up to 3 reviews per rolling hour; 1 remains after this review.
36c48fc to
c57337f
Compare
Build the CLI package once and exercise that immutable artifact on the minimum Node runtime plus Linux, macOS arm64, and Windows. The installed smoke uses production configuration and protocol paths so native modules, TUI setup, Runtime Host teardown, and a filesystem-backed model turn are validated without repository dependencies. Keep publishing authority out of this workflow and fail release builds on reviewed artifact growth. Generated-by: OpenAI Codex
Run the installed CLI checks in a child process so Windows unloads native addons and ConPTY before the parent removes the temporary installation. Preserve validation and cleanup failures independently, and always settle an exact Runtime Host before reporting lifecycle regressions. Use the installed cross-spawn implementation for Windows command shims, rely on stable TUI command tokens, and parse JSON records independently of ConPTY control sequences. Generated-by: OpenAI Codex
c57337f to
268a026
Compare
PR Summary by QodoValidate immutable npm CLI tarball across supported platforms
AI Description
Diagram
High-Level Assessment
Files changed (6)
|
|
I have manually reviewed this PR, confirmed that it has no major impact and is easy to roll back, and I take responsibility for this result. |
Code Review by Qodo🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)
Great, no issues found!Qodo reviewed your code and found no material issues that require reviewTip of the day💡 Did you know, you can keep summaries lean with Finding overflow, which tucks the rest behind 'View more' |
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: c64c1cdf-774a-47fa-b820-b392c62c5d42
📒 Files selected for processing (2)
.github/workflows/cli-package-validation.ymlscripts/smoke-release-cli-package.mjs
🚧 Files skipped from review as they are similar to previous changes (1)
- scripts/smoke-release-cli-package.mjs
Included review availability: Your plan includes up to 3 reviews per rolling hour; 0 remain after this review.
| node-version: '22.19.0' | ||
| cache: npm | ||
| - name: Select the release npm toolchain | ||
| run: npm install --global --no-audit --no-fund npm@11.12.1 |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- workflow references ---'
rg -n -C 4 'npm install --global|npm@11\.12\.1|package-lock|corepack|pnpm|yarn|npm ci|npm install' .github/workflows/cli-package-validation.yml
printf '%s\n' '--- repository package-manager files ---'
git ls-files | rg '(^|/)(package\.json|package-lock\.json|npm-shrinkwrap\.json|pnpm-lock\.yaml|yarn\.lock|\.npmrc|\.nvmrc|\.node-version|packageManager)$' || true
printf '%s\n' '--- package-manager declarations ---'
for f in package.json .npmrc .nvmrc .node-version; do
if [ -f "$f" ]; then
echo "### $f"
cat -n "$f"
fi
doneRepository: maka-agent/maka-agent
Length of output: 8024
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
import io
import json
import tarfile
import urllib.request
version = "11.12.1"
url = f"https://registry.npmjs.org/npm/{version}"
with urllib.request.urlopen(url, timeout=20) as response:
metadata = json.load(response)
print("--- registry metadata ---")
print("name:", metadata.get("name"))
print("version:", metadata.get("version"))
dist = metadata.get("dist", {})
print("tarball:", dist.get("tarball"))
print("integrity:", dist.get("integrity"))
print("shasum:", dist.get("shasum"))
tarball_url = dist["tarball"]
with urllib.request.urlopen(tarball_url, timeout=30) as response:
data = response.read()
print("--- published package files ---")
with tarfile.open(fileobj=io.BytesIO(data), mode="r:gz") as archive:
names = archive.getnames()
for name in names:
if name in {
"package/package.json",
"package/package-lock.json",
"package/npm-shrinkwrap.json",
} or name.endswith(("/package-lock.json", "/npm-shrinkwrap.json")):
print(name)
package_json = json.load(archive.extractfile("package/package.json"))
print("--- npm package manifest ---")
print("packageManager:", package_json.get("packageManager"))
print("dependencies:", len(package_json.get("dependencies", {})))
print("optionalDependencies:", len(package_json.get("optionalDependencies", {})))
print("bundledDependencies:", package_json.get("bundledDependencies"))
PYRepository: maka-agent/maka-agent
Length of output: 2445
🏁 Script executed:
#!/bin/bash
set -eu
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
curl -fsSLk 'https://registry.npmjs.org/npm/11.12.1' > "$tmpdir/metadata.json"
python3 - "$tmpdir/metadata.json" <<'PY'
import json, sys
metadata = json.load(open(sys.argv[1]))
dist = metadata["dist"]
print("--- registry metadata ---")
print("name:", metadata["name"])
print("version:", metadata["version"])
print("tarball:", dist["tarball"])
print("integrity:", dist.get("integrity"))
print("shasum:", dist.get("shasum"))
PY
tarball="$(python3 - "$tmpdir/metadata.json" <<'PY'
import json, sys
print(json.load(open(sys.argv[1]))["dist"]["tarball"])
PY
)"
curl -fsSLk "$tarball" > "$tmpdir/npm.tgz"
printf '%s\n' '--- published lockfile names ---'
tar -tzf "$tmpdir/npm.tgz" | grep -E '(^|/)(package-lock\.json|npm-shrinkwrap\.json)$' || true
printf '%s\n' '--- published npm manifest dependency fields ---'
tar -xOf "$tmpdir/npm.tgz" package/package.json |
python3 -c 'import json,sys; p=json.load(sys.stdin); print("packageManager:",p.get("packageManager")); print("dependencies:",len(p.get("dependencies",{}))); print("optionalDependencies:",len(p.get("optionalDependencies",{}))); print("bundledDependencies:",p.get("bundledDependencies"))'Repository: maka-agent/maka-agent
Length of output: 591
Pin npm’s dependency tree, not only its version.
The root package.json pins npm@11.12.1, but the published package has no lockfile or shrinkwrap and declares 65 dependencies. Both global installs can therefore resolve transitive dependencies outside the repository lockfile. Add a reviewed mechanism that pins the complete npm toolchain.
🧰 Tools
🪛 zizmor (1.29.0)
[warning] 82-82: ad-hoc installation of packages (adhoc-packages): installs a package outside of a lockfile
(adhoc-packages)
Sources: Path instructions, Linters/SAST tools
1 participant
English
Summary
Adds a read-only release gate for the npm CLI artifact introduced by #3173. The workflow builds one tarball, then tests that exact artifact on Linux (minimum and current Node), macOS arm64, and Windows x64.
The installed-product smoke runs outside the repository with an empty offline npm cache and covers both bins, Eval assets, PTY and native locks, TUI setup, Runtime Host lifecycle, and a filesystem-backed model turn. It also fails packaging when the artifact exceeds reviewed size or file-count limits.
Refs #3166
Verification
check:releasestopped on a local pre-existing@maka/uistale-build timestamp; its affected checks were run individually.AI use
Select exactly one:
Tool(s) and scope: OpenAI Codex implemented the workflow, artifact policy, installed-product smoke, and tests under human direction and review.
Checklist
Does this PR entail a change in behavior?
中文
概要
为 #3173 引入的 npm CLI 产物增加只读发布门禁。工作流只构建一次 tarball，并在 Linux（最低及当前 Node）、macOS arm64 和 Windows x64 上验证同一个产物。
安装态冒烟测试在仓库外、空离线 npm 缓存中运行，覆盖两个 bin、Eval 资源、PTY 和原生文件锁、TUI 配置、Runtime Host 生命周期，以及通过文件系统 worker 完成的受控模型回合。打包产物超过已审定的体积或文件数上限时也会失败。
Refs #3166
验证
check:release因本地已有的@maka/ui构建时间戳陈旧而停止；其受影响检查已分别通过。AI 使用
请且仅选择一项：
工具及范围：OpenAI Codex 在人工指导与审核下实现了工作流、产物策略、安装态冒烟测试及相关测试。
检查清单
本 PR 是否包含行为变更？