Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/build-binaries.yml
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,11 @@ jobs:
node scripts/release-artifacts.mjs freeze .artifacts/release
npm run build:offline
git diff --exit-code HEAD
- name: Setup pinned Bun for real package-manager fixtures
if: steps.restore.outputs.restored != 'true'
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: '1.3.11'
- name: Install pinned pnpm for real package-manager fixtures
if: steps.restore.outputs.restored != 'true'
run: npm install --global --ignore-scripts pnpm@10.28.2
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ jobs:

- uses: ./.github/actions/setup

- name: Setup pinned Bun for real package-manager fixtures
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: '1.3.11'

- name: Install pinned pnpm for real package-manager fixtures
run: npm install --global --ignore-scripts pnpm@10.28.2

Expand Down
14 changes: 10 additions & 4 deletions .github/workflows/daytona-behaviors.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,13 @@ jobs:
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: '22.19.0'
- name: Install pinned npm and locked dependencies
- name: Setup pinned Bun for installation harness tests
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: '1.3.11'
- name: Install pinned npm, pnpm and locked dependencies
run: |
npm install -g npm@11.16.0 --ignore-scripts
npm install -g npm@11.16.0 pnpm@10.28.2 --ignore-scripts
npm ci --ignore-scripts
- name: Freeze, build offline, prepare and verify exact candidate bytes
run: |
Expand All @@ -48,16 +52,18 @@ jobs:
"$RUNNER_TEMP/behavior-test-venv/bin/python" -m pip install -r scripts/daytona-behaviors/requirements.txt
"$RUNNER_TEMP/behavior-test-venv/bin/python" -m unittest discover -s scripts/daytona-behaviors -p 'test_*.py'
"$RUNNER_TEMP/behavior-test-venv/bin/python" -m unittest discover -s scripts -p 'test_verify_candidate_artifacts.py'
node --test scripts/artifact-installation.test.mjs scripts/artifact-acceptance.test.mjs scripts/daytona-behaviors-workflow.test.mjs scripts/daytona-behaviors/fixture.test.mjs scripts/daytona-behaviors/live-guard.test.mjs
node --test scripts/artifact-installation.test.mjs scripts/artifact-acceptance.test.mjs scripts/daytona-behaviors-workflow.test.mjs scripts/daytona-behaviors/fixture.test.mjs scripts/daytona-behaviors/live-guard.test.mjs scripts/daytona-behaviors/npm-global-install.test.mjs
- name: Bundle checked behavior harness without rebuilding the candidate
run: |
mkdir -p .artifacts/daytona-bundle
tar -czf .artifacts/daytona-bundle/bundle.tar.gz \
scripts/artifact-installation.mjs \
scripts/npm-lifecycle-registry.mjs \
scripts/verify-candidate-artifacts.py \
scripts/fixtures/artifact-handoff.mjs \
scripts/daytona-behaviors/installed.py \
scripts/daytona-behaviors/install.mjs \
scripts/daytona-behaviors/npm-global-install.mjs \
scripts/daytona-behaviors/fixture.mjs \
scripts/daytona-behaviors/live-guard.mjs \
scripts/daytona-behaviors/baselines
Expand Down Expand Up @@ -301,7 +307,7 @@ jobs:
failed=1
fi
if [[ "$mode" == deterministic ]]; then
expected='["artifact-integrity","npm-package-integrity","bun-package-integrity","npm-installation","npm-print","npm-builtin-auth-transport","npm-terminal-tools-cancel-resume","npm-artifact-terminal","bun-installation","bun-print","bun-builtin-auth-transport","bun-terminal-tools-cancel-resume","bun-artifact-terminal"]'
expected='["artifact-integrity","npm-global-installation","npm-package-integrity","bun-package-integrity","npm-installation","npm-print","npm-builtin-auth-transport","npm-terminal-tools-cancel-resume","npm-artifact-terminal","bun-installation","bun-print","bun-builtin-auth-transport","bun-terminal-tools-cancel-resume","bun-artifact-terminal"]'
else
expected='["artifact-integrity","npm-package-integrity","npm-live-openai"]'
fi
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Diffex is a lightweight, self-evolving coding agent that lives in your CLI. Ask
Requires Node.js 22.19 or newer and npm or pnpm.

```bash
npm install -g --ignore-scripts @diffexai/diffex
npm i -g @diffexai/diffex
cd your-project
diffex
```
Expand Down
24 changes: 24 additions & 0 deletions docs/daytona-behaviors.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ The [2026-10-27 qualification record](daytona-behaviors-qualification.md) docume
This is a standalone acceptance gate for the packaged Diffex CLI, not the hosted product's runtime image, a release publisher or a replacement for native artifact acceptance.
It installs the exact reviewed candidate tarballs into isolated consumers outside the repository.
The deterministic mode controls inference while exercising installed behavior; the live mode uses a bounded real OpenAI request path.
Npm behavior consumers explicitly enable dependency lifecycle scripts; Bun consumers retain the scripts-disabled path.
A separate deterministic `npm-global-installation` scenario checks normal global npm installation of both the exact candidate tarball and the currently published CLI, each in its own fresh isolated prefix and cache.
A passing deterministic run is not live-provider evidence.
The snapshot is toolchain-only; each run uploads its candidate artifacts and checked harness rather than baking repository source or credentials into a snapshot.
Version 1 covers the Linux installed CLI only; hosted-runtime behavior and subscription/OAuth login qualification are out of scope.
Expand Down Expand Up @@ -47,10 +49,12 @@ The workflow never sets its own qualification variable.

```text
scripts/artifact-installation.mjs
scripts/npm-lifecycle-registry.mjs
scripts/verify-candidate-artifacts.py
scripts/fixtures/artifact-handoff.mjs
scripts/daytona-behaviors/installed.py
scripts/daytona-behaviors/install.mjs
scripts/daytona-behaviors/npm-global-install.mjs
scripts/daytona-behaviors/fixture.mjs
scripts/daytona-behaviors/live-guard.mjs
scripts/daytona-behaviors/baselines/
Expand Down Expand Up @@ -92,6 +96,26 @@ The release and bundle artifacts contain no credentials; per-mode and aggregate
They are workflow artifacts, not npm or release assets; that distinction does not make them private.
Never rely on GitHub log masking or artifact access controls to protect keys.

## Normal npm installation validation

The `npm-global-installation` scenario runs the advertised `npm i -g @diffexai/diffex` command without `--ignore-scripts`, after verifying that isolated npm configuration has `ignore-scripts=false`.
The separate candidate installation serves its retained tarball through a loopback-only registry, not an unpublished public registry version.
The fixture advertises npm's `_hasShrinkwrap` metadata only after validating the embedded lock. This matches published-package resolution: npm 10 can ignore a nested shrinkwrap when the dependency is supplied directly as a local tarball.
Synchronous local npm consumers seed their root installation lock from the same verified candidate shrinkwrap, including exact archive integrity, rather than letting a file-based install re-resolve reviewed dependency ranges.
The public installation uses the independently verified public npm archive: a candidate and the public package may share a version while having different bytes, so their inventories must never be interchanged.
Public metadata and tarball fetches reject redirects, enforce bounded reads, and verify SHA-512 registry integrity before installation; the public latest identity must remain unchanged across the run.
Each installation checks its own expected version, help, models, executable resolution through PATH, and installed runtime file hashes against its own pre-install archive inventory.
The report records script policy, cold-cache state, package identities, command outcomes, successful execution of both exact reviewed dependency hooks, and confirmed temporary-prefix cleanup.
Hook evidence is extracted from isolated npm debug logs as reviewed package/version/hook/exit tuples; raw logs, hook output, auth files and configuration are not copied into the report.
The global validator shares a 600-second deadline across downloads and commands, terminates active command process groups on timeout or cancellation, and cleans prefixes before returning.
Its Python supervisor allows 630 seconds, then sends SIGTERM to permit validator cleanup before a bounded SIGKILL fallback; a supervisory timeout remains a failure.
Missing entries, disabled scripts, hash mismatches or failed cleanup fail the installed scenario and aggregate.
The complete deterministic/live behavior suite still exercises the candidate rather than substituting public release behavior for candidate coverage.

This is validation infrastructure, not a record that the changed workflow has passed remotely.
Public installation guidance uses the ordinary npm command, with scripts-disabled installation retained as an option. Local command validation is not remote Daytona qualification: do not enable automatic runs until the qualification below is complete.
Preparation, publishing and self-updates continue to disable lifecycle scripts. Native and Daytona npm candidate consumers explicitly opt into reviewed hooks; pnpm and Bun consumers remain scripts-disabled. This does not authorize enabling scripts throughout CI.

## Configure access and a snapshot

A repository administrator and a Daytona account owner must perform this setup.
Expand Down
12 changes: 10 additions & 2 deletions docs/release/artifact-acceptance.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@ Consumers live outside the repository in paths containing spaces, with separate
npm and Bun start with an empty registry cache shared only among consumers in that acceptance run, and Bun registry concurrency is bounded.
pnpm consumers use isolated stores, homes, configuration and global executable directories; no user store or credentials are inherited.
Installed file identities are rechecked in every consumer even when downloaded dependencies are cached.
Lifecycle scripts are disabled and the Diffex registry scope points to an unavailable loopback endpoint, so a missing local override fails instead of fetching an older Diffex package.
Native candidate npm consumers explicitly enable dependency lifecycle scripts; pnpm and Bun consumers keep scripts disabled.
The shared `installConsumer` helper still disables scripts by default; acceptance opts npm into the reviewed script-enabled path.
The Diffex registry scope points to an unavailable loopback endpoint, so a missing local override fails instead of fetching an older Diffex package.
External npm dependencies still resolve from the public registry, except for the reviewed Mistral provider bundle.
Only that provider implementation absorbs its required pure-JavaScript SDK code; other implementation paths, declarations, lazy loading, extensions, native bindings and WASM remain unchanged.
Bundling checks exact approved npm versions/licenses, records input hashes and registry integrity, preserves complete license/NOTICE text, and rehashes the final staged runtime inventory.
Expand Down Expand Up @@ -38,6 +40,7 @@ This validates the npm installation with controlled system helpers; it does not
The acceptance command also runs `scripts/npm-lifecycle.mjs` against the exact reviewed CLI tarball.
It installs globally into an isolated prefix, checks PATH, version, help, and models, then invokes the installed self-updater with real npm subprocesses.
Only discovery and the Diffex-scoped registry are controlled fixtures; dependency downloads use npmjs.org.
This updater/recovery harness intentionally retains scripts-disabled installation and self-update behavior; the candidate consumer and public fresh-install checks separately cover script-enabled npm installation.
An isolated version-metadata predecessor exercises upgrade to the reviewed candidate without publishing a fake release.
Fresh caches require actual target downloads.
Malformed and foreign metadata, redirects, outages, no release, and checksum mismatches are rejected without changing the installed prefix, including force.
Expand Down Expand Up @@ -75,7 +78,7 @@ node scripts/artifact-acceptance.mjs /path/to/artifacts /path/to/private-evidenc
```

The command accepts both the local manifest/tarballs layout and the frozen release artifact layout.
`acceptance.json` records the source commit, OS, architecture, Node and package-manager versions, each package's imports and operation result, exact tarball checksums, process outcomes, and cleanup.
`acceptance.json` records the source commit, OS, architecture, Node and package-manager versions, each package's imports and operation result, lifecycle script modes for package and process consumers, exact tarball checksums, process outcomes, and cleanup.
A failed run records failure and the completed subset; it cannot produce a passing overall status.
Terminal transcripts and smoke events are retained beside it in private evidence storage.
Every test-owned consumer, session, cache, configuration directory, and server process is removed, including on failure.
Expand All @@ -91,6 +94,11 @@ Regression coverage alters archive bytes and confirms verification fails before
The release workflow retains the source snapshot and manifests privately, while npm publication supplies registry integrity through the existing publisher.
Trusted publishing authenticates the private source workflow, but npm does not generate provenance for private repositories.
Public acceptance must compare the actual downloaded npm bytes with these retained identities after separately authorized publication.
Its initial predecessor install and fresh release reinstall require npm's effective `ignore-scripts` configuration to be `false` before each install and omit the `--ignore-scripts` flag.
These public lifecycle installs use exact versions so the predecessor and release identities remain deterministic; the separate Daytona `npm-global-installation` scenario checks the unversioned `npm i -g @diffexai/diffex` command.
Public pnpm and Bun installs continue to disable scripts, and the installed self-updater intentionally remains scripts-disabled for every manager.
Public evidence records each initial/fresh installation's lifecycle mode and npm configuration result separately from the updater mode.
Changed Daytona scenarios remain a pending remote execution and evidence-review gate; local tests do not establish that result.

The baseline CLI-only acceptance passed on Darwin arm64 but emitted no per-package operation or hosted-server result.
The previous expanded acceptance covered eleven packages and the hosted server lifecycle before the owner reduced the public inventory on 2026-09-13.
Expand Down
61 changes: 61 additions & 0 deletions docs/release/npm-install-scripts-review.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Normal npm installation review

## Status

The public npm installation command is `npm i -g @diffexai/diffex`.
Static dependency review and isolated script-enabled installation checks support that command; scripts-disabled installation remains supported.
The modified Daytona scenario still requires explicit remote qualification and evidence review before automatic behavior runs are enabled or the remote path is treated as qualified. Its workflow is manual or qualified post-merge, not an automatic pull-request check; environment secrets remain restricted to `main`.
This record does not claim that GitHub Actions or Daytona has executed the changed tests.

The normal-install follow-up changes instructions and acceptance coverage, not dependency versions, lockfiles, self-update behavior or publishing safeguards.
The generated npm README now shows the ordinary command; no install or postinstall hook is added to Diffex itself.

## Reviewed scope

The review compared the public `@diffexai/diffex@0.4.0` archive, the current CLI shrinkwrap, and the generated installer lock.
The public archive has 228 dependency entries; the current candidate graph has 140 because the reviewed Mistral provider implementation is bundled.
These are different artifacts even though they currently share version `0.4.0`.
The public archive must not be checked against the candidate's runtime inventory or source commit.

Read-only metadata and tarball inspection covered 225 unique dependency identities across these graphs, including optional platform packages and required peer dependencies.
Each downloaded tarball matched its registry SHA-512 integrity.
No implicit `binding.gyp` installation or additional preinstall/install/postinstall hook was found.
The package staging code removes the CLI's and bundled workspaces' own scripts.
Build/prepare/publishing hooks declared in registry dependencies are not consumer-install hooks.

Public CLI archive:

```text
https://registry.npmjs.org/@diffexai/diffex/-/diffex-0.4.0.tgz
sha512-l76f89GQRAv4M2DOVc6CuT6RakqvK3d1bDwdtol9ciPJ8Lbi9p97pMDy7uuA3VDUUdR+8OlWk0pfEIkhapEZUw==
```

Only two install-time hooks were found:

| Package | Hook | Reviewed behavior |
| --- | --- | --- |
| `@google/genai@2.31.0` | `preinstall`: `echo 'preinstall: no-op'` | Prints a message; no dependency build or download. |
| `protobufjs@7.6.5` | `postinstall`: `node scripts/postinstall` | Reads package metadata and may print a version-scheme warning; no network, subprocess, compilation or filesystem writes. |

The published protobufjs `scripts/postinstall.js` SHA-256 is `5af8463b97ee8e309b4a2111f9479bacdf0c180de0ca0155527679b1fc6d9e6c`.
The existing exact-version allowlists in `scripts/generate-coding-agent-shrinkwrap.mjs` and `scripts/generate-coding-agent-install-lock.mjs` match these two packages.
An allowlist comment alone is not a replacement for reviewing the actual published hook bytes.
Re-review lifecycle behavior when dependency versions, peers, optional dependencies or the published package change.
This inspection does not prove that arbitrary future package versions or unrelated dependencies are safe.

## Validation contract

- The reusable artifact installer continues to disable lifecycle scripts by default.
- Native candidate acceptance explicitly enables scripts for npm package/process consumers and records that mode. pnpm and Bun acceptance remain scripts-disabled.
- Public-release npm initial and fresh installations require `ignore-scripts=false` and omit the flag from the install command; self-updates intentionally remain scripts-disabled.
- The explicit npm-only consumer option enables scripts and rejects opt-in for other package managers.
- Local audited fixtures prove that enabled npm hooks actually run, inherited credentials/configuration do not enter them, and scripts-disabled consumers remain disabled.
- Daytona's candidate npm consumers enable scripts and retain the full installed CLI behavior tests; Bun remains on its existing scripts-disabled path.
- The separate `npm-global-installation` scenario tests the retained candidate and independently verified public latest package in separate cold prefixes.
- Candidate registry metadata and seeded consumer installation locks preserve the reviewed shrinkwrap. npm 10 file/tarball transport can otherwise ignore a nested shrinkwrap and re-resolve transitive ranges; a real check caught unreviewed `protobufjs@7.6.6` before this correction.
- Both global installs require positive debug-log evidence that the exact two reviewed hook commands started and completed successfully. Configuration alone is not execution evidence. Raw npm logs are not retained.
- The public path executes exactly `npm i -g @diffexai/diffex`, with no `--ignore-scripts` flag and verified default configuration of `false`.
- Both global paths require executable resolution through PATH, expected version/help/models, exact installed runtime inventories and positive cleanup.
- Missing evidence, public latest movement, hash mismatches, scripts-disabled configuration or cleanup failure remain failures.

See [the Daytona operator guide](../daytona-behaviors.md) for workflow authorization, execution, evidence and cleanup requirements.
Loading
Loading