From 3456965577a19ee3b44816b593304a8c8a0970cc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=A0=D0=BE=D0=B1=D0=BE=D1=82?= Date: Mon, 21 Sep 2026 11:35:16 +0200 Subject: [PATCH 1/3] feat(harness): define workbench contracts from selective ECC review --- .claude-plugin/marketplace.json | 2 +- CHANGELOG.md | 14 ++ README.md | 11 +- SKILL-CARD.md | 2 +- .../research/2026-09-21-ecc-harness.md | 120 ++++++++++++ .../specs/2026-09-21-ecc-harness-brief.md | 43 +++++ package.json | 2 +- .../agent-stack/.claude-plugin/plugin.json | 2 +- .../agent-stack/skills/agent-harness/SKILL.md | 2 + .../skills/agent-harness/references/layers.md | 11 +- .../references/workbench-contracts.md | 174 ++++++++++++++++++ 11 files changed, 376 insertions(+), 7 deletions(-) create mode 100644 docs/evidence/research/2026-09-21-ecc-harness.md create mode 100644 docs/evidence/specs/2026-09-21-ecc-harness-brief.md create mode 100644 plugins/agent-stack/skills/agent-harness/references/workbench-contracts.md diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index b6206fe..299e556 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -12,7 +12,7 @@ "displayName": "Agent Stack", "source": "./plugins/agent-stack", "description": "Four skills: agent-orchestrator — tool-calling loops, pipelines with checkpoints, provider routing with fallback, memory architecture, plus the wallet side of reselling LLM access; agent-evals — run/trace/thread evals, LLM judges, and fixtures grown from production; agent-interop — MCP servers and clients, A2A agent cards, the MCP Registry, and gateways; agent-harness — system prompts, tool shaping, workflow-vs-agent, and auditing an agent system.", - "version": "0.24.3", + "version": "0.25.0", "author": { "name": "ssheleg", "url": "https://x.com/sshlg93" diff --git a/CHANGELOG.md b/CHANGELOG.md index 8ce73de..3aa41f0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,17 @@ +## 0.25.0 — workbench harness contracts, with a bounded ECC transfer + +- Distinguish a kernel harness from the operating layer around an existing agent. + The family is the latter; skills are components, not an OS security boundary. +- Add an on-demand `agent-harness` reference for install ownership and loaded + state, bounded hooks and failure policy, evidence-bearing handoff, artifact-bound + eval receipts, replay limits and scoped learning. Existing family owners remain + authoritative; no duplicate runtime, schemas or receipt store is introduced. +- Attribute the selective ECC review to commit + `2b6e839771e53096d8451a213d40dc64ec8acac0`; retain the adoption matrix, + source digests, acceptance cases and explicit limits in the research ledger. +- No new hook, network dependency, telemetry or service is installed. Structural + checks are not represented as measured improvements in agent outcomes. + ## 0.24.3 — the budget stops being estimated, and two skills stop naming a file they do not ship Hygiene from the 2026-09-13 family audit (HK-11). diff --git a/README.md b/README.md index 8907782..a452265 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,10 @@ add` puts this pack — at rank 500. Production patterns for building AI agent orchestrators — and for billing the LLM access they burn. -Part of the [ssheleg skill family](https://github.com/ssheleg/sshlg-skills). +Part of the [ssheleg agent harness](https://github.com/ssheleg/sshlg-skills): +a workbench layer of routing, delivery contracts and evidence around existing +agent hosts. This pack provides its agent-engineering skills. It does not replace +the host runtime or provide an OS security boundary. --- @@ -88,7 +91,7 @@ ACP, AGNTCY, AP2, Agent Skills — so an agent stops guessing. model, and the one where most agent bugs actually live: *the biggest performance improvements often come from clearly explaining tool usage in the system prompt*, and *even small refinements to tool descriptions can yield dramatic improvements*. Before adding a -retry or a sub-agent, it asks four questions about the text. Seven references — +retry or a sub-agent, it asks four questions about the text. Eight references — `system-prompt.md` (right altitude, enumerated vocabulary, and the three things reasoning models changed — starting with **do not add chain-of-thought**), `tools.md` (the agent–computer interface, with a worked before/after and poka-yoke), `techniques.md` @@ -98,6 +101,10 @@ environment's job), `audit.md` (seven tracks, evidence tiers, a plan instead of plus **`pi.md` and `pi-sdk.md`, the doctrine as a worked implementation**: Pi read end to end, each mechanism matched to the rule it instantiates, its divergences named, and the eight extension seams where a permission gate or a context rewrite can actually live. +`workbench-contracts.md` adds the operating layer around a host: managed install +ownership, bounded hooks, evidence-bearing handoff, artifact-bound evaluation and +observation limits, adapted selectively from a pinned ECC review. These are design +contracts; the pack installs no new hook, service or telemetry collector. It runs in both directions: **building a harness and auditing somebody else's are one checklist read forwards and backwards.** `scripts/audit_agent.py` is the mechanical half — diff --git a/SKILL-CARD.md b/SKILL-CARD.md index 38c3c09..c56f5f5 100644 --- a/SKILL-CARD.md +++ b/SKILL-CARD.md @@ -5,7 +5,7 @@ | Field | Value | |---|---| | Pack | `agent-stack` | -| Version | `0.24.3` | +| Version | `0.25.0` | | Skills | `agent-orchestrator`, `agent-evals`, `agent-interop`, `agent-harness` | | License | MIT | | Source | https://github.com/ssheleg/agent-stack | diff --git a/docs/evidence/research/2026-09-21-ecc-harness.md b/docs/evidence/research/2026-09-21-ecc-harness.md new file mode 100644 index 0000000..739d1f7 --- /dev/null +++ b/docs/evidence/research/2026-09-21-ecc-harness.md @@ -0,0 +1,120 @@ +ssheleg skills — agent-harness · make-skill · task-pipeline · agent-sync + +# ECC → ssheleg harness: selective adoption ledger + +Research date: 2026-09-21. Owner: agent-stack / agent-harness. Task packet: +[bounded brief](../specs/2026-09-21-ecc-harness-brief.md). + +## Source boundary + +ECC was cloned read-only from `https://github.com/affaan-m/ECC` at +[`2b6e839771e53096d8451a213d40dc64ec8acac0`](https://github.com/affaan-m/ECC/tree/2b6e839771e53096d8451a213d40dc64ec8acac0). +Its package declares 2.2.2. The source ledger below names exact files and SHA-256 +values. Source observations are **static inspection**, not execution results. +No ECC installer, hook, test, package installation or background observer ran. +The wider repository was inventoried by paths; it was not audited in full. + +ECC root LICENSE: MIT, Copyright (c) 2026 Affaan Mustafa. The selected files show +no separate license headers overriding it. This change adapts methods in newly +written prose; no implementation, schema, command text or substantial upstream +text is copied. Do not infer the same review for uninspected assets/subtrees. + +Local baseline: agent-stack +[`c681ef3af6157d13becf53250373b418cca6c919`](https://github.com/ssheleg/agent-stack/tree/c681ef3af6157d13becf53250373b418cca6c919). +Family comparison: sshlg-skills +[`564d1f42a895abb8f0f53417695414b74dbf91a5`](https://github.com/ssheleg/sshlg-skills/tree/564d1f42a895abb8f0f53417695414b74dbf91a5), +README and `lib/lifecycle.js` read. This is a selected-contract comparison, not a +whole-project audit or a benchmark between the two projects. + +## Narrative recommendation + +**The ssheleg harness is an operating layer for agent work.** It routes work to +specialist skills, carries tasks through explicit contracts, preserves evidence +and resume context, and exposes what is installed and actually loaded. The skill +family is its expertise layer. Project Observatory can be its optional observation +component once its own readiness and privacy checks pass. + +The host owns model execution and tool permissions. The harness does not promise +sandboxing, universal hook support or complete leak prevention. Observatory's +credential requirements and private state are separate from the skills' runtime +requirements. Avoid calling this a replacement for Claude Code, Codex or other +agent hosts. Avoid publishing vendor leak counts without reviewed evidence. + +## Borrow / adapt / reject + +| Method | Verdict and reason | Family owner / destination | Evidence | +|---|---|---|---| +| Explicit managed install state, preservation of unowned files | **Adapt.** A skipped write must not grant uninstall ownership. Keep user changes visible. | Installer owner + make-skill; contract in new reference | ECC ownership guard; lifecycle inspection | +| Installed versus loaded status | **Retain existing family contract.** ECC doctor diagnoses managed files; family already distinguishes disk and reload receipts. No second store. | sshlg-skills lifecycle | ECC doctor; local `lib/lifecycle.js` | +| Hook profiles and per-hook disable | **Adapt.** Make effective capabilities, effects and failure behavior inspectable. | Host adapter; contract in agent-harness | ECC hook-flags | +| Input caps and subprocess timeout | **Adapt with stronger bounds.** Input bytes alone do not bound stream duration; subprocess timeout does not bound in-process awaits. | Host adapter | ECC hook-input; run-with-flags lines 249–287 | +| Fail-open generic hook exception | **Reject as a universal policy.** A required gate must not silently pass when its check failed. This is a design assessment, not an exploit claim. | Host adapter | run-with-flags lines 267–270 and 303–306 | +| Evidence-bearing save/resume | **Adapt content, retain local format.** Confirmed work, failures and next task are useful; a new global session store would duplicate task-pipeline. | task-pipeline existing handoff | save-session and resume-session | +| Hash-bound eval receipts and fixture-only replay | **Adapt contract.** Bind candidate, taskset and checker; integrity is distinct from correctness. | agent-evals / existing result envelopes | eval-harness receipt, capsule and architecture | +| Arbitrary candidate execution | **Do not import.** ECC itself disables it without verified OS containment. Keep that limitation explicit. | Host/environment security boundary | eval-harness gate opening contract | +| Instinct learning/background observer | **Defer.** Scope, consent, retention, cost and proven improvement need evaluation; do not auto-promote session content globally. | agent-orchestrator memory owner | continuous-learning-v2 SKILL | +| Entire ECC installer, hook bundle and host configs | **Reject wholesale import.** Adds parallel orchestration and unreviewed dependency closure. | Existing family routers remain entry points | Inventory plus bounded task scope | + +ECC observations above are grounded in the source files below. Family policy +choices (adapt/reject) are author judgment and await parent review; they are not +measured performance findings. No claim of superiority or outcome improvement. + +## Implemented delta and dependency closure + +- [Workbench contracts](../../../plugins/agent-stack/skills/agent-harness/references/workbench-contracts.md): + new on-demand doctrine and nine review cases. +- [Harness entry point](../../../plugins/agent-stack/skills/agent-harness/SKILL.md): + reference routing and kernel/workbench checklist distinction. +- [Layer reference](../../../plugins/agent-stack/skills/agent-harness/references/layers.md): + explicit broader workbench meaning, preserving kernel responsibilities. +- README, skill card and synchronized 0.25.0 manifests/changelog. + +Runtime additions: none. No new packages, interpreters, services, credentials, +telemetry, foreign scripts, schema stores or automatic downloads. Existing +agent-stack audit script still needs Python as before. Research files are Git +artifacts; only the reference ships inside the skill package. + +## Verification limits and next work + +Repository tests establish structural/reference/installer regressions. Strict +plugin validation checks manifest conformance. Package inspection confirms the +new reference is included and research/private machine state are not. These do +not measure agent behavior. The nine adapter review cases are design acceptance +criteria; no new adapter is implemented by this prose change. + +Before a future runtime transfer: implement in the existing owning module, +freeze representative baseline tasks, exercise no-op/conflict/unsupported-host +and tampered-evidence paths, and record real outcomes under agent-evals. Parent +review and release evidence are recorded in the adjacent handoff. + +## Source digest ledger + +`sha256(file bytes)` at the pinned ECC commit; line references above resolve there. +Sources marked “selected sections” were not semantically audited end to end. + +| Source | SHA-256 | Read scope | +|---|---|---| +| [LICENSE](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/LICENSE) | `326146379f01bb137c0a5d3c54770c1aa31076705c8b88a7f6b26a460f6221b2` | full | +| [package.json](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/package.json) | `650eaa8de580a3c8eadbc658aa5ffe3f8d50ba2be075dd327a62e1d70e1d7a69` | selected sections | +| [scripts/lib/install/ownership-guard.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/install/ownership-guard.js) | `000d8de192edceb32169b8725a797f908256f5a7e124c1803c29328eb07a36cc` | selected sections | +| [scripts/lib/install-lifecycle.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/install-lifecycle.js) | `4814affb0f3ffd3648e62a4987ce21d3a5318f6c8d4f5e2069b01e5ecbe5dfe5` | selected sections | +| [scripts/doctor.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/doctor.js) | `4ec5a1c7fe9124ebeea5d3d939270a16b0974b789558a8a9cc313cf73ec70b6e` | selected sections | +| [scripts/lib/hook-flags.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/hook-flags.js) | `1f5fbf2d2ebd0ab07a3e54406db18c2932ae7bf965513ec12c521da1be54425d` | full | +| [scripts/hooks/hook-input.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/hooks/hook-input.js) | `edae519f794befa4cf1dcbfb3c985d0db6f589342b7036ffb7ace61a9c3868b0` | full | +| [scripts/hooks/run-with-flags.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/hooks/run-with-flags.js) | `a5b3c3f4ed27f1d9d9c607d6819aab9985ad446239ea7d2825d4b040d2d8d87e` | selected sections | +| [commands/save-session.md](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/commands/save-session.md) | `a9488a8819d6c65fe4777f07ea72de6b0c4ee94f90c6bb33b3b8841916222db4` | selected sections | +| [commands/resume-session.md](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/commands/resume-session.md) | `b09bcb22feb10adbba50accd5dd97d09b60897aa94d05c6c12b6e579cb69cbac` | full | +| [scripts/lib/eval-harness/receipt.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/eval-harness/receipt.js) | `eacacccf8a6d06f38e83c8cee1bb3a6c608e9125115a989fd7f8a7f7e371d5e7` | selected sections | +| [scripts/lib/eval-harness/capsule.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/eval-harness/capsule.js) | `fd0bbbde8f383226fbcb093210e0f16594acd9f6ea0ae209e47a37cc39834b0c` | selected sections | +| [scripts/lib/eval-harness/gate.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/eval-harness/gate.js) | `76d6613aa7aacb9863c61859d20e554826c6f08e71c10221f0b6fea4d04e3c4f` | selected sections | +| [docs/architecture/eval-harness-frameworks.md](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/docs/architecture/eval-harness-frameworks.md) | `eec9ad255f71f68feb4d5e2ee9b2ececef6cd4dc4d1b246d015f3cc047406a61` | selected sections | +| [skills/continuous-learning-v2/SKILL.md](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/skills/continuous-learning-v2/SKILL.md) | `ddad12451a0d582c6ef1b197b7be193bef613cbc74eba4117fcb74fc7a4ecd00` | selected sections | + +--- + +**Made with [ssheleg skills](https://github.com/ssheleg/sshlg-skills)** + +- [`agent-harness`](https://github.com/ssheleg/agent-stack) — workbench contracts and ECC comparison +- [`make-skill`](https://github.com/ssheleg/make-skill) — attribution and package checks +- [`task-pipeline`](https://github.com/ssheleg/task-pipeline) — bounded brief and handoff +- [`agent-sync`](https://github.com/ssheleg/agent-sync) — local claims for shared metadata diff --git a/docs/evidence/specs/2026-09-21-ecc-harness-brief.md b/docs/evidence/specs/2026-09-21-ecc-harness-brief.md new file mode 100644 index 0000000..900cc3f --- /dev/null +++ b/docs/evidence/specs/2026-09-21-ecc-harness-brief.md @@ -0,0 +1,43 @@ +# ECC workbench-contract transfer — bounded task + +Objective: study ECC at a frozen revision and improve the agent-stack harness +reference without importing a foreign installer, hook or runtime. Authorized by +the operator's 2026-09-21 autonomous harness request; this is the agent-stack +packet in the parent multi-repository program. Existing model settings inherited. + +## Scope, dependencies and evidence + +- REQ-ECC-1: pin ECC and record a borrow/adapt/reject matrix, source paths and + digests. Check: research ledger and reproducible SHA-256 comparison. +- REQ-ECC-2: define a workbench harness and its limits; use existing family + contracts for routing, delivery, leases and evals. Check: reference review. +- REQ-ECC-3: document bounded hooks, install ownership, portable handoff and + artifact-bound evaluation. Check: linked reference plus repository validators. +- REQ-ECC-4: keep dependency-free doctrine and no new always-on behavior. + Check: diff and packed artifact inventory. +- REQ-ECC-5: preserve attribution, release metadata and a cold-reader handoff. + Check: version consistency, strict plugin validation and handoff entry. + +Inputs read: agent-harness SKILL and layers reference; make-skill SKILL and +enterprise reference; task-pipeline SKILL; agent-sync SKILL and local config; +agent-stack README, package and validator; sshlg-skills README and lifecycle +module; the ECC ledger names the external sources read. Contradictions: the +existing layers table uses harness narrowly for a kernel; the requested umbrella +uses it for a workbench. Name both meanings explicitly rather than implying the +family owns model execution or sandboxing. + +## Plan and boundaries + +1. Inspect ECC read-only; do not run its installers, hooks, dependencies or tests. +2. Write one on-demand reference and integrate its route into agent-harness. +3. Record matrix, source digests, review cases and limits; sync README/changelog. +4. Run existing repository tests, strict manifest validation and package check. +5. Commit this packet, send the exact diff/version to the parent for independent + review, integration, release and umbrella pin update. No push/release until + the parent coordinates it. No edits in sibling repositories. + +This is a documentation/reference change: no product UI or command behavior +changes. No new UX chain, deployment or runtime permissions. Shared metadata is +edited under local advisory agent-sync claims; the host has no enforcement hook. + +Resume: read ../research/2026-09-21-ecc-harness.md and the handoff beside it. diff --git a/package.json b/package.json index ddcdefa..369b078 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@ssheleg/agent-stack", - "version": "0.24.3", + "version": "0.25.0", "scripts": { "test": "python3 test/validate.py && python3 test/plant_guard_test.py && node test/installer_test.js && npm run test:audit", "test:audit": "for t in test/audit_regressions/*.py; do python3 \"$t\" || exit 1; done" diff --git a/plugins/agent-stack/.claude-plugin/plugin.json b/plugins/agent-stack/.claude-plugin/plugin.json index 1268618..d22e81a 100644 --- a/plugins/agent-stack/.claude-plugin/plugin.json +++ b/plugins/agent-stack/.claude-plugin/plugin.json @@ -3,7 +3,7 @@ "name": "agent-stack", "displayName": "Agent Stack", "description": "Four skills: agent-orchestrator — tool-calling loops, pipelines with checkpoints, provider routing with fallback, memory architecture, plus the wallet side of reselling LLM access; agent-evals — run/trace/thread evals, LLM judges, and fixtures grown from production; agent-interop — MCP servers and clients, A2A agent cards, the MCP Registry, and gateways; agent-harness — system prompts, tool shaping, workflow-vs-agent, and auditing an agent system.", - "version": "0.24.3", + "version": "0.25.0", "author": { "name": "ssheleg", "url": "https://x.com/sshlg93" diff --git a/plugins/agent-stack/skills/agent-harness/SKILL.md b/plugins/agent-stack/skills/agent-harness/SKILL.md index f02e472..44396f2 100644 --- a/plugins/agent-stack/skills/agent-harness/SKILL.md +++ b/plugins/agent-stack/skills/agent-harness/SKILL.md @@ -129,6 +129,7 @@ and this pack's repository validator fails the build on a reference that does no | [`references/system-prompt.md`](references/system-prompt.md) | you are **writing or fixing the prompt** — altitude, structure, vocabulary, dynamic context, and what changes for reasoning models | | [`references/tools.md`](references/tools.md) | the model **picks the wrong tool, or none** — the agent–computer interface: how many, named how, described how, returning what | | [`references/techniques.md`](references/techniques.md) | you are choosing between **ReAct, reflection, voting, planning** and the rest — every entry carries a verdict for production, not a benchmark score | +| [`references/workbench-contracts.md`](references/workbench-contracts.md) | assembling a **harness around an existing agent** — install ownership, bounded hooks, handoff, artifact-bound evidence and observation limits; selective ECC methods | | [`references/layers.md`](references/layers.md) | deciding **what your harness owns** — kernel, workbench and product layers, and why permission boundaries are usually somebody else's job | | [`references/audit.md`](references/audit.md) | reviewing **an agent system you did not build** — seven tracks, evidence tiers, and a prioritized plan | | [`references/pi.md`](references/pi.md) | you want the doctrine above as a **worked implementation** — Pi's sessions, compaction, config, skills, trust and containerization, each matched to the rule it implements, and the places it deliberately disagrees | @@ -202,6 +203,7 @@ prompt. ## Checklist — a harness worth shipping +- [ ] Kernel or workbench harness named explicitly; installed, loaded and enforced capabilities distinguished - [ ] Workflow-versus-agent decided deliberately, and the simpler option was actually tried - [ ] Static-versus-dynamic decided too — static preferred for predictability; a run that must be auditable keeps a complete execution record (not merely a static shape) - [ ] System prompt at the **right altitude** — heuristics, not hardcoded branches, not vague hope diff --git a/plugins/agent-stack/skills/agent-harness/references/layers.md b/plugins/agent-stack/skills/agent-harness/references/layers.md index a5f62e2..2869b61 100644 --- a/plugins/agent-stack/skills/agent-harness/references/layers.md +++ b/plugins/agent-stack/skills/agent-harness/references/layers.md @@ -35,7 +35,16 @@ will not converge no matter how long it runs. The distinction is not quality. A kernel is *supposed* to be smaller than a workbench; that is what makes it legible. -## What a harness owns +## The broader workbench meaning + +“Harness” also names an operating layer around an existing agent: skills, routing, +work contracts, hooks, installation and evidence. This is the family’s umbrella +meaning. It does not imply ownership of the model loop or OS permissions. The +three-layer table above uses the narrower kernel meaning. State the meaning +before comparing products; [workbench-contracts.md](workbench-contracts.md) gives +the operational contract and selective ECC implementation lessons. + +## What a kernel harness owns If you are building at the kernel layer, these are yours and nobody else's: diff --git a/plugins/agent-stack/skills/agent-harness/references/workbench-contracts.md b/plugins/agent-stack/skills/agent-harness/references/workbench-contracts.md new file mode 100644 index 0000000..3505842 --- /dev/null +++ b/plugins/agent-stack/skills/agent-harness/references/workbench-contracts.md @@ -0,0 +1,174 @@ +# Workbench contracts — a harness around an existing agent + +**Load this when:** assembling skills, hooks, installers and observation tools into +an operational harness, or evaluating what to adopt from another collection. + +**Spec pinned:** ECC 2b6e839771e53096d8451a213d40dc64ec8acac0 (2.2.2), selective source review · read 2026-09-21 + +## Contents + +- What the word promises +- Installation is an owned change +- A hook needs an execution contract +- Resume from evidence +- Bind evaluations to artifacts +- Observation and learning permissions +- Adoption review cases +- Source and attribution + +## What the word promises + +A **workbench harness** is the operating layer around an agent host: it selects +instructions, defines work and evidence, resumes interrupted tasks, manages its +installed components and makes their state inspectable. Skills are one component. +A **kernel harness** owns the model/tool loop. Say which one you mean; a workbench +can support an existing agent without replacing that agent's runtime. + +For the ssheleg family, routing and specialist skills guide the work; +`task-pipeline` owns scope, evidence, dependencies and resume artifacts; +`agent-sync` owns coordination where configured; `make-skill` owns packaging and +installation review; `agent-evals` owns behavioral evidence. An observation tool +may report repository, credential or project health alongside these contracts. +It does not become a sandbox, an authorization service or proof that every secret +has been detected. Its optional service dependencies must remain separate from +the dependency-free skill layer. + +These are design contracts, not a claim that every family host already implements +every mechanism below. Inventory each host's capabilities before claiming parity. + +## 1. Installation is an owned change + +A package name on disk is not an installation receipt. Record the source revision +or package integrity, component version, host/channel, intended root, files or +configuration keys owned, installed digests, and previous state needed to undo it. +Keep local absolute paths in local receipts; public evidence uses sanitized paths. + +- Plan the exact changes first. Existing unowned files remain user-owned even when + their names match a package file. A skipped write must not create an ownership + claim that lets uninstall remove it later. +- Merge only owned configuration keys; preserve unrelated configuration. Treat + changed owned files as conflicts to review, not permission to overwrite them. +- Validate containment and symlinks at mutation time as well as planning time. + A digest describes bytes; it does not establish safe filesystem ownership. +- Diagnose missing, changed, conflicting and unknown separately. Update, repair + and uninstall must preserve user edits or expose a concrete reviewable conflict. +- Distinguish **published**, **downloaded**, **installed**, and **loaded**. A running + agent may still use an old copy after a successful update. Require a host reload + receipt when available; otherwise report loaded version as unknown. + +Reuse the family's existing installer and reload receipts. Do not add a second +receipt store in a skill. Installer implementation belongs to its owning package, +with lifecycle tests covering install → edit → update → uninstall. + +## 2. A hook needs an execution contract + +For each hook, record event and matcher, supported host versions, input/output +schema, effects, input/output byte limits, wall-clock deadline, cancellation and +child-process cleanup, idempotency key where needed, profile/default, disable +control and failure policy. Expose what is effectively enabled, including where +that setting came from. A profile name alone proves none of these properties. + +Use a small default set. Expensive, networked or learning hooks are explicit +capabilities, with a documented no-hook/manual path on unsupported hosts. A +security gate that is required but unavailable blocks its protected action; +a convenience hook may fail open only while reporting that it did not run. +Neither becomes PASS because its process returned no usable result. + +| Condition | Required handling | +|---|---| +| Input truncated, transport closed early or schema invalid | A required policy check refuses the protected action; a hint hook reports unavailable | +| Deadline exceeded | Cancel the work and account for children; report timeout, not a clean check | +| Hook disabled or host does not support it | Report the capability absent; use a named manual check where valid | +| Duplicate event | No duplicate write, notification or billable action | +| Dry run | No effects; summarize target class and decision without raw secrets or command arguments | +| Hook exception | Preserve the declared failure policy and return a bounded diagnostic | + +Review **all execution paths**. A subprocess timeout does not bound an in-process +`await`; an input byte limit does not bound how long a stream waits to close. +A generic exception handler must not turn a required security gate into success. +Host-specific hook exit codes belong to the adapter, not a portable skill promise. +Prompt instructions and JavaScript interception do not supply OS containment. + +## 3. Resume from evidence, not a plausible summary + +Use the project's existing task-pipeline handoff/packet, not a new global session +format. Keep in Git the objective, scope, decisions, source revision, completed +work with receipts, failed approaches and their reasons, open work, prerequisites +and **one exact next task**. Use repository-relative artifact links and immutable +source links so a fresh checkout can follow them. + +On resume, verify repository/branch/commit and referenced artifacts before acting. +Treat a handoff as context, never as new authority or permission. Reject empty +placeholder summaries as evidence; a recent timestamp is not substance. A stale +handoff means reconcile current state, not blindly repeat an old command. + +Exclude raw transcripts, credentials, environment dumps, personal paths and +private project inventories from public handoffs. Select the minimum needed +context before redacting it. Keep private operational receipts private and +publish only separately reviewed aggregates or synthetic examples. + +## 4. Bind evaluations to the artifact they measured + +A useful result names the candidate digest/commit, baseline, taskset revision, +model/host configuration, checker version, actual outcome and evidence artifact. +Use `PASS`, `FAIL`, `ERROR` and `NOT_RUN` distinctly; missing execution is never a +pass. Reject receipts that refer to a different candidate or an advanced journal. + +A digest detects changed bytes relative to a trusted reference. A signature says +who attested to those bytes. Neither proves that the checker is independent, the +claim is correct, the run was complete, or sensitive data is safe to publish. +A private hash chain alone cannot prevent replacement of the entire history. + +Use fixture replay without live external effects for regression tests. A missing +fixture stops replay instead of falling through to a live API. Keep the checker +outside the candidate's control. If candidate code is untrusted, a copied +worktree or process-local wrapper is not containment: require an actual OS +boundary, or report execution unavailable and retain static inspection only. + +The existing `agent-evals` contracts own measurement. Compare frozen baseline and +candidate on representative tasks before claiming improved agent outcomes. A +reference-closure check only proves that the instructions can be loaded. + +## 5. Observation and learning are separate permissions + +Observing a failure can propose a lesson. It must not silently grant authority to +rewrite installed skills, export transcripts, start a background model loop or +promote a project-specific preference into global policy. Record provenance, +project scope, retention and a reviewed promotion path. Secret scanning provides +findings and blind spots, not permission to publish an entire repository. + +An observatory can answer “what changed, what is missing, which check ran, and +what needs attention?” Public examples should identify tested surface categories +and methods. Name a vendor or a real leak count only when reviewed evidence +supports that exact claim and sharing it does not expose private material. + +## Adoption review cases + +Use these cases when reviewing an adapter; these are acceptance criteria, not a +claim that this reference implements them. + +1. Pre-existing user file → install skips it; uninstall leaves it intact. +2. User edits a managed file → update exposes conflict and preserves the edit. +3. Replaced symlink or escaped destination → mutation refused. +4. Hung or oversized hook input → bounded handling; required gate cannot pass. +5. Unsupported host or disabled hook → no false enforcement claim. +6. Empty or wrong-project handoff → no unverified resume. +7. Changed candidate or missing replay fixture → evaluation cannot pass. +8. Receipt verifies but checker was never run → evidence is NOT_RUN. +9. Private canary in an evidence payload → export rejected or safely transformed + before publishing; the raw value never appears in diagnostics. + +## Source and attribution + +Methods reviewed from **ECC**, Copyright (c) 2026 Affaan Mustafa, MIT: +[license](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/LICENSE), +[installation ownership](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/install/ownership-guard.js), +[hook execution](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/hooks/run-with-flags.js), +[hook profiles](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/hook-flags.js), +[session evidence](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/commands/save-session.md), +[eval contracts and limits](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/docs/architecture/eval-harness-frameworks.md). + +This is independently written doctrine adapting bounded methods. No ECC source, +installer, hook, schema, session format or runtime dependency is vendored. The +family retains its own artifact owners. Recheck the pinned implementation before +using these references to assess a newer ECC release. From 331f7eb6bfb19bd7e4144b5fb69a8c06c1c2ff29 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=A0=D0=BE=D0=B1=D0=BE=D1=82?= Date: Mon, 21 Sep 2026 11:35:56 +0200 Subject: [PATCH 2/3] docs: record ECC harness transfer checks and next task --- .../research/2026-09-21-ecc-handoff.md | 64 +++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 docs/evidence/research/2026-09-21-ecc-handoff.md diff --git a/docs/evidence/research/2026-09-21-ecc-handoff.md b/docs/evidence/research/2026-09-21-ecc-handoff.md new file mode 100644 index 0000000..d106df5 --- /dev/null +++ b/docs/evidence/research/2026-09-21-ecc-handoff.md @@ -0,0 +1,64 @@ +# Agent-stack ECC transfer — handoff + +Objective: improve the agent-engineering component of the ssheleg harness with +selective ECC methods, keeping existing family contracts and runtime boundaries. + +Entry point: [research and transfer matrix](2026-09-21-ecc-harness.md). +Task packet: [scope, dependencies and checks](../specs/2026-09-21-ecc-harness-brief.md). + +## Completed + +Implementation commit: +[`3456965577a19ee3b44816b593304a8c8a0970cc`](https://github.com/ssheleg/agent-stack/commit/3456965577a19ee3b44816b593304a8c8a0970cc). +Branch: `codex/ecc-harness-contracts`. Prepared version: **0.25.0**. +The commit link becomes remotely accessible only after the parent pushes it. + +- New workbench contract reference with install ownership, hook bounds and + failure semantics, durable handoff, artifact-bound eval evidence and observation + limits; linked from the skill and layers reference. +- README positions this pack within the ssheleg agent harness; all release version + surfaces synchronized, including the skill card and changelog. +- ECC pinned with 15 file digests, selected-read scope, attribution and a + borrow/adapt/reject matrix. No ECC executable code or runtime dependency added. + +## Checks actually run + +| Check | Observed result | +|---|---| +| `npm test` | EXIT=0: structural checks, plant guard 9 cases, installer 11 cases, audit regressions green | +| `python3 test/validate.py` after final reference TOC correction | EXIT=0, 15 checks, v0.25.0 | +| `claude plugin validate . --strict` | EXIT=0, marketplace passed | +| `claude plugin validate ./plugins/agent-stack --strict` | EXIT=0, plugin passed | +| make-skill `audit_skill.py plugins/agent-stack/skills/agent-harness --house --quiet` | First run reported missing reference TOC; corrected; final 0 GAP, 19 PASS | +| `npm pack --dry-run --json` | 40 files; new reference included; research, `.env`, `.agent-sync` and `.git` absent | +| SHA-256 ledger comparison | All 15 pinned source hashes matched | +| Relative-link inspection of brief/research | All Markdown relative links resolved | +| `git diff --check` | EXIT=0 | + +The repository validator reports one unavailable check: strict family front-matter +reader is not found above this standalone checkout. The make-skill house auditor +ran separately. No behavioral model evaluation was run; structural checks do not +prove improved outcomes. The nine review cases are contract criteria, not executed +new-adapter tests. No deployment or npm publication has occurred in this packet. + +## Decisions and open work + +- Reuse task-pipeline packets, agent-sync coordination and family lifecycle + receipts; do not create another session store or global observer. +- Distinguish kernel and workbench harness meanings. Host capabilities and + missing enforcement stay visible. +- Observatory remains an optional integration proposal here. Its open-source + readiness, actual features and publication belong to the parent program. +- Parent owns independent semantic/security review, push/integration policy, + version release, installed-skill refresh, website edits and umbrella pin. + +**Exact next task:** independently review the new workbench reference and transfer +matrix against the pinned sources, then push/integrate the reviewed branch under +repository policy. Release 0.25.0 only after that review; update the umbrella pin +and verify the installed/loaded version separately. The prepared change does not +claim those follow-up operations have happened. + +Local-only residue: a read-only temporary ECC clone and package inventory JSON +used for research remain outside Git. No credential or private project inventory +was read for this packet. Local advisory coordination claims are released when +this subtask hands back; the pre-existing foreign expired lease is left untouched. From 6f6ddc1e647a43369397c913b9330949bc1c91ee Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=A0=D0=BE=D0=B1=D0=BE=D1=82?= Date: Mon, 21 Sep 2026 11:39:33 +0200 Subject: [PATCH 3/3] fix(release): preserve protected tag and prepare 0.25.1 --- .claude-plugin/marketplace.json | 2 +- CHANGELOG.md | 6 +++++- SKILL-CARD.md | 2 +- docs/evidence/research/2026-09-21-ecc-handoff.md | 12 ++++++++++++ package.json | 2 +- plugins/agent-stack/.claude-plugin/plugin.json | 2 +- 6 files changed, 21 insertions(+), 5 deletions(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 299e556..31b3aac 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -12,7 +12,7 @@ "displayName": "Agent Stack", "source": "./plugins/agent-stack", "description": "Four skills: agent-orchestrator — tool-calling loops, pipelines with checkpoints, provider routing with fallback, memory architecture, plus the wallet side of reselling LLM access; agent-evals — run/trace/thread evals, LLM judges, and fixtures grown from production; agent-interop — MCP servers and clients, A2A agent cards, the MCP Registry, and gateways; agent-harness — system prompts, tool shaping, workflow-vs-agent, and auditing an agent system.", - "version": "0.25.0", + "version": "0.25.1", "author": { "name": "ssheleg", "url": "https://x.com/sshlg93" diff --git a/CHANGELOG.md b/CHANGELOG.md index 3aa41f0..96e3a3a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,4 +1,4 @@ -## 0.25.0 — workbench harness contracts, with a bounded ECC transfer +## 0.25.1 — workbench harness contracts, with a bounded ECC transfer - Distinguish a kernel harness from the operating layer around an existing agent. The family is the latter; skills are components, not an OS security boundary. @@ -9,6 +9,10 @@ - Attribute the selective ECC review to commit `2b6e839771e53096d8451a213d40dc64ec8acac0`; retain the adoption matrix, source digests, acceptance cases and explicit limits in the research ledger. +- Version 0.25.0 was not released: its protected tag was mistakenly created + at the previous revision after GitHub rejected a merge method. The queued + release was cancelled. Version 0.25.1 preserves the tag history and publishes + the reviewed change from its actual merged commit. - No new hook, network dependency, telemetry or service is installed. Structural checks are not represented as measured improvements in agent outcomes. diff --git a/SKILL-CARD.md b/SKILL-CARD.md index c56f5f5..f8b008c 100644 --- a/SKILL-CARD.md +++ b/SKILL-CARD.md @@ -5,7 +5,7 @@ | Field | Value | |---|---| | Pack | `agent-stack` | -| Version | `0.25.0` | +| Version | `0.25.1` | | Skills | `agent-orchestrator`, `agent-evals`, `agent-interop`, `agent-harness` | | License | MIT | | Source | https://github.com/ssheleg/agent-stack | diff --git a/docs/evidence/research/2026-09-21-ecc-handoff.md b/docs/evidence/research/2026-09-21-ecc-handoff.md index d106df5..388a198 100644 --- a/docs/evidence/research/2026-09-21-ecc-handoff.md +++ b/docs/evidence/research/2026-09-21-ecc-handoff.md @@ -62,3 +62,15 @@ Local-only residue: a read-only temporary ECC clone and package inventory JSON used for research remain outside Git. No credential or private project inventory was read for this packet. Local advisory coordination claims are released when this subtask hands back; the pre-existing foreign expired lease is left untouched. + +## Release correction + +Parent independently reviewed the new reference and authorized publication. PR +#33 checks passed at 331f7eb. The initial merge method was rejected by GitHub, +but a shell command sequence incorrectly continued and tagged the old c681ef3 +commit as v0.25.0. The queued release run 35584412545 was cancelled before +publication. Repository rules refused deletion; no rules were bypassed and no +force push was used. The immutable incorrect tag is preserved. Publication now +targets **0.25.1** with the same reviewed content and this explicit correction. +Future merge, head verification, tag creation and tag push are separate checked +operations; a failed step cannot continue to the next mutation. diff --git a/package.json b/package.json index 369b078..e97b21c 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@ssheleg/agent-stack", - "version": "0.25.0", + "version": "0.25.1", "scripts": { "test": "python3 test/validate.py && python3 test/plant_guard_test.py && node test/installer_test.js && npm run test:audit", "test:audit": "for t in test/audit_regressions/*.py; do python3 \"$t\" || exit 1; done" diff --git a/plugins/agent-stack/.claude-plugin/plugin.json b/plugins/agent-stack/.claude-plugin/plugin.json index d22e81a..51b1c10 100644 --- a/plugins/agent-stack/.claude-plugin/plugin.json +++ b/plugins/agent-stack/.claude-plugin/plugin.json @@ -3,7 +3,7 @@ "name": "agent-stack", "displayName": "Agent Stack", "description": "Four skills: agent-orchestrator — tool-calling loops, pipelines with checkpoints, provider routing with fallback, memory architecture, plus the wallet side of reselling LLM access; agent-evals — run/trace/thread evals, LLM judges, and fixtures grown from production; agent-interop — MCP servers and clients, A2A agent cards, the MCP Registry, and gateways; agent-harness — system prompts, tool shaping, workflow-vs-agent, and auditing an agent system.", - "version": "0.25.0", + "version": "0.25.1", "author": { "name": "ssheleg", "url": "https://x.com/sshlg93"