diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index b6206fe..31b3aac 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -12,7 +12,7 @@ "displayName": "Agent Stack", "source": "./plugins/agent-stack", "description": "Four skills: agent-orchestrator — tool-calling loops, pipelines with checkpoints, provider routing with fallback, memory architecture, plus the wallet side of reselling LLM access; agent-evals — run/trace/thread evals, LLM judges, and fixtures grown from production; agent-interop — MCP servers and clients, A2A agent cards, the MCP Registry, and gateways; agent-harness — system prompts, tool shaping, workflow-vs-agent, and auditing an agent system.", - "version": "0.24.3", + "version": "0.25.1", "author": { "name": "ssheleg", "url": "https://x.com/sshlg93" diff --git a/CHANGELOG.md b/CHANGELOG.md index 8ce73de..96e3a3a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,21 @@ +## 0.25.1 — workbench harness contracts, with a bounded ECC transfer + +- Distinguish a kernel harness from the operating layer around an existing agent. + The family is the latter; skills are components, not an OS security boundary. +- Add an on-demand `agent-harness` reference for install ownership and loaded + state, bounded hooks and failure policy, evidence-bearing handoff, artifact-bound + eval receipts, replay limits and scoped learning. Existing family owners remain + authoritative; no duplicate runtime, schemas or receipt store is introduced. +- Attribute the selective ECC review to commit + `2b6e839771e53096d8451a213d40dc64ec8acac0`; retain the adoption matrix, + source digests, acceptance cases and explicit limits in the research ledger. +- Version 0.25.0 was not released: its protected tag was mistakenly created + at the previous revision after GitHub rejected a merge method. The queued + release was cancelled. Version 0.25.1 preserves the tag history and publishes + the reviewed change from its actual merged commit. +- No new hook, network dependency, telemetry or service is installed. Structural + checks are not represented as measured improvements in agent outcomes. + ## 0.24.3 — the budget stops being estimated, and two skills stop naming a file they do not ship Hygiene from the 2026-09-13 family audit (HK-11). diff --git a/README.md b/README.md index 8907782..a452265 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,10 @@ add` puts this pack — at rank 500. Production patterns for building AI agent orchestrators — and for billing the LLM access they burn. -Part of the [ssheleg skill family](https://github.com/ssheleg/sshlg-skills). +Part of the [ssheleg agent harness](https://github.com/ssheleg/sshlg-skills): +a workbench layer of routing, delivery contracts and evidence around existing +agent hosts. This pack provides its agent-engineering skills. It does not replace +the host runtime or provide an OS security boundary. --- @@ -88,7 +91,7 @@ ACP, AGNTCY, AP2, Agent Skills — so an agent stops guessing. model, and the one where most agent bugs actually live: *the biggest performance improvements often come from clearly explaining tool usage in the system prompt*, and *even small refinements to tool descriptions can yield dramatic improvements*. Before adding a -retry or a sub-agent, it asks four questions about the text. Seven references — +retry or a sub-agent, it asks four questions about the text. Eight references — `system-prompt.md` (right altitude, enumerated vocabulary, and the three things reasoning models changed — starting with **do not add chain-of-thought**), `tools.md` (the agent–computer interface, with a worked before/after and poka-yoke), `techniques.md` @@ -98,6 +101,10 @@ environment's job), `audit.md` (seven tracks, evidence tiers, a plan instead of plus **`pi.md` and `pi-sdk.md`, the doctrine as a worked implementation**: Pi read end to end, each mechanism matched to the rule it instantiates, its divergences named, and the eight extension seams where a permission gate or a context rewrite can actually live. +`workbench-contracts.md` adds the operating layer around a host: managed install +ownership, bounded hooks, evidence-bearing handoff, artifact-bound evaluation and +observation limits, adapted selectively from a pinned ECC review. These are design +contracts; the pack installs no new hook, service or telemetry collector. It runs in both directions: **building a harness and auditing somebody else's are one checklist read forwards and backwards.** `scripts/audit_agent.py` is the mechanical half — diff --git a/SKILL-CARD.md b/SKILL-CARD.md index 38c3c09..f8b008c 100644 --- a/SKILL-CARD.md +++ b/SKILL-CARD.md @@ -5,7 +5,7 @@ | Field | Value | |---|---| | Pack | `agent-stack` | -| Version | `0.24.3` | +| Version | `0.25.1` | | Skills | `agent-orchestrator`, `agent-evals`, `agent-interop`, `agent-harness` | | License | MIT | | Source | https://github.com/ssheleg/agent-stack | diff --git a/docs/evidence/research/2026-09-21-ecc-handoff.md b/docs/evidence/research/2026-09-21-ecc-handoff.md new file mode 100644 index 0000000..388a198 --- /dev/null +++ b/docs/evidence/research/2026-09-21-ecc-handoff.md @@ -0,0 +1,76 @@ +# Agent-stack ECC transfer — handoff + +Objective: improve the agent-engineering component of the ssheleg harness with +selective ECC methods, keeping existing family contracts and runtime boundaries. + +Entry point: [research and transfer matrix](2026-09-21-ecc-harness.md). +Task packet: [scope, dependencies and checks](../specs/2026-09-21-ecc-harness-brief.md). + +## Completed + +Implementation commit: +[`3456965577a19ee3b44816b593304a8c8a0970cc`](https://github.com/ssheleg/agent-stack/commit/3456965577a19ee3b44816b593304a8c8a0970cc). +Branch: `codex/ecc-harness-contracts`. Prepared version: **0.25.0**. +The commit link becomes remotely accessible only after the parent pushes it. + +- New workbench contract reference with install ownership, hook bounds and + failure semantics, durable handoff, artifact-bound eval evidence and observation + limits; linked from the skill and layers reference. +- README positions this pack within the ssheleg agent harness; all release version + surfaces synchronized, including the skill card and changelog. +- ECC pinned with 15 file digests, selected-read scope, attribution and a + borrow/adapt/reject matrix. No ECC executable code or runtime dependency added. + +## Checks actually run + +| Check | Observed result | +|---|---| +| `npm test` | EXIT=0: structural checks, plant guard 9 cases, installer 11 cases, audit regressions green | +| `python3 test/validate.py` after final reference TOC correction | EXIT=0, 15 checks, v0.25.0 | +| `claude plugin validate . --strict` | EXIT=0, marketplace passed | +| `claude plugin validate ./plugins/agent-stack --strict` | EXIT=0, plugin passed | +| make-skill `audit_skill.py plugins/agent-stack/skills/agent-harness --house --quiet` | First run reported missing reference TOC; corrected; final 0 GAP, 19 PASS | +| `npm pack --dry-run --json` | 40 files; new reference included; research, `.env`, `.agent-sync` and `.git` absent | +| SHA-256 ledger comparison | All 15 pinned source hashes matched | +| Relative-link inspection of brief/research | All Markdown relative links resolved | +| `git diff --check` | EXIT=0 | + +The repository validator reports one unavailable check: strict family front-matter +reader is not found above this standalone checkout. The make-skill house auditor +ran separately. No behavioral model evaluation was run; structural checks do not +prove improved outcomes. The nine review cases are contract criteria, not executed +new-adapter tests. No deployment or npm publication has occurred in this packet. + +## Decisions and open work + +- Reuse task-pipeline packets, agent-sync coordination and family lifecycle + receipts; do not create another session store or global observer. +- Distinguish kernel and workbench harness meanings. Host capabilities and + missing enforcement stay visible. +- Observatory remains an optional integration proposal here. Its open-source + readiness, actual features and publication belong to the parent program. +- Parent owns independent semantic/security review, push/integration policy, + version release, installed-skill refresh, website edits and umbrella pin. + +**Exact next task:** independently review the new workbench reference and transfer +matrix against the pinned sources, then push/integrate the reviewed branch under +repository policy. Release 0.25.0 only after that review; update the umbrella pin +and verify the installed/loaded version separately. The prepared change does not +claim those follow-up operations have happened. + +Local-only residue: a read-only temporary ECC clone and package inventory JSON +used for research remain outside Git. No credential or private project inventory +was read for this packet. Local advisory coordination claims are released when +this subtask hands back; the pre-existing foreign expired lease is left untouched. + +## Release correction + +Parent independently reviewed the new reference and authorized publication. PR +#33 checks passed at 331f7eb. The initial merge method was rejected by GitHub, +but a shell command sequence incorrectly continued and tagged the old c681ef3 +commit as v0.25.0. The queued release run 35584412545 was cancelled before +publication. Repository rules refused deletion; no rules were bypassed and no +force push was used. The immutable incorrect tag is preserved. Publication now +targets **0.25.1** with the same reviewed content and this explicit correction. +Future merge, head verification, tag creation and tag push are separate checked +operations; a failed step cannot continue to the next mutation. diff --git a/docs/evidence/research/2026-09-21-ecc-harness.md b/docs/evidence/research/2026-09-21-ecc-harness.md new file mode 100644 index 0000000..739d1f7 --- /dev/null +++ b/docs/evidence/research/2026-09-21-ecc-harness.md @@ -0,0 +1,120 @@ +ssheleg skills — agent-harness · make-skill · task-pipeline · agent-sync + +# ECC → ssheleg harness: selective adoption ledger + +Research date: 2026-09-21. Owner: agent-stack / agent-harness. Task packet: +[bounded brief](../specs/2026-09-21-ecc-harness-brief.md). + +## Source boundary + +ECC was cloned read-only from `https://github.com/affaan-m/ECC` at +[`2b6e839771e53096d8451a213d40dc64ec8acac0`](https://github.com/affaan-m/ECC/tree/2b6e839771e53096d8451a213d40dc64ec8acac0). +Its package declares 2.2.2. The source ledger below names exact files and SHA-256 +values. Source observations are **static inspection**, not execution results. +No ECC installer, hook, test, package installation or background observer ran. +The wider repository was inventoried by paths; it was not audited in full. + +ECC root LICENSE: MIT, Copyright (c) 2026 Affaan Mustafa. The selected files show +no separate license headers overriding it. This change adapts methods in newly +written prose; no implementation, schema, command text or substantial upstream +text is copied. Do not infer the same review for uninspected assets/subtrees. + +Local baseline: agent-stack +[`c681ef3af6157d13becf53250373b418cca6c919`](https://github.com/ssheleg/agent-stack/tree/c681ef3af6157d13becf53250373b418cca6c919). +Family comparison: sshlg-skills +[`564d1f42a895abb8f0f53417695414b74dbf91a5`](https://github.com/ssheleg/sshlg-skills/tree/564d1f42a895abb8f0f53417695414b74dbf91a5), +README and `lib/lifecycle.js` read. This is a selected-contract comparison, not a +whole-project audit or a benchmark between the two projects. + +## Narrative recommendation + +**The ssheleg harness is an operating layer for agent work.** It routes work to +specialist skills, carries tasks through explicit contracts, preserves evidence +and resume context, and exposes what is installed and actually loaded. The skill +family is its expertise layer. Project Observatory can be its optional observation +component once its own readiness and privacy checks pass. + +The host owns model execution and tool permissions. The harness does not promise +sandboxing, universal hook support or complete leak prevention. Observatory's +credential requirements and private state are separate from the skills' runtime +requirements. Avoid calling this a replacement for Claude Code, Codex or other +agent hosts. Avoid publishing vendor leak counts without reviewed evidence. + +## Borrow / adapt / reject + +| Method | Verdict and reason | Family owner / destination | Evidence | +|---|---|---|---| +| Explicit managed install state, preservation of unowned files | **Adapt.** A skipped write must not grant uninstall ownership. Keep user changes visible. | Installer owner + make-skill; contract in new reference | ECC ownership guard; lifecycle inspection | +| Installed versus loaded status | **Retain existing family contract.** ECC doctor diagnoses managed files; family already distinguishes disk and reload receipts. No second store. | sshlg-skills lifecycle | ECC doctor; local `lib/lifecycle.js` | +| Hook profiles and per-hook disable | **Adapt.** Make effective capabilities, effects and failure behavior inspectable. | Host adapter; contract in agent-harness | ECC hook-flags | +| Input caps and subprocess timeout | **Adapt with stronger bounds.** Input bytes alone do not bound stream duration; subprocess timeout does not bound in-process awaits. | Host adapter | ECC hook-input; run-with-flags lines 249–287 | +| Fail-open generic hook exception | **Reject as a universal policy.** A required gate must not silently pass when its check failed. This is a design assessment, not an exploit claim. | Host adapter | run-with-flags lines 267–270 and 303–306 | +| Evidence-bearing save/resume | **Adapt content, retain local format.** Confirmed work, failures and next task are useful; a new global session store would duplicate task-pipeline. | task-pipeline existing handoff | save-session and resume-session | +| Hash-bound eval receipts and fixture-only replay | **Adapt contract.** Bind candidate, taskset and checker; integrity is distinct from correctness. | agent-evals / existing result envelopes | eval-harness receipt, capsule and architecture | +| Arbitrary candidate execution | **Do not import.** ECC itself disables it without verified OS containment. Keep that limitation explicit. | Host/environment security boundary | eval-harness gate opening contract | +| Instinct learning/background observer | **Defer.** Scope, consent, retention, cost and proven improvement need evaluation; do not auto-promote session content globally. | agent-orchestrator memory owner | continuous-learning-v2 SKILL | +| Entire ECC installer, hook bundle and host configs | **Reject wholesale import.** Adds parallel orchestration and unreviewed dependency closure. | Existing family routers remain entry points | Inventory plus bounded task scope | + +ECC observations above are grounded in the source files below. Family policy +choices (adapt/reject) are author judgment and await parent review; they are not +measured performance findings. No claim of superiority or outcome improvement. + +## Implemented delta and dependency closure + +- [Workbench contracts](../../../plugins/agent-stack/skills/agent-harness/references/workbench-contracts.md): + new on-demand doctrine and nine review cases. +- [Harness entry point](../../../plugins/agent-stack/skills/agent-harness/SKILL.md): + reference routing and kernel/workbench checklist distinction. +- [Layer reference](../../../plugins/agent-stack/skills/agent-harness/references/layers.md): + explicit broader workbench meaning, preserving kernel responsibilities. +- README, skill card and synchronized 0.25.0 manifests/changelog. + +Runtime additions: none. No new packages, interpreters, services, credentials, +telemetry, foreign scripts, schema stores or automatic downloads. Existing +agent-stack audit script still needs Python as before. Research files are Git +artifacts; only the reference ships inside the skill package. + +## Verification limits and next work + +Repository tests establish structural/reference/installer regressions. Strict +plugin validation checks manifest conformance. Package inspection confirms the +new reference is included and research/private machine state are not. These do +not measure agent behavior. The nine adapter review cases are design acceptance +criteria; no new adapter is implemented by this prose change. + +Before a future runtime transfer: implement in the existing owning module, +freeze representative baseline tasks, exercise no-op/conflict/unsupported-host +and tampered-evidence paths, and record real outcomes under agent-evals. Parent +review and release evidence are recorded in the adjacent handoff. + +## Source digest ledger + +`sha256(file bytes)` at the pinned ECC commit; line references above resolve there. +Sources marked “selected sections” were not semantically audited end to end. + +| Source | SHA-256 | Read scope | +|---|---|---| +| [LICENSE](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/LICENSE) | `326146379f01bb137c0a5d3c54770c1aa31076705c8b88a7f6b26a460f6221b2` | full | +| [package.json](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/package.json) | `650eaa8de580a3c8eadbc658aa5ffe3f8d50ba2be075dd327a62e1d70e1d7a69` | selected sections | +| [scripts/lib/install/ownership-guard.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/install/ownership-guard.js) | `000d8de192edceb32169b8725a797f908256f5a7e124c1803c29328eb07a36cc` | selected sections | +| [scripts/lib/install-lifecycle.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/install-lifecycle.js) | `4814affb0f3ffd3648e62a4987ce21d3a5318f6c8d4f5e2069b01e5ecbe5dfe5` | selected sections | +| [scripts/doctor.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/doctor.js) | `4ec5a1c7fe9124ebeea5d3d939270a16b0974b789558a8a9cc313cf73ec70b6e` | selected sections | +| [scripts/lib/hook-flags.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/hook-flags.js) | `1f5fbf2d2ebd0ab07a3e54406db18c2932ae7bf965513ec12c521da1be54425d` | full | +| [scripts/hooks/hook-input.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/hooks/hook-input.js) | `edae519f794befa4cf1dcbfb3c985d0db6f589342b7036ffb7ace61a9c3868b0` | full | +| [scripts/hooks/run-with-flags.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/hooks/run-with-flags.js) | `a5b3c3f4ed27f1d9d9c607d6819aab9985ad446239ea7d2825d4b040d2d8d87e` | selected sections | +| [commands/save-session.md](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/commands/save-session.md) | `a9488a8819d6c65fe4777f07ea72de6b0c4ee94f90c6bb33b3b8841916222db4` | selected sections | +| [commands/resume-session.md](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/commands/resume-session.md) | `b09bcb22feb10adbba50accd5dd97d09b60897aa94d05c6c12b6e579cb69cbac` | full | +| [scripts/lib/eval-harness/receipt.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/eval-harness/receipt.js) | `eacacccf8a6d06f38e83c8cee1bb3a6c608e9125115a989fd7f8a7f7e371d5e7` | selected sections | +| [scripts/lib/eval-harness/capsule.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/eval-harness/capsule.js) | `fd0bbbde8f383226fbcb093210e0f16594acd9f6ea0ae209e47a37cc39834b0c` | selected sections | +| [scripts/lib/eval-harness/gate.js](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/eval-harness/gate.js) | `76d6613aa7aacb9863c61859d20e554826c6f08e71c10221f0b6fea4d04e3c4f` | selected sections | +| [docs/architecture/eval-harness-frameworks.md](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/docs/architecture/eval-harness-frameworks.md) | `eec9ad255f71f68feb4d5e2ee9b2ececef6cd4dc4d1b246d015f3cc047406a61` | selected sections | +| [skills/continuous-learning-v2/SKILL.md](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/skills/continuous-learning-v2/SKILL.md) | `ddad12451a0d582c6ef1b197b7be193bef613cbc74eba4117fcb74fc7a4ecd00` | selected sections | + +--- + +**Made with [ssheleg skills](https://github.com/ssheleg/sshlg-skills)** + +- [`agent-harness`](https://github.com/ssheleg/agent-stack) — workbench contracts and ECC comparison +- [`make-skill`](https://github.com/ssheleg/make-skill) — attribution and package checks +- [`task-pipeline`](https://github.com/ssheleg/task-pipeline) — bounded brief and handoff +- [`agent-sync`](https://github.com/ssheleg/agent-sync) — local claims for shared metadata diff --git a/docs/evidence/specs/2026-09-21-ecc-harness-brief.md b/docs/evidence/specs/2026-09-21-ecc-harness-brief.md new file mode 100644 index 0000000..900cc3f --- /dev/null +++ b/docs/evidence/specs/2026-09-21-ecc-harness-brief.md @@ -0,0 +1,43 @@ +# ECC workbench-contract transfer — bounded task + +Objective: study ECC at a frozen revision and improve the agent-stack harness +reference without importing a foreign installer, hook or runtime. Authorized by +the operator's 2026-09-21 autonomous harness request; this is the agent-stack +packet in the parent multi-repository program. Existing model settings inherited. + +## Scope, dependencies and evidence + +- REQ-ECC-1: pin ECC and record a borrow/adapt/reject matrix, source paths and + digests. Check: research ledger and reproducible SHA-256 comparison. +- REQ-ECC-2: define a workbench harness and its limits; use existing family + contracts for routing, delivery, leases and evals. Check: reference review. +- REQ-ECC-3: document bounded hooks, install ownership, portable handoff and + artifact-bound evaluation. Check: linked reference plus repository validators. +- REQ-ECC-4: keep dependency-free doctrine and no new always-on behavior. + Check: diff and packed artifact inventory. +- REQ-ECC-5: preserve attribution, release metadata and a cold-reader handoff. + Check: version consistency, strict plugin validation and handoff entry. + +Inputs read: agent-harness SKILL and layers reference; make-skill SKILL and +enterprise reference; task-pipeline SKILL; agent-sync SKILL and local config; +agent-stack README, package and validator; sshlg-skills README and lifecycle +module; the ECC ledger names the external sources read. Contradictions: the +existing layers table uses harness narrowly for a kernel; the requested umbrella +uses it for a workbench. Name both meanings explicitly rather than implying the +family owns model execution or sandboxing. + +## Plan and boundaries + +1. Inspect ECC read-only; do not run its installers, hooks, dependencies or tests. +2. Write one on-demand reference and integrate its route into agent-harness. +3. Record matrix, source digests, review cases and limits; sync README/changelog. +4. Run existing repository tests, strict manifest validation and package check. +5. Commit this packet, send the exact diff/version to the parent for independent + review, integration, release and umbrella pin update. No push/release until + the parent coordinates it. No edits in sibling repositories. + +This is a documentation/reference change: no product UI or command behavior +changes. No new UX chain, deployment or runtime permissions. Shared metadata is +edited under local advisory agent-sync claims; the host has no enforcement hook. + +Resume: read ../research/2026-09-21-ecc-harness.md and the handoff beside it. diff --git a/package.json b/package.json index ddcdefa..e97b21c 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@ssheleg/agent-stack", - "version": "0.24.3", + "version": "0.25.1", "scripts": { "test": "python3 test/validate.py && python3 test/plant_guard_test.py && node test/installer_test.js && npm run test:audit", "test:audit": "for t in test/audit_regressions/*.py; do python3 \"$t\" || exit 1; done" diff --git a/plugins/agent-stack/.claude-plugin/plugin.json b/plugins/agent-stack/.claude-plugin/plugin.json index 1268618..51b1c10 100644 --- a/plugins/agent-stack/.claude-plugin/plugin.json +++ b/plugins/agent-stack/.claude-plugin/plugin.json @@ -3,7 +3,7 @@ "name": "agent-stack", "displayName": "Agent Stack", "description": "Four skills: agent-orchestrator — tool-calling loops, pipelines with checkpoints, provider routing with fallback, memory architecture, plus the wallet side of reselling LLM access; agent-evals — run/trace/thread evals, LLM judges, and fixtures grown from production; agent-interop — MCP servers and clients, A2A agent cards, the MCP Registry, and gateways; agent-harness — system prompts, tool shaping, workflow-vs-agent, and auditing an agent system.", - "version": "0.24.3", + "version": "0.25.1", "author": { "name": "ssheleg", "url": "https://x.com/sshlg93" diff --git a/plugins/agent-stack/skills/agent-harness/SKILL.md b/plugins/agent-stack/skills/agent-harness/SKILL.md index f02e472..44396f2 100644 --- a/plugins/agent-stack/skills/agent-harness/SKILL.md +++ b/plugins/agent-stack/skills/agent-harness/SKILL.md @@ -129,6 +129,7 @@ and this pack's repository validator fails the build on a reference that does no | [`references/system-prompt.md`](references/system-prompt.md) | you are **writing or fixing the prompt** — altitude, structure, vocabulary, dynamic context, and what changes for reasoning models | | [`references/tools.md`](references/tools.md) | the model **picks the wrong tool, or none** — the agent–computer interface: how many, named how, described how, returning what | | [`references/techniques.md`](references/techniques.md) | you are choosing between **ReAct, reflection, voting, planning** and the rest — every entry carries a verdict for production, not a benchmark score | +| [`references/workbench-contracts.md`](references/workbench-contracts.md) | assembling a **harness around an existing agent** — install ownership, bounded hooks, handoff, artifact-bound evidence and observation limits; selective ECC methods | | [`references/layers.md`](references/layers.md) | deciding **what your harness owns** — kernel, workbench and product layers, and why permission boundaries are usually somebody else's job | | [`references/audit.md`](references/audit.md) | reviewing **an agent system you did not build** — seven tracks, evidence tiers, and a prioritized plan | | [`references/pi.md`](references/pi.md) | you want the doctrine above as a **worked implementation** — Pi's sessions, compaction, config, skills, trust and containerization, each matched to the rule it implements, and the places it deliberately disagrees | @@ -202,6 +203,7 @@ prompt. ## Checklist — a harness worth shipping +- [ ] Kernel or workbench harness named explicitly; installed, loaded and enforced capabilities distinguished - [ ] Workflow-versus-agent decided deliberately, and the simpler option was actually tried - [ ] Static-versus-dynamic decided too — static preferred for predictability; a run that must be auditable keeps a complete execution record (not merely a static shape) - [ ] System prompt at the **right altitude** — heuristics, not hardcoded branches, not vague hope diff --git a/plugins/agent-stack/skills/agent-harness/references/layers.md b/plugins/agent-stack/skills/agent-harness/references/layers.md index a5f62e2..2869b61 100644 --- a/plugins/agent-stack/skills/agent-harness/references/layers.md +++ b/plugins/agent-stack/skills/agent-harness/references/layers.md @@ -35,7 +35,16 @@ will not converge no matter how long it runs. The distinction is not quality. A kernel is *supposed* to be smaller than a workbench; that is what makes it legible. -## What a harness owns +## The broader workbench meaning + +“Harness” also names an operating layer around an existing agent: skills, routing, +work contracts, hooks, installation and evidence. This is the family’s umbrella +meaning. It does not imply ownership of the model loop or OS permissions. The +three-layer table above uses the narrower kernel meaning. State the meaning +before comparing products; [workbench-contracts.md](workbench-contracts.md) gives +the operational contract and selective ECC implementation lessons. + +## What a kernel harness owns If you are building at the kernel layer, these are yours and nobody else's: diff --git a/plugins/agent-stack/skills/agent-harness/references/workbench-contracts.md b/plugins/agent-stack/skills/agent-harness/references/workbench-contracts.md new file mode 100644 index 0000000..3505842 --- /dev/null +++ b/plugins/agent-stack/skills/agent-harness/references/workbench-contracts.md @@ -0,0 +1,174 @@ +# Workbench contracts — a harness around an existing agent + +**Load this when:** assembling skills, hooks, installers and observation tools into +an operational harness, or evaluating what to adopt from another collection. + +**Spec pinned:** ECC 2b6e839771e53096d8451a213d40dc64ec8acac0 (2.2.2), selective source review · read 2026-09-21 + +## Contents + +- What the word promises +- Installation is an owned change +- A hook needs an execution contract +- Resume from evidence +- Bind evaluations to artifacts +- Observation and learning permissions +- Adoption review cases +- Source and attribution + +## What the word promises + +A **workbench harness** is the operating layer around an agent host: it selects +instructions, defines work and evidence, resumes interrupted tasks, manages its +installed components and makes their state inspectable. Skills are one component. +A **kernel harness** owns the model/tool loop. Say which one you mean; a workbench +can support an existing agent without replacing that agent's runtime. + +For the ssheleg family, routing and specialist skills guide the work; +`task-pipeline` owns scope, evidence, dependencies and resume artifacts; +`agent-sync` owns coordination where configured; `make-skill` owns packaging and +installation review; `agent-evals` owns behavioral evidence. An observation tool +may report repository, credential or project health alongside these contracts. +It does not become a sandbox, an authorization service or proof that every secret +has been detected. Its optional service dependencies must remain separate from +the dependency-free skill layer. + +These are design contracts, not a claim that every family host already implements +every mechanism below. Inventory each host's capabilities before claiming parity. + +## 1. Installation is an owned change + +A package name on disk is not an installation receipt. Record the source revision +or package integrity, component version, host/channel, intended root, files or +configuration keys owned, installed digests, and previous state needed to undo it. +Keep local absolute paths in local receipts; public evidence uses sanitized paths. + +- Plan the exact changes first. Existing unowned files remain user-owned even when + their names match a package file. A skipped write must not create an ownership + claim that lets uninstall remove it later. +- Merge only owned configuration keys; preserve unrelated configuration. Treat + changed owned files as conflicts to review, not permission to overwrite them. +- Validate containment and symlinks at mutation time as well as planning time. + A digest describes bytes; it does not establish safe filesystem ownership. +- Diagnose missing, changed, conflicting and unknown separately. Update, repair + and uninstall must preserve user edits or expose a concrete reviewable conflict. +- Distinguish **published**, **downloaded**, **installed**, and **loaded**. A running + agent may still use an old copy after a successful update. Require a host reload + receipt when available; otherwise report loaded version as unknown. + +Reuse the family's existing installer and reload receipts. Do not add a second +receipt store in a skill. Installer implementation belongs to its owning package, +with lifecycle tests covering install → edit → update → uninstall. + +## 2. A hook needs an execution contract + +For each hook, record event and matcher, supported host versions, input/output +schema, effects, input/output byte limits, wall-clock deadline, cancellation and +child-process cleanup, idempotency key where needed, profile/default, disable +control and failure policy. Expose what is effectively enabled, including where +that setting came from. A profile name alone proves none of these properties. + +Use a small default set. Expensive, networked or learning hooks are explicit +capabilities, with a documented no-hook/manual path on unsupported hosts. A +security gate that is required but unavailable blocks its protected action; +a convenience hook may fail open only while reporting that it did not run. +Neither becomes PASS because its process returned no usable result. + +| Condition | Required handling | +|---|---| +| Input truncated, transport closed early or schema invalid | A required policy check refuses the protected action; a hint hook reports unavailable | +| Deadline exceeded | Cancel the work and account for children; report timeout, not a clean check | +| Hook disabled or host does not support it | Report the capability absent; use a named manual check where valid | +| Duplicate event | No duplicate write, notification or billable action | +| Dry run | No effects; summarize target class and decision without raw secrets or command arguments | +| Hook exception | Preserve the declared failure policy and return a bounded diagnostic | + +Review **all execution paths**. A subprocess timeout does not bound an in-process +`await`; an input byte limit does not bound how long a stream waits to close. +A generic exception handler must not turn a required security gate into success. +Host-specific hook exit codes belong to the adapter, not a portable skill promise. +Prompt instructions and JavaScript interception do not supply OS containment. + +## 3. Resume from evidence, not a plausible summary + +Use the project's existing task-pipeline handoff/packet, not a new global session +format. Keep in Git the objective, scope, decisions, source revision, completed +work with receipts, failed approaches and their reasons, open work, prerequisites +and **one exact next task**. Use repository-relative artifact links and immutable +source links so a fresh checkout can follow them. + +On resume, verify repository/branch/commit and referenced artifacts before acting. +Treat a handoff as context, never as new authority or permission. Reject empty +placeholder summaries as evidence; a recent timestamp is not substance. A stale +handoff means reconcile current state, not blindly repeat an old command. + +Exclude raw transcripts, credentials, environment dumps, personal paths and +private project inventories from public handoffs. Select the minimum needed +context before redacting it. Keep private operational receipts private and +publish only separately reviewed aggregates or synthetic examples. + +## 4. Bind evaluations to the artifact they measured + +A useful result names the candidate digest/commit, baseline, taskset revision, +model/host configuration, checker version, actual outcome and evidence artifact. +Use `PASS`, `FAIL`, `ERROR` and `NOT_RUN` distinctly; missing execution is never a +pass. Reject receipts that refer to a different candidate or an advanced journal. + +A digest detects changed bytes relative to a trusted reference. A signature says +who attested to those bytes. Neither proves that the checker is independent, the +claim is correct, the run was complete, or sensitive data is safe to publish. +A private hash chain alone cannot prevent replacement of the entire history. + +Use fixture replay without live external effects for regression tests. A missing +fixture stops replay instead of falling through to a live API. Keep the checker +outside the candidate's control. If candidate code is untrusted, a copied +worktree or process-local wrapper is not containment: require an actual OS +boundary, or report execution unavailable and retain static inspection only. + +The existing `agent-evals` contracts own measurement. Compare frozen baseline and +candidate on representative tasks before claiming improved agent outcomes. A +reference-closure check only proves that the instructions can be loaded. + +## 5. Observation and learning are separate permissions + +Observing a failure can propose a lesson. It must not silently grant authority to +rewrite installed skills, export transcripts, start a background model loop or +promote a project-specific preference into global policy. Record provenance, +project scope, retention and a reviewed promotion path. Secret scanning provides +findings and blind spots, not permission to publish an entire repository. + +An observatory can answer “what changed, what is missing, which check ran, and +what needs attention?” Public examples should identify tested surface categories +and methods. Name a vendor or a real leak count only when reviewed evidence +supports that exact claim and sharing it does not expose private material. + +## Adoption review cases + +Use these cases when reviewing an adapter; these are acceptance criteria, not a +claim that this reference implements them. + +1. Pre-existing user file → install skips it; uninstall leaves it intact. +2. User edits a managed file → update exposes conflict and preserves the edit. +3. Replaced symlink or escaped destination → mutation refused. +4. Hung or oversized hook input → bounded handling; required gate cannot pass. +5. Unsupported host or disabled hook → no false enforcement claim. +6. Empty or wrong-project handoff → no unverified resume. +7. Changed candidate or missing replay fixture → evaluation cannot pass. +8. Receipt verifies but checker was never run → evidence is NOT_RUN. +9. Private canary in an evidence payload → export rejected or safely transformed + before publishing; the raw value never appears in diagnostics. + +## Source and attribution + +Methods reviewed from **ECC**, Copyright (c) 2026 Affaan Mustafa, MIT: +[license](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/LICENSE), +[installation ownership](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/install/ownership-guard.js), +[hook execution](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/hooks/run-with-flags.js), +[hook profiles](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/scripts/lib/hook-flags.js), +[session evidence](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/commands/save-session.md), +[eval contracts and limits](https://github.com/affaan-m/ECC/blob/2b6e839771e53096d8451a213d40dc64ec8acac0/docs/architecture/eval-harness-frameworks.md). + +This is independently written doctrine adapting bounded methods. No ECC source, +installer, hook, schema, session format or runtime dependency is vendored. The +family retains its own artifact owners. Recheck the pinned implementation before +using these references to assess a newer ECC release.