Repository navigation
Implement skills-cache-continuity exec-harness scenario #33
Description
Objective
Implement the first cache-sensitive Codex Lab scenario proving skills, project instructions, prompt-prefix stability, and resumed thread continuity behave together.
Finish Line
The exec harness has a deterministic skills/cache/thread-continuity scenario that protects the first Codex Lab MVP invariant.
Current Status
State: Active. Next action: wait for exec-harness automation fixes and cache assertion primitives, then draft the scenario fixture.
Scope
In:
- Workspace fixture with
AGENTS.mdand multiple skills. - Mandatory and delegated skill trigger checks.
- Two-turn continuation/resume check.
- Prompt-prefix stability and cache-ratio assertions using deterministic fake Responses usage.
- Bounded context checks so skill/project docs appear once.
Out:
- No local-LLM dogfood variant in the first PR.
- No broad skill system redesign unless the scenario exposes a blocking product bug.
Acceptance Criteria
- Scenario is deterministic with fake Responses.
- Turn 1 proves correct project and skill context injection.
- Turn 2 proves continuation without duplicate instruction bloat.
- Token count, cached-token count, cache ratio, and prompt-prefix stability are asserted.
- Failure output makes the broken invariant obvious.
Relationships
Child of #28. Blocked by exec-harness automation fixes and cache assertion primitives.
Validation
Run the specific exec-harness scenario locally and in CI. Record expected token/cache thresholds in the scenario or test comments.
Decisions
This is the first product-quality gate because prompt/context continuity underpins later agents, local LLM roles, review, and Desktop work.
Open Questions
Should the scenario require exact prefix equality or a percentage threshold such as >95% over a fixed prefix window?
Activity
shiny-code-bot commentedon Jun 12, 2026
Completed by PR #40, merged as
9d07f955f452cc2f2475b2d3c7f67c41472e2fbd.What landed:
skills-cache-continuityexec-harness scenario.AGENTS.md, multiple skills, mandatory/delegated trigger guidance, prompt-prefix stability, exact token usage, cached token count, cache ratio, and prompt bloat bounds.Validation:
python3 tools/codex-exec-harness/harness.py tools/codex-exec-harness/scenarios/skills-cache-continuity.json --codex-bin "$PWD/codex-rs/target/debug/codex"python3 -m unittest tools/codex-exec-harness/test_harness.py(19 tests passed)python3 -m unittest discover -s tools/codex-exec-harness -p 'test_*.py'(23 tests passed)python3 tools/codex-exec-harness/run_all.py --codex-bin "$PWD/codex-rs/target/debug/codex" --report-json .tmp/codex-exec-harness/report-skills-cache-continuity.jsonjust fmt(completed with pre-existing Ruff/TOMLexclude-newer = "7 days"warnings)git diff --checkReview: