Skip to content

feat(skills): skip unavailable candidate evaluation - #158

Merged
Pepps233 merged 10 commits into
fix/skills-evolution-uxfrom
feat/skip-skill-evaluator
Aug 31, 2026
Merged

Pepps233 merged 10 commits into
fix/skills-evolution-uxfrom
feat/skip-skill-evaluator

Conversation

@Pepps233

Copy link
Copy Markdown
Contributor

Summary

Allow skill evolution to complete without a configured retrieval and behavior evaluator. Deterministically validated candidates now record an explicit evaluation-skip transition, become eligible for operator-confirmed activation, and can be installed into the selected harness revision.

Reasoning

Production Diffex does not currently construct a skill evaluator. Candidate processing therefore stopped after deterministic validation, leaving generated drafts structurally valid but permanently parked and unavailable for activation.

A persisted skip transition keeps the temporary bypass explicit instead of treating structural validation as evaluation. The writer accepts an unevaluated candidate only when that exact transition is present, while retaining provenance, immutable artifact, workspace, inventory, harness-base, lease, and explicit-confirmation checks.

Changes

  • Resume parked structurally valid and evaluation-pending candidate work when no evaluator is available.
  • Persist retrieval-and-behavior-evaluation-skipped before accepting an unevaluated candidate.
  • Complete candidate work and prevent accepted skipped candidates from being rescheduled.
  • Allow mutation staging and activation with a null evaluation only when the explicit skip transition is valid.
  • Preserve the existing evaluator path and consume already-persisted evaluation reports during recovery.
  • Label missing evaluation as skipped in /evolve while keeping activation confirmation mandatory.
  • Replace the fixture-evaluator end-to-end test with the production no-evaluator flow across concurrent runtimes.
  • Document the temporary evaluation gap and its security limitations.
  • No dependency or changelog changes.

Testing

  • npm run check
  • cd packages/evolution && node ../../node_modules/vitest/dist/cli.js --run test/skill-candidate-worker.test.ts test/skill-candidate-registry.test.ts test/skill-writer.test.ts
  • cd packages/coding-agent && node ../../node_modules/vitest/dist/cli.js --run test/evolution-runtime.test.ts test/evolution-menu.test.ts
  • ./test.sh attempted: the evolution suite passed 217 tests, while the aggregate run failed in three unrelated coding-agent tests (agent-session-concurrent.test.ts and two existing harness skill-version assertions) plus an unhandled missing Anthropic API key rejection.

@Pepps233 Pepps233 self-assigned this Aug 30, 2026
@Pepps233
Pepps233 force-pushed the feat/skip-skill-evaluator branch from 70a72e6 to 1b2e496 Compare August 30, 2026 02:56
@Pepps233
Pepps233 force-pushed the feat/skip-skill-evaluator branch from 9e3daaf to 9cc0900 Compare August 30, 2026 15:10
@Pepps233
Pepps233 force-pushed the feat/skip-skill-evaluator branch from a139c89 to 045b668 Compare August 30, 2026 21:32
@Pepps233
Pepps233 merged commit 2eeee0c into main Aug 31, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant