Skip to content

fix(agent): reconcile and classify skill candidate retries - #161

Merged
Pepps233 merged 2 commits into
feat/skip-skill-evaluatorfrom
fix/skills-evolution-production
Aug 31, 2026
Merged

Pepps233 merged 2 commits into
feat/skip-skill-evaluatorfrom
fix/skills-evolution-production

Conversation

@Pepps233

@Pepps233 Pepps233 commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Reconcile legacy skill-candidate work and make synthesis retries follow durable error classification. Deterministic schema, artifact, and input failures now become terminal immediately, transient provider and transport failures retain bounded retries by synthesis identity across source jobs, and later qualified candidates continue through validation and evaluation.

Reasoning

Candidate synthesis previously counted retries against the source evolution job and ignored EvolutionExecutionError.retryable. Deterministic failures were retried three times, while the same synthesis identity could receive a fresh retry budget from a later job.

Legacy exhausted synthesis and completed candidate work awaiting evaluation could also remain stranded after startup. Persisting the synthesis idempotency key with failure diagnostics allows retries to follow the immutable synthesis identity, while explicit terminal reason codes distinguish non-retryable failures from exhausted transient attempts.

Changes

  • Reconcile legacy exhausted synthesis and completed candidate work that still requires evaluation handling.
  • Stop synthesis immediately for classified non-retryable schema, artifact, and input failures.
  • Retry only classified transient provider and transport failures.
  • Persist synthesis idempotency keys in failure diagnostics and count attempts across source evolution jobs.
  • Record distinct terminal reasons for non-retryable failures and exhausted transient attempts.
  • Continue processing later qualified candidate groups when another group cannot be retried.
  • Preserve the existing bounded backoff and legacy recovery behavior.
  • Add reusable provider-error message classification for thrown transport failures.
  • Cover deterministic termination, transient retries, cross-job identity accounting, legacy recovery, and persistence round trips.
  • Update recovery and package changelog documentation. No dependency changes.

Testing

  • npm run check
  • cd packages/ai && node ../../node_modules/vitest/dist/cli.js --run test/retry.test.ts (18 passed)
  • cd packages/coding-agent && node ../../node_modules/vitest/dist/cli.js --run test/evolution-runtime.test.ts (32 passed)
  • cd packages/evolution && node ../../node_modules/vitest/dist/cli.js --run test/skill-candidate-worker.test.ts test/skill-candidate-registry.test.ts test/filesystem-store.test.ts (33 passed)
  • ./test.sh attempted: all evolution tests passed, while the aggregate run failed in two existing test/suite/agent-session-harness-version.test.ts assertions that expect skill content digests instead of package versions. The same two failures reproduce unchanged on base commit a139c8972090e55002d84f9d8e0c702213cecfb6.

@Pepps233 Pepps233 self-assigned this Aug 30, 2026
@Pepps233
Pepps233 force-pushed the fix/skills-evolution-production branch from b9452f7 to 27b2f32 Compare August 30, 2026 21:32
@Pepps233
Pepps233 merged commit bf6f8d5 into main Aug 31, 2026
1 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant