agent: non-atomic outcome semantics for mutating tool calls - #2458
Merged
Merged
Conversation
When a mutating tool call (run_command, git_commit, write_file, ...) fails ambiguously -- timeout, cancellation mid-execution, or panic -- the side effect may be partially or fully applied, but the harness returned a bare error with no semantics. The model assumes nothing happened and re-issues the identical call, risking duplicate commits and doubled edits. Research basis: "Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failures" (arXiv:2608.02645) recommends the framework carry execution semantics: classify outcomes and guide verification-before-retry. Implementation (execution semantics at the tool boundary, same layer as transient_retry; not a detector): - classifyMutatingFailure: pre-exec rejection (validation/permission/ user-declined -> [no-side-effect] safe to retry) vs ambiguous mid-execution failure ([non-atomic] verify state, do not blind re-issue). Shell tools are never classified pre-exec: cmd1 && cmd2 failing at cmd2 still applied cmd1. Read-only tools are excluded entirely. - mutatingLedger: per-run FIFO-bounded map of ambiguous attempts keyed by (tool, fnv32a(args)). A re-issue of the identical call is annotated with double-apply risk whether it succeeds or fails again. - Wired into executeToolCall around executeWithTransientRetry; ledger resets per run; [non-atomic] registered as a critical guidance tag so the note cannot be suppressed by the per-turn budget. Co-Authored-By: ggcode <noreply@ggcode.dev>
Agent literals in tests (&Agent{}) leave mutateLedger nil; the
executeToolCall wiring probes it on every mutating call. Make
lookupAmbiguous/recordAmbiguous/reset nil-safe (lookup returns 0,
record/reset no-op) so future tests constructing bare Agents cannot
panic, and add TestMutatingLedgerNilSafe covering the full nil-Agent
annotate path.
Co-Authored-By: ggcode <noreply@ggcode.dev>
topcheer
commented
Sep 16, 2026
topcheer
left a comment
Owner
Author
There was a problem hiding this comment.
复审通过 ✅(协查两重点核销)
① 非原子判定白名单准确性:hasNonAtomicSemantics 组合三个既有维护的集合(shellSideEffectTools+gitStateTools+mutatesSourceTree)而非新造平行清单——漂移风险最小化设计;漏判面严格等于 mutatesSourceTree 的既有覆盖面(与 #2452 守卫同源,#1475 家族历史已多轮加固)。表测覆盖分类边界。
② 与 #2452 部分并行交互:mutating 工具本就 batch-wide skip 回落串行(#2452 守卫),注解发生在串行路径的 tool result 上——preExec 路径无关性有专测(TestAnnotateMutatingOutcomePreExec);双重应用检测(retry 成功后回访 ledger)有 DoubleApplyOnRetrySuccess 钉住。
语义设计认可:框架承载执行语义(arXiv:2608.02645 方向)——歧义失败(timeout/cancel/panic)注解"副作用可能已生效,先验证再重试";ledger 有界淘汰(Eviction 测试);成功清账(SuccessClean)。与 effect ledger(#2449)互补:那个管"重试时提示先前尝试",这个管"失败结果本身标注非原子性"——正交域。
8 测试全语义面。CI 绿。可合并。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
当一个变更型工具调用(
run_command、git_commit、write_file、git_stash等)歧义性失败——超时、执行中被取消、panic——副作用可能已部分或全部生效,但 harness 只返回一个裸错误(tool error: context deadline exceeded)。模型无从知晓这一点,会假设调用未发生并原样重发,导致重复提交、重复编辑、半途生效的批量操作。研究依据
实现(工具边界的执行语义,与 transient_retry 同层;非 detector)
classifyMutatingFailure:pre-exec(校验/权限/用户拒绝 → 肯定未生效):追加[no-side-effect],提示安全修参重试,省去一轮验证;ambiguous(超时/取消/panic → 未知):追加[non-atomic],指示先git log/status、读文件验证实际状态,再按缺失部分补偿,不要盲目重发;cmd1 && cmd2在 cmd2 失败时 cmd1 已生效);只读工具完全排除。mutatingLedger(每 run 重置、FIFO 有界 128 条,键为(tool, fnv32a(args))):模型重发完全相同的变更调用时,无论成功/再失败都追加双重生效风险警告。executeToolCall在executeWithTransientRetry前后各一次查找/标注(变更型工具本就不参与自动重试,单次执行语义成立);ledger 随 run 重置;[non-atomic]注册为 critical guidance tag,不受每轮预算抑制。测试
mutate_outcome_test.go:分类矩阵(超时/取消/panic/pre-exec/shell 退出码不误判/只读工具不标注)、ledger 记录/重置/驱逐、四类标注路径、critical tag 注册断言;-race通过;go test -tags goolm ./internal/agent/全量通过;go vet/go build(pre-commit)通过。