feat(runeval): offline trajectory evaluation scorecard (/runreport) - #2535
Merged
Merged
Conversation
- internal/runeval: pure, deterministic trajectory analyzer. Walks session messages to score tool-loop efficiency: duplicate tool calls (identical read-only repeats penalized, stateful writes reported only), superseded failed attempts, error rate, LLM overhead share (compaction/strategist/ verify), wasted token estimate, and actionable findings. - Wasted-call semantics: a call key's LAST result is the informative one; earlier superseded attempts count as wasted when they failed or returned byte-identical read-only output. A successful retry of a failed call is penalized once (the failed attempt), not twice. - TUI: /runreport slash command evaluates the live session offline (no LLM calls) and prints the scorecard; wired into busy-safe commands, Tab completion, and en/zh i18n descriptions. - docs: document /runreport in docs/guide/slash-commands.md. Co-Authored-By: ggcode <noreply@ggcode.dev>
topcheer
pushed a commit
that referenced
this pull request
Sep 19, 2026
3 PRs this cycle: #2533 memory recall-time conflict arbitration, #2535 runeval offline scorecard (/runreport), #2542 startup freeze 3-layer + usage probe trigger/URL-matching/key-resolution/render chain (fetch Cmd-as-Msg drop fix). verify-ci passed (GOFLAGS=-p=1). Co-Authored-By: ggcode <noreply@ggcode.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Offline Trajectory Evaluation（/runreport scorecard）
研究依据（在线调研，sa-86）
Gap
ggcode 有 191 个在线 detector 与丰富的运行时遥测，但没有任何离线 run 复盘机制——用户无法量化"这个 session 跑得怎么样"（迭代效率/工具浪费/验证充分度），只能靠印象。
实施（9 文件 +754，commit 88fde5e）
新包 internal/runeval（runeval.go 470 行 + 247 行测试）
TUI 接线：新
/runreportslash 命令（三语 i18n + completion + slash info）验证
go build -tags goolm ./...通过internal/runeval测试全绿（0.7s）；tui 定向编译/回归通过交付说明
agent 验证阶段命令僵死（acp 测试进程 S 态挂起，与本包无关），commit 已安全落盘；父级核验构建+定向测试后代为完成 push+PR。
Co-Authored-By: ggcode noreply@ggcode.dev