You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
zuiho-kai/omni-reviewbot#15 — every review job now records what it produced: severity histogram (P0–P3), inline anchor rate, gate failures, posted review ids. The dashboard shows the aggregates; NULL renders as "未知", never as 0, and every rate is displayed with its sample size.
zuiho-kai/omni-reviewbot#19 — the "was it good" half: a daily read-only collector gathers 👍/👎 reactions on the bot's inline comments, thread replies, and each PR's final state. Maintainer convention: react 👍/👎 on the bot's inline comments — that is the signal the dashboard's useful-rate is computed from (GitHub doesn't support reactions on top-level review bodies).
eval: monthly review-quality probe of omni-reviewbot as a harness arm #109 — a monthly ground-truth probe: bash eval/dataset/run_reviewbot_monthly.sh <YYYY-MM> re-reviews the 15 train+val benchmark PRs and judges them blind against the pinned CC+Opus 5 baseline (paired, item-clustered statistics; results in eval/dataset/results/reviewbot/). This answers "is the review quality holding" with actual recall/precision deltas, monthly.
Closing as implemented. Live numbers appear on the kanban as reviews flow; the first monthly probe report will land under results/reviewbot/.
Activity
Merge pull request #104 from JiusiServe/docs/metrics-contract
zengchuang-hw commentedon Aug 27, 2026
以版本为周期进行评估,更新到kanban
指标:质量(GPT 5.6 Sol,已经合入的PR与人工检视意见)、效率(平均一个检视PR耗时)
zengchuang-hw commentedon Aug 27, 2026
1、metrics评估方案:抽样三个人工检视意见最多的PR,进行评估
2、workflow学习trace更新
tzhouam commentedon Aug 28, 2026
Shipped in three parts on the review-bot side:
bash eval/dataset/run_reviewbot_monthly.sh <YYYY-MM>re-reviews the 15 train+val benchmark PRs and judges them blind against the pinned CC+Opus 5 baseline (paired, item-clustered statistics; results ineval/dataset/results/reviewbot/). This answers "is the review quality holding" with actual recall/precision deltas, monthly.Closing as implemented. Live numbers appear on the kanban as reviews flow; the first monthly probe report will land under
results/reviewbot/.