Skip to content

All Bots share one GitHub identity, so an independent review cannot be proved #644

Closed as not planned

Description

@hsliuustc0106

Summary

Every Bot on the machine uses the same GitHub credentials (gh auth in the user's keyring), so a PR authored by one Bot and reviewed by another Bot shows the same GitHub user for both. "Independent review PASS before merge" (AGENTS.md) cannot be proved from the repository, and GitHub refuses to let that account approve its own PR, so review verdicts can only be left as COMMENTS.

What happens today

  • PR tree: add move_bot for reorganising an existing tree #636 was authored by hsliuustc0106; its review verdict was posted by hsliuustc0106 as well, even though the reviewing Bot was a different agent (审校官) on a different line of the team.
  • The Bot that writes the code and the Bot that reviews it share one host account, so the repository's audit trail cannot distinguish them.
  • gh pr review --approve on your own PR fails; the workaround is a COMMENT that nobody can verify as independent.
  • The repo's merge rule (independent review at the tip SHA, see AGENTS.md / docs/code-standards.md) is therefore enforced only by convention, not by GitHub.

This is the same class of gap as the message-buffer issue (#642) and the message-routing issue (#641): the team is multi-agent in the product, but the outside systems still see one identity.

Options

A. Per-Bot GitHub identity. A dedicated bot account (or a fine-grained token per review role) configured through a saved secret and used by that Bot's environment. GitHub then sees a real second author/reviewer and can enforce review rules. Cost: one account per reviewer role, token provisioning and rotation.

B. Attestation instead of identity. The review verdict is a signed, machine-readable artifact (comment with a marker, or a CI job re-running the checks and recording the verdict). The merge gate reads the attestation rather than the GitHub reviewer. No extra accounts, but the gate must be built and trusted.

C. Human-in-the-loop. The reviewing Bot produces the verdict, and the human owner posts the approve from their own identity. Accurate audit trail, but it puts a human in every PR.

My preference is B first, A later: B is enough to make the gate real and works today with one machine; A is the cleaner long-term model once multiple machines/contributors exist.

Where

  • host gh credentials shared by every Bot (no per-Bot identity today)
  • request_secret / Bot environment, if A is chosen
  • repo CI: the existing reviewed tip (report only) job plus docs/code-standards.md merge gate, if B is chosen

Acceptance

  • A PR authored by Bot A carries a review that GitHub (or the merge gate) can verify as coming from a different reviewer.
  • The merge gate refuses a PR whose only "review" is from the author's own identity.
  • No secret is printed or committed; provisioning a second identity does not require patching DSH.

Activity

  1. Birfy commented on Oct 21, 2026

    @Birfy
    Collaborator

    Written-by: Triage

    Cross-reference: this is the same root cause as #627 ("An owner decision on a PR cannot be told apart from Agent-written text: no proxy signature, one shared identity"), filed first and carrying the scope (required Written-by: first line, a fail-closed queue gate, and the write-layer half in #633).

    Keeping this open rather than closing it as a duplicate because the ask here is narrower and testable: an independent review cannot be proved. If #627 lands, the provable part becomes "who wrote the verdict"; it still does not prove the reviewer was independent of the author's identity. Say the word if you want that half tracked here, and it stays open with a concrete acceptance line; otherwise close it pointing at #627.

    Labelled area:security + area:automation so it is not left without an area.

  2. added
    needs-humanThe agent loop stopped; a person must decide (reason in the last comment)
    and removed
    questionFurther information is requested
    on Oct 23, 2026
  3. Birfy commented on Oct 23, 2026

    @Birfy
    Collaborator

    Triaged on be48393b. This is a decision, not a scoped fix: every option changes how review trust is established, and two of them need the owner's resources.

    What is true today, checked here: one shared GitHub account writes every PR and every verdict, so the repository cannot separate the author from the verifier; Written-by: is a self-report that can be omitted or name the wrong Bot (#633), and gh pr review --approve on one's own PR is refused, so PASS arrives as a COMMENT. The merge gate (enqueue-guard) reads the text, not an identity.

    The choice the owner has to make:

    • A. Per-Bot GitHub identity. Needs one account (or fine-grained token) per review role, provisioned and rotated by the owner. Then GitHub itself can enforce "review from someone other than the author". Cost is real and ongoing.
    • B. Attestation without identity. The verdict becomes something the gate can check mechanically: either a CI job re-running the focused checks at the tip and recording the result (already close to tests-ok + reviewed tip), or a signed marker whose key the gate holds. More text in a comment is not attestation — Written-by: is a self-report, not evidence: only the write layer knows which Bot wrote a comment #633 is exactly why.

    Until one is chosen, this stays needs-human: option A is a provisioning task for the owner, option B is a policy change about what "independent" means. Say A or B and I will write the acceptance criteria and flip this to agent-ready in one edit; option B can be split further (gate rule first, signed marker later) if you prefer a small first slice.

  4. added a commit that references this issue on Oct 25, 2026
  5. Birfy commented on Oct 25, 2026

    @Birfy
    Collaborator

    Written-by: Triage

    Pointer so the residual risk does not live only on a closed issue: the owner decided on 2026-10-25 not to introduce per-workstation signing keys (#1111, closed not planned), so “an independent review cannot be proved” still stands exactly as this issue describes — one shared Birfy identity, and a Verdict: is still text anyone with that account can write. The three cheap gates (a conclusion must name the current tip, one conclusion per tip, Attestation: as a self-report) catch stale-sha and replay accidents but not deliberate forgery; that is an accepted risk, recorded, not a fix. The signed-verdict code from #1137 is in main and inert (signers: null, no signers.json), so if this is ever revisited it is two config lines, not a rewrite.

  6. Birfy commented on Oct 29, 2026

    @Birfy
    Collaborator

    Written-by: Chief
    用户 2026-10-25 已决定不引入每工作台签名密钥(#1111 已 closed not planned)。替代做法:三道门已落地——结论必须点名当前 tip、一个 tip 只许一条结论、Attestation 自述;签名代码留在 main 但不启用。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:automationScheduled tasks and automationarea:securityAuth, grants, approvals, secretsenhancementNew feature or requestneeds-humanThe agent loop stopped; a person must decide (reason in the last comment)

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions