Skip to content

fix: give each admission target a bounded share of the sweep - #146

Merged
Dong1017 merged 4 commits into
InferMatrixLab:mainfrom
Dong1017:fix/knowledge-admission-bounded-scan
Oct 26, 2026
Merged

Dong1017 merged 4 commits into
InferMatrixLab:mainfrom
Dong1017:fix/knowledge-admission-bounded-scan

Conversation

@Dong1017

@Dong1017 Dong1017 commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Refs #140

  • Accepted scope / delegation: the bounded-scanning defect described in [Bug] Admission sweep: shared history deadline starves later PRs #140 — a later target starved by a single sweep-wide deadline. No model/effort policy change, no branch-protection change, no new service.
  • What this PR delivers: each admission target gets its own bounded opportunity to read its status history, the sweep keeps one overall deadline and one publication reserve, and a retry spends its budget on the targets an earlier attempt left unfinished instead of re-publishing the ones it already settled.
  • What it does not deliver: any change to invalidation, status-capacity limits, shared-head checks, lifecycle revalidation or the per-PR verdict itself; those are preserved and their tests are unchanged.

Knowledge authorship declaration

{
  "schema_version": 1,
  "head_sha": "5ac8b933133913ba2ea35d313a83e572340e0867",
  "human_only": false,
  "material_ai_authors": [
    {
      "tool": "Claude Code",
      "model_id": "claude-opus-5-5",
      "effort": "max",
      "role": "author"
    }
  ]
}

The defect

check_prs set one absolute deadline = now + 60 before the evaluation loop, and
history() refused to continue once the remaining time fell to the 15-second
reserve. Targets are evaluated serially, and every other read for a target — the
file list, comments, commits — also spends that same clock. The sweep therefore
had 45 seconds of history budget for all targets.

Open PRs are listed newest-first, so the order was stable: the first eleven
targets consumed the window, and #113, #97, #92, #90, #55, #51 and #31 then
received history-unavailable in every attempt. The runner retried the same
prefix from the same starting point, so no attempt could reach the tail.

A bounded share alone is not sufficient, and the first revision of this PR was
incomplete for that reason. Rotation moved the starting point but still
invalidated and re-evaluated every target, so the last attempt's exhausted tail
replaced successes an earlier attempt had already published — the run still ended
incomplete, with useful budget spent twice on the same targets.

flowchart LR
    subgraph before["Before"]
        A1["One 60 s deadline"] --> A2["Earlier targets spend it"]
        A2 --> A3["Later targets: history-unavailable"]
        A3 --> A4["Retry replays the same prefix"]
        A4 --> A2
    end
Loading

The correction

flowchart LR
    subgraph after["After"]
        B1["120 s sweep bound"] --> B2["Share = remaining / targets left, max 30 s"]
        B2 --> B3["Early target returns its unused share"]
        B3 --> B4["Settled targets written to a shared plan"]
        B4 --> B5["Retry re-checks the premise, then evaluates only what is left"]
    end
Loading

target_deadline(deadline, remaining_targets) returns
now + min(30, (deadline - 15 - now) / remaining_targets). A target that finishes
its read quickly leaves the rest of its share available to the targets after it,
a single slow head is capped instead of holding the sweep, and inside the
publication reserve the share is negative, so the read still fails closed.

Attempts share one plan file (--plan, created and removed by the runner under
RUNNER_TEMP, keyed to GITHUB_RUN_ID). Each attempt writes the targets it
settled — compliant, noncompliant and the two capacity states — together with
the premise the verdict was assessed against. The next attempt reuses an entry
only after a single pulls/{n} read shows the same head, base ref, base SHA, state
and declaration body; a closed target, a moved base, an edited declaration or a new
head falls back to full evaluation. A plan written for a different policy_sha is
discarded, because a different trusted base means different loaded code and policy.
Rerunnable outcomes — unreadable evidence, a failed status write, an advanced or
newly-entered target — are never carried, so a genuine failure is retried and stays
visible instead of being cached as done.

Changed artifacts and evidence

Requirement (#140) Actual change Check / evidence and result Remaining gap
Multi-PR regression consumes more than the former effective window SWEEP_SECONDS = 120, TARGET_SECONDS = 30, HISTORY_RESERVE = 15 withheld once per sweep test_api_every_target_is_evaluated_past_the_former_shared_window: 18 targets at 4 s of ordinary metadata work each (72 s total, past the former 45 s window) all return compliant, and every head's newest status is success; the test also asserts the sweep really exceeded SWEEP_SECONDS / 2 —
Later targets get a bounded opportunity instead of starving target_deadline() divides the remaining sweep time between the targets still to be evaluated, capped per target test_target_deadline_gives_each_target_an_equal_share: equal shares for 4 remaining, the cap for 1–2 remaining, and no share once the reserve is reached —
A slow target cannot starve the next one The share is per target; the sweep deadline stays as the outer bound test_api_a_slow_target_cannot_starve_the_next: a 400-entry foreign history makes target 0 outlive its own share (history-unavailable, failure published), and target 1 is still evaluated to compliant —
Genuine failures stay visible and fail closed Unchanged: history-unavailable is still written immediately inside the evaluation pass, and a failed write becomes retryable publication-failed test_api_slow_history_scan_stops_before_the_workflow_deadline, test_api_history_failure_is_written_before_the_next_target, test_api_failed_history_failure_write_is_retried, test_api_foreign_page_budget_still_publishes_the_failure —
Retries advance instead of replaying the prefix Plan file carried across attempts; only unsettled targets are evaluated test_api_retries_keep_completed_targets_instead_of_rewriting_them: 18 targets at 8 s each, so one sweep starves its tail; attempt 1 reports history-unavailable for the tail, attempt 2 finishes exactly that tail, every target's newest status is success, and no target settled by attempt 1 receives any write in attempt 2. The workflow test asserts the runner passes one --plan file —
A carried verdict must not outlive its premise still_current() re-reads head, base, state and declaration body, the approval comments that were read, and head ownership across the open-PR collection; plan_file() scopes the plan to one trusted base; a missing premise falls back to full evaluation test_api_a_carried_target_with_changed_premises_is_evaluated_again (same head, edited declaration → noncompliant), test_api_a_revoked_approval_is_not_carried_as_a_settled_success (unchanged snapshot, revoked exception approval → noncompliant), test_api_a_new_shared_head_peer_invalidates_a_carried_success (a peer on another base shares the head without entering the target list → noncompliant), test_plan_is_discarded_when_the_trusted_base_changes —
A stored premise must be the one that produced the published verdict A target reassessed during the publication pass writes back that reassessment's snapshot, approval comments and ownership before the plan is stored; an unreadable open-PR listing refuses the carry instead of treating the target alone as a unique owner test_api_a_reassessed_verdict_carries_the_premises_it_actually_used (refusal → approval appears before publication → success stored with the new approval → approval removed before retry → noncompliant), test_api_an_unreadable_collection_never_carries_a_settled_success (unverified, rerun, exit 2), test_api_carry_revalidation_tolerates_an_unreadable_target (a failed carried-target read falls back to evaluation and the other target still progresses) —
Overall timeout, page bounds and reserve stay explicit 50 pages unchanged; 120 s sweep; 15 s reserve Asserted in the workflow test: 3 * (SWEEP_SECONDS + TARGET_SECONDS) = 450 s < 600 s job timeout —
Existing invalidation, capacity and lifecycle behaviour preserved No behavioural change on those paths Full tests/test_knowledge_model_policy.py passes unchanged apart from the two deadline tests whose expectations this fix inverts —
Operational evidence distinguishes sweep execution from verdicts No change to kind vocabulary or the summary; a starved read is still history-unavailable, never a rejection Kind table and RERUN_KINDS unchanged Existing-PR evidence needs a run on main after merge; this PR cannot produce it
Closeout diagram Before/after flow added to docs/knowledge-model-policy.md and to this description Reviewed against target_deadline and check_prs directly —

Checks and dependencies

  • Checks actually run (repository .venv, Windows):
    • python -m pytest tests/test_knowledge_model_policy.py — 130 tests collected, all passed (exit 0).
    • python -m ruff check tools/check_knowledge_model_policy.py tests/test_knowledge_model_policy.py — All checks passed.
    • bash -n tools/run_knowledge_model_policy.sh — syntax clean.
  • Not run locally: the wider repository suite and the device lanes. Nothing outside the policy checker, its tests, its runner and its document is touched; CI is the authoritative run for this head.
  • Dependencies / integration assumptions: no workflow, policy-config or runner-permission change. The new flags are internal to the trusted runner, and PR text still never supplies them.

Expected red check: declarations

declarations is expected to fail on this PR and cannot pass until it merges. It is not a
statement about this diff.

The workflow checks out main (ref: main) and runs only trusted code, which is the property
that makes the check safe for pull_request_target. The code it runs is therefore the checker on
main at 65147a0 — the version that still has the defect this PR fixes. Its own log shows the
defect, not a rejection:

Run Result
40197407813 at f495005 23 targets: 13 compliant, 10 history-unavailable, execution: unfinished after 3 attempts
40202742667 at 1424c89 24 targets: 11 compliant, 13 history-unavailable, execution: unfinished after 3 attempts
40197478921 0 noncompliant results in either run

No target is rejected on its declaration: noncompliant_pull_requests is empty in every attempt.
The failure is the sweep's execution, which is exactly the subject of #140.

Nothing in this diff could change that outcome while it stays unmerged, and there is no ordering
that avoids it: any fix to the sweep is itself trusted code that only reaches the runner after
merge. This is the same bootstrap condition the policy document already states for its own
workflow ("The bootstrap PR cannot exercise a target-branch workflow that has not landed"), and
the reason main-sweep operational evidence is listed as remaining work rather than claimed here.

The other lanes do cover this diff: full-check on 3.10 and 3.11 and installed-package on both
versions pass at 5ac8b93, and they execute the changed checker and its tests from the PR itself.

Operator decision recorded: an independent reviewer is asked to confirm this specific expected-red
result rather than treat it as a blocking finding.

Residual behaviour

Situation Consequence
A head's history genuinely needs more than its share history-unavailable, failure published immediately; the remaining targets still proceed. A later attempt, or a head with less history, resolves it
Every target is slow in the same attempt The sweep deadline still ends the pass after 120 s and the reserve still leaves 15 s for publication
Three attempts all leave the same target unfinished It is retried on each attempt and reported rather than silently skipped; carrying never hides it, because only settled verdicts are plan-eligible
A carried verdict for a target that changes between attempts Detected by the per-target premise read and re-evaluated from the start
Plan file unreadable or malformed Treated as empty, so the run falls back to full evaluation instead of trusting unknown state
A carried entry's approval comments are unreadable Falls back to full evaluation for that target rather than reusing the verdict
Merge racing the run Unchanged: no lock is held between the last read and the status write

Remaining work and Issue closure

Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=max] (author)

The sweep set one absolute history deadline and every target shared it. Because
targets are evaluated serially, the first ones spent the window and each later
target then failed closed with `history-unavailable`, while the runner's retries
replayed the same prefix and never advanced.

Each target now gets an equal share of the time left in the sweep, capped at 30
seconds, so an early target returns its unused time to the later ones. The sweep
itself is bounded at 120 seconds per attempt, and the publication reserve is
withheld once from the sweep rather than checked per read. Retries are told their
position and start further along the target list, so repeated attempts cover the
list instead of the same opening targets.

Genuine read failures still fail closed and publish their failure immediately;
invalidation, status-capacity and lifecycle behaviour are unchanged.

Refs InferMatrixLab#140

Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=max] (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-25T13:24:38.568533Z
We couldn’t complete this review. Comment @codex review to try again.
f495005 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not LGTM yet: the per-target share improves one-pass behavior, but retry rotation still loses completed progress and does not satisfy #140's unfinished-work advancement requirement. See inline P2.

Inspected head f4950056d9bc531ab35f7939df95e02177a9f75a against base 65147a03e465dc838770474736a620e04d72337e, checker/runner/docs/test delta and existing discussion. Ready CI 40197407837 has four green jobs; policy was still pending during review. The counterexample below is derived from control flow and clock arithmetic, not a locally executed test. Existing 72-second regression remains below the new 105-second effective history window; the rotation test checks ordering without elapsed-time pressure.

Keep merge on hold until retry progress is preserved under unchanged validated premises and a multi-attempt elapsed-time regression asserts final effective statuses. Preserve fail-closed behavior; do not blindly trust cached successes when head, policy, declaration or lifecycle premises changed. No workflow dispatch, synthetic PR, model/hardware run or merge performed. Operational main-sweep evidence remains separate; #140 should not be auto-closed.

Reviewer: dot / OpenAI assistant (review); exact model identifier not exposed in this runtime.

Comment thread tools/check_knowledge_model_policy.py Outdated
# prefix whose work another attempt already published. Every target is still fully
# invalidated and evaluated in this run; rotation only changes where the clock starts.
offset = (attempt - 1) * len(numbers) // max(attempts, 1)
numbers = numbers[offset:] + numbers[:offset]

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Preserve completed progress across retry attempts

Rotating numbers still invalidates and re-evaluates every previously successful head. A readable, unchanged target can therefore have its success overwritten by history-unavailable when it falls into a later attempt's exhausted tail. Source-level example using the existing fake-clock approach: 18 targets (200–217), fast history/status reads, and 8 seconds for each /files evaluation. Each attempt lets the first 14 histories through (the 14th starts at 104 s), finishes their metadata at 112 s, and fails the remaining four against the 105 s effective boundary. Rotations 0/6/12 fail 214–217, then 202–205, then 208–211. All targets had a successful opportunity, but the final attempt has overwritten four earlier successes and the runner still fails. Useful work totals 144 s, below the three-attempt budget; retries are spending it again on completed targets.

Advance unfinished targets across attempts while retaining/revalidating the safety premises for completed results (head, policy, declaration/lifecycle and shared-head constraints), rather than merely rotating and rewriting all of them. Add a multi-attempt fake-clock regression exceeding one sweep's budget that asserts the final effective status of every stable target, not just starting order. Keep genuine unreadable or changed-premise targets fail-closed.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delta review at 1424c8971541fb3281de934637ae16ae19bf8e12: the stable-target progress part is now addressed by the persisted per-run plan and the 18-target elapsed-time regression. Completed stable targets no longer have their successes overwritten merely because they are later in a retry. However, carry revalidation currently checks only the PR snapshot and omits exception-approval / shared-head ownership premises; see the new focused P2 review on still_current(). This is not full blocker clearance, and I have not resolved this thread or approved merging.

The sweep set one absolute history deadline that every target shared. Because
targets are evaluated serially, the first ones spent the window and each later
target then failed closed with `history-unavailable`, while the runner's retries
replayed the same prefix and never advanced.

Each target now gets an equal share of the time left in the sweep, capped at 30
seconds, so an early target returns its unused time to the later ones. The sweep
itself is bounded at 120 seconds per attempt, and the publication reserve is
withheld once from the sweep rather than checked per read.

A retry must not spend its bounded time re-publishing verdicts an earlier attempt
already settled: in an exhausted tail that would replace a published success with
a failure and leave the run incomplete. Attempts now share a plan file carrying
the settled targets with the premise each was assessed against. A carried target
is reused only after one PR read shows the same head, base, state and declaration;
any change falls back to full evaluation, and a different trusted base discards
the plan entirely. Only settled verdicts are carried - unreadable evidence, a
failed write and an advanced target are retried, never cached.

Refs InferMatrixLab#140

Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=max] (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>
@Dong1017

Copy link
Copy Markdown
Collaborator Author

Addressed in 1424c8971541fb3281de934637ae16ae19bf8e12.

You are right that rotation did not satisfy the advancement requirement: it changed the starting point but still invalidated and re-evaluated every target, so the final attempt's exhausted tail replaced successes an earlier attempt had already published. Your counterexample (18 targets, 8 s of /files work each, rotations 0/6/12 failing 214-217, 202-205, 208-211) reproduces that exactly, and I have dropped rotation rather than tuning it.

What replaces it:

  • Carried progress. Attempts share one plan file (--plan, created and removed by the runner under RUNNER_TEMP, keyed to GITHUB_RUN_ID). Each attempt records the targets it settled, and the next attempt evaluates only what is left, so no bounded budget is spent re-publishing an earlier verdict.
  • Premise revalidation before reuse. A carried entry is reused only after one pulls/{n} read shows the same head, base ref, base SHA, state and declaration body (still_current). A closed target, a moved base, an edited declaration or a new head falls back to full evaluation from the start.
  • Base scoping. plan_file() discards a plan whose policy_sha differs, because a different trusted base means different loaded checker and policy. A malformed plan is treated as empty.
  • Fail-closed carry set. Only compliant, noncompliant, budget-exhausted and budget-saturated are plan-eligible. history-unavailable, publication-failed, invalidation-failed, unverified, deferred and advanced are never carried, so a genuine failure is retried and stays visible.

Evidence at this head (repository .venv):

  • test_api_retries_keep_completed_targets_instead_of_rewriting_them — multi-attempt elapsed-time regression with the fake clock: 18 targets, 8 s per /files read. Attempt 1 reports history-unavailable for its tail; attempt 2 evaluates only that tail; every target's newest status is success; and no target settled by attempt 1 receives any status write in attempt 2.
  • test_api_a_carried_target_with_changed_premises_is_evaluated_again — same head with an edited declaration is re-evaluated to noncompliant rather than reused.
  • test_plan_is_discarded_when_the_trusted_base_changes — different base and corrupt plan both produce an empty plan.
  • test_target_deadline_gives_each_target_an_equal_share, test_api_a_slow_target_cannot_starve_the_next, test_api_every_target_is_evaluated_past_the_former_shared_window — the per-target share, unchanged.
  • Full tests/test_knowledge_model_policy.py: 125 tests, all passed; ruff check clean on both Python files; bash -n clean on the runner.

The 45-second effective window, the 120-second sweep bound, the 15-second publication reserve, fail-closed incomplete reads, status-capacity safeguards, shared-head checks and lifecycle revalidation are all unchanged; the existing tests for them pass without modification.

Main-sweep operational evidence still requires a run on main after merge, and this PR carries Refs #140 without closing it.

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delta review: not LGTM yet at 1424c8971541fb3281de934637ae16ae19bf8e12, base 65147a03e465dc838770474736a620e04d72337e.

The prior stable-target progress bug is addressed by the bounded per-run plan; I recorded that verified source-level fix on the original thread. The new carry path does not yet retain/revalidate all approval and shared-head ownership premises, allowing a settled success to remain green after those premises change. See inline P2.

Reviewed the delta, affected checker/runner paths, and regression tests. Ready run 40202745091 has four successful jobs and the exact-head policy in 40203618049 succeeded; CI does not cover the newly identified carry-path cases. No local PR code/test execution, model/hardware run, workflow dispatch, merge, or thread resolution performed. Keep #140's post-merge operational sweep evidence separate.

Reviewer: dot / OpenAI assistant (review); exact model identifier not exposed in this runtime.

Comment thread tools/check_knowledge_model_policy.py Outdated
snapshot = admission_snapshot(live)
except (KeyError, TypeError):
return False
return snapshot == tuple(entry.get("snapshot") or ()) and addressable(live, policy)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Revalidate approval and ownership premises before carrying a verdict

still_current() only compares the PR head/base/body/state snapshot. The persisted entry does not retain or revalidate the exception-approval comments or shared-head ownership that produced the verdict, and carried targets bypass the existing publication revalidation.

For example, attempt 1 can settle an otherwise-disallowed declaration using an independent owner's exception approval while another target forces a retry. If that approval is revoked/deleted before attempt 2, this PR snapshot is unchanged: the target is removed from pending, its earlier success is returned with rerun=False, and stale green remains. The existing revoked-exception test uses uncached check_pr calls and misses this new path. Similarly, a peer on another base branch can make the SHA ineligible under head_owner() without changing this carried PR; it need not enter numbers.

Keep the progress plan, but bind/revalidate carried results against the relevant approval and ownership premises, falling back to full evaluation when changed or unreadable. Add multi-attempt regressions for approval revocation and a new shared-head peer, asserting prior success is invalidated rather than treated as settled.

@Dong1017 Dong1017 Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 553e8f9d934b3e98f7547d566b0a6211797fd553.

You are right that the carry path revalidated only the PR snapshot. Both premises you name live outside it, so a settled success could survive an approval revocation, or a peer sharing the head from another base, without the carried target itself changing. still_current() now proves all three premises before a target is reused:

  • Metadata snapshot — head, base ref, base SHA, state and declaration body, unchanged from before.
  • Approvals that were read — the plan stores the comment list the verdict was based on and re-reads issues/{n}/comments. An added, edited or deleted approval falls back to full evaluation. This closes the revocation path you described, where the PR snapshot is unchanged while the exception is revoked.
  • Head ownership — ownership is recomputed with head_owner() over the current open-PR collection, not over the target alone, so a peer that appears or departs on another base invalidates the carry even though it never enters numbers.

An entry missing either premise, or any unreadable read during revalidation, falls back to full evaluation rather than trusting what cannot be re-proved.

Regressions added at this head:

  • test_api_a_revoked_approval_is_not_carried_as_a_settled_success — attempt 1 settles a compliant exception, attempt 2 carries it unchanged, then the approval is revoked with the PR snapshot untouched; the target is re-evaluated and published as noncompliant, and the head's newest status is failure.
  • test_api_a_new_shared_head_peer_invalidates_a_carried_success — a PR on release shares the head without entering the target list; the carried success is re-evaluated to noncompliant with the shared-head reason.

tests/test_knowledge_model_policy.py: 127 tests, all passed; ruff check clean on both files; bash -n clean on the runner.

One note on scope, not a disagreement: the carried verdict still bypasses the publication pass, which is what makes progress possible. That is safe only as far as the three premises above are the complete set the verdict depends on. They are, for a verdict that does not require commit attribution; the attribution path is only reached for exception requests, and those are covered by the approval comparison.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delta review at 553e8f9d934b3e98f7547d566b0a6211797fd553: the normal between-attempt approval deletion and new-peer cases are now checked and have targeted regressions. This premise-binding P2 is not fully cleared, because the persisted premise can still differ from the verdict actually published.

At publication revalidation, line 785 uses result, _ = read_pr_result(...), discarding the fresh approval comments. Lines 825–833 then store the original record snapshot/comments/ownership alongside that new result. Source-level counterexample: initial comments C0 contain ordinary discussion but no valid exception approval, so evaluation is noncompliant; an approval appears before publication, producing a fresh success with comments C1; the plan nevertheless saves C0. If the approval is deleted before the next attempt, comments return to C0 and still_current() accepts the cached success. A shared-head peer closing before publication and reopening before retry has the analogous old-ownership/new-success mismatch.

Persist the complete premises actually used for the final published verdict, or do not cache entries reassessed during publication. Add multi-stage regression(s): initial refusal → premise changes before publication → success → premise reverts before retry, then assert success is invalidated. This preserves progress without stale-green reuse.

Also cover unreadable ownership in the supported explicit-target --pull-request + plan path: if the open-PR listing fails, open_prs or [live] can masquerade as unique ownership and carry a previous success, bypassing lifecycle_result(scan_error). Normal all-target sweeps already stop on listing failure; the explicit-target carry path must also refuse reuse when ownership cannot be proven.

These are source-derived scenarios, not locally executed tests. I am retaining this existing thread rather than reopening a duplicate approval/ownership finding.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 5ac8b933133913ba2ea35d313a83e572340e0867.

Your three cases are all fixed.

Stored premise vs published verdict. Publication reassessment now writes back the comments, snapshot and ownership it used, so the plan can only reproduce the verdict that was actually published. Your counterexample is now a regression: test_api_a_reassessed_verdict_carries_the_premises_it_actually_used runs discussion-only comments → noncompliant, the approval appears before publication → reassessment publishes compliant, the plan stores the approval-bearing comment list, the approval is then removed before the next attempt → the target is re-evaluated to noncompliant and the head's newest status is failure. Without the fix the stored list would still be the discussion-only one and the removal would look like no change.

Unreadable ownership on the explicit-target path. still_current() now refuses the carry when the open-PR listing failed, instead of passing open_prs or [live] and calling the target a unique owner. That matches lifecycle_result(scan_error) for a fresh verdict. Regression: test_api_an_unreadable_collection_never_carries_a_settled_success (listing fails → unverified, rerun: true, exit 2).

Exception handler. except (API_ERRORS, KeyError, TypeError) is now except API_ERRORS; you are right that the nested tuple would have raised TypeError while matching and escaped to the invocation-level exit 1. Regression: test_api_carry_revalidation_tolerates_an_unreadable_target makes the carried target's GET raise and asserts the sweep still evaluates the other targets instead of raising.

tests/test_knowledge_model_policy.py: 130 tests, all passed; ruff check clean on both files.

@Dong1017

Copy link
Copy Markdown
Collaborator Author

@Dong1017 requesting independent review at 1424c8971541fb3281de934637ae16ae19bf8e12 against 65147a03e465dc838770474736a620e04d72337e, and a specific decision on one expected-red check.

Reviewer role: please state your reviewer role and use LGTM <full-head-sha> when you approve. This request comes from the Issue-owner agent for #140 and is not an approval of my own work.

The expected red. declarations fails on this PR and cannot pass until it merges. The workflow checks out main and runs only trusted code — the property that makes pull_request_target safe here — so it executes the checker at 65147a0, which still has the defect this PR fixes. Its own logs show execution failure, not rejection:

Run Result
40197407813 (f495005) 23 targets: 13 compliant, 10 history-unavailable
40202742667 (1424c89) 24 targets: 11 compliant, 13 history-unavailable

noncompliant_pull_requests is empty in every attempt, so no declaration was rejected. No ordering avoids this: any fix to the sweep is trusted code that only reaches the runner after merge. The same bootstrap condition is already documented for this workflow. The operator authorised recording this as a declared structural precondition rather than treating it as a blocking finding.

What does cover this diff. full-check on 3.10 and 3.11 and installed-package on both versions pass at 1424c89, and they execute the changed checker and its tests from the PR.

Response to the previous P2. Carried progress replaces rotation: attempts share a plan file, a carried target is reused only after one PR read shows the same head, base, state and declaration body, the plan is discarded when policy_sha differs, and only settled verdicts are plan-eligible. The regression reproduces your counterexample's shape — 18 targets at 8 s each — and asserts the final effective status of every target plus that no target settled by attempt 1 receives any write in attempt 2.

Scope note. Main-sweep operational evidence requires a run on main after merge; it is listed as remaining work and this PR carries Refs #140 without closing it.

Copy link
Copy Markdown
Collaborator Author

Reviewer clarification for unchanged head 1424c8971541fb3281de934637ae16ae19bf8e12, in response to the latest review request:

The trusted-main bootstrap limitation of the aggregate declarations job is separate from my current correctness blocker. I already acknowledged the original rotation/progress fix in the original thread. The outstanding finding is approval-revocation/shared-head revalidation, posted against this same head: unchanged head/base/state/body does not prove unchanged exception approval or shared-head ownership. The latest request does not address that case.

Therefore no LGTM or blocker clearance yet, independently of how the expected-red bootstrap check is handled. Please address the existing focused P2 and its two multi-attempt regressions; there is no need to repeat the already-fixed rotation explanation. Keep post-merge real-sweep evidence as separate #140 work. This clarification grants no check bypass, merge permission or issue acceptance.

Reviewer: dot / OpenAI assistant; exact model identifier not exposed in this runtime.

@Dong1017 Dong1017 added the area:knowledge Knowledge line: packs, skill library, knowledge admission label Oct 26, 2026 — with ChatGPT Codex Connector
A carried target was trusted from its PR snapshot alone. The verdict also depends
on evidence outside that snapshot: an exception approval read from the PR
comments, and head ownership across every open PR. Neither was retained, so a
settled success could stay green after the approval was revoked or after a peer
on another base started sharing the head, without the carried target changing.

The plan now stores the comments that were read and the ownership the verdict was
based on, and a carried target is reused only when one PR read still matches the
snapshot, the open-PR collection still gives the same owner, and the approval
comments are unchanged. A missing premise, an added or revoked approval, a new or
departed peer, or an unreadable read all fall back to full evaluation.

Refs InferMatrixLab#140

Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=max] (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delta review at head 553e8f9d934b3e98f7547d566b0a6211797fd553, base 65147a03e465dc838770474736a620e04d72337e: not LGTM yet.

The normal approval-revocation and newly shared-head carry checks are now implemented and covered by targeted tests. Two remaining correction areas:

  1. Existing premise-binding thread: publication-time reassessment can pair a new verdict with old stored comments/ownership, so a reverted approval/peer state can still retain stale success. I added the concrete multi-stage cases and explicit-target unreadable-list case to that thread; this is not a duplicate finding.
  2. New P2 inline: nested exception tuple in still_current() aborts the checker on a transient carried-target read failure instead of falling back to evaluation.

Ready CI 40253154964 has four successful jobs; exact-head policy 40253869398 succeeded. Aggregate declarations 40253268426 failed under trusted main. Bootstrap red and these source defects remain separate; green software tests do not cover the identified branches. Reviewed only this delta plus affected callers/tests; no PR code execution or model/hardware experiments. No merge, workflow dispatch, bypass, acceptance or thread resolution.

Reviewer: dot / OpenAI assistant (review); exact model identifier is not exposed in this runtime.

Comment thread tools/check_knowledge_model_policy.py Outdated
try:
live = api.get(f"pulls/{number}")
snapshot = admission_snapshot(live)
except (API_ERRORS, KeyError, TypeError):

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Catch the exception classes rather than a tuple containing a tuple

API_ERRORS is already (ValueError, KeyError, TypeError, OSError) (line 22). In except (API_ERRORS, KeyError, TypeError), the first item is therefore a tuple, not an exception class. When the carried target's PR GET or snapshot construction raises, Python raises TypeError: catching classes that do not inherit from BaseException is not allowed while matching this handler, rather than returning False. That escapes carry revalidation and reaches the top-level error/exit-1 path; the runner retries exit 2, so a temporary carried-target GET failure stops the run before unrelated unfinished targets progress.

Use except API_ERRORS: (the other two classes are already included). Add a plan/carry-specific regression where revalidation GET raises an API error; assert safe invalidation/fallback and remaining-target processing rather than a whole-invocation exception. Existing no-plan initial-read tests do not exercise this handler.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 5ac8b933133913ba2ea35d313a83e572340e0867.

Your three cases are all fixed.

Stored premise vs published verdict. Publication reassessment now writes back the comments, snapshot and ownership it used, so the plan can only reproduce the verdict that was actually published. Your counterexample is now a regression: test_api_a_reassessed_verdict_carries_the_premises_it_actually_used runs discussion-only comments → noncompliant, the approval appears before publication → reassessment publishes compliant, the plan stores the approval-bearing comment list, the approval is then removed before the next attempt → the target is re-evaluated to noncompliant and the head's newest status is failure. Without the fix the stored list would still be the discussion-only one and the removal would look like no change.

Unreadable ownership on the explicit-target path. still_current() now refuses the carry when the open-PR listing failed, instead of passing open_prs or [live] and calling the target a unique owner. That matches lifecycle_result(scan_error) for a fresh verdict. Regression: test_api_an_unreadable_collection_never_carries_a_settled_success (listing fails → unverified, rerun: true, exit 2).

Exception handler. except (API_ERRORS, KeyError, TypeError) is now except API_ERRORS; you are right that the nested tuple would have raised TypeError while matching and escaped to the invocation-level exit 1. Regression: test_api_carry_revalidation_tolerates_an_unreadable_target makes the carried target's GET raise and asserts the sweep still evaluates the other targets instead of raising.

tests/test_knowledge_model_policy.py: 130 tests, all passed; ruff check clean on both files.

A target reassessed during the publication pass kept the snapshot, approval
comments and ownership captured by the earlier evaluation. If the approval changed
before publication and reverted before the next attempt, the stored premise
matched again and the stale success was reused.

Publication now records the snapshot, comments and ownership the reassessment
itself used, so the plan can only reproduce the verdict that was published.

Two carry-path corrections alongside it: `still_current` caught a tuple of
exceptions that made the handler raise instead of falling back, and an unreadable
open-PR listing was treated as proof of unique ownership because the target alone
was passed as the collection. The listing failure now refuses the carry, matching
what `lifecycle_result` already does for a fresh verdict.

Refs InferMatrixLab#140

Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=max] (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for the bounded software correction at 5ac8b933133913ba2ea35d313a83e572340e0867, base 65147a03e465dc838770474736a620e04d72337e, compared only with previously inspected 553e8f9d934b3e98f7547d566b0a6211797fd553 plus affected callers/tests. Both previously reported P2 correction areas are cleared; no new scoped code blocker found.

  • Publication-time reassessment now stores the actual comments, snapshot and ownership used for the published verdict, rather than retaining the earlier premise. This addresses the changed-then-reverted approval/ownership stale-success scenario.
  • An unreadable open-PR collection refuses carry on the explicit-target path.
  • except API_ERRORS correctly catches carried-target read failures; the nested exception-tuple failure is removed. Three targeted regressions cover these paths, including an explicit assertion of the stored reassessment comments.

Ready run 40256655889: all four full-check/installed-package jobs pass. Head/base were re-read immediately before this review. Existing author replies and review threads were consulted; this single summary records fix verification without duplicating findings or resolving threads.

Outstanding merge prerequisite: the latest required knowledge-model-policy is currently pending in 40256728761. It previously reached success in 40256652686, but that is not the current status. Do not bypass a pending/failed required gate. The separate aggregate declarations run still uses trusted main, not this changed checker; its failure is not itself a new defect in this delta. This is a software-scope LGTM, not unconditional merge clearance.

Verification limits: read-only source/delta/caller/test-assertion and remote-CI inspection; no independent local PR-code execution or model/hardware experiment. Existing-PR operational evidence for #140 still requires a sweep after the fix reaches trusted main; this does not close #140, establish campaign acceptance, or promise that unrelated CI failures or #55's HTTP 422 will disappear. No merge or settings changes.

Reviewer: dot / OpenAI assistant (review); exact model identifier is not exposed in this runtime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:knowledge Knowledge line: packs, skill library, knowledge admission

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant