Skip to content

knowledge: add bottleneck-driven mechanism decision cases - #133

Closed
Dong1017 wants to merge 11 commits into
mainfrom
codex/issue131-foundational-decisions
Closed

Dong1017 wants to merge 11 commits into
mainfrom
codex/issue131-foundational-decisions

Conversation

@Dong1017

@Dong1017 Dong1017 commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Refs #131.

Retain the foundational mechanism decisions and original six curated cases. The existing six-column P3 matrix row is preserved. The earlier 300-second fixture execution timeout remains historical INCONCLUSIVE; the fixture was neither changed nor rerun. Source pins, readsets, licenses, candidate maturity and empirical not_run remain unchanged.

Reconcile the published head fad2e952ac7c8f2edbe991240fd0cd760c6578c7 with frozen canonical main a4a0fbd62defe2bc53ab090e1fa50bd7139663a6. Canonical source 732769fd2611551b23e92e22f5564c09069453e5 has exactly those two parents; generated child and current head 7b3e2e499975e24afabc0f8193bcf5c786477262 has the source commit as its sole parent. The repair preserves frozen-main content, all existing contribution content and matrix additions, including Hopper, MLX and Ascend material. Existing intake-to-candidate matrix replacements remain the original contribution's changes.

Unchanged stock tools/build_skill_index.py --source-revision 732769fd2611551b23e92e22f5564c09069453e5 and its matching --check passed. All 60 generated records bind 269 indexed hashes to committed source bytes. Native catalog, package, resource and bounded context reads passed against the explicit checkout. Other records retain current-main semantics apart from InferRSI provenance and applicable shared-registry hashes. These checks add no upstream runtime, backend/device/model, quality, performance or agent-efficacy result. Current-head independent review and clean installed-package Ready CI remain separate publication checks.

The later main advance 0c78a7e257dfaa3573b77581a39539e8a3a5da4c changes only Flare documentation, its validator and two test files. Those four paths overlap no registered knowledge input, maintained knowledge file, generator or native reader; the frozen source and checked projection remain applicable.

Current-head verification at 7b3e2e499975e24afabc0f8193bcf5c786477262: an independent gpt-6.1-sol/ultra review found no actionable issues. All four Ready jobs (Python 3.10/3.11 full checks and clean installed-package checks) and lint-and-smoke passed. The automatic evaluation-report publisher failed with HTTP 403 from the GitHub API; its execution job was skipped. Declaration compliance is a separate status in the current-head checks.

{
  "schema_version": 1,
  "head_sha": "7b3e2e499975e24afabc0f8193bcf5c786477262",
  "human_only": false,
  "material_ai_authors": [
    {
      "tool": "Codex",
      "model_id": "gpt-6.1-sol",
      "effort": "xhigh",
      "role": "author"
    },
    {
      "tool": "Codex",
      "model_id": "gpt-6.1-sol",
      "effort": "max",
      "role": "author"
    }
  ]
}

Consolidate six source-bounded scheduling, caching, and parallelism/pipelining cases in the existing optimization-search-policy owner. Test real discovery, resources, and revision contracts without claiming semantic selection or measured efficacy.

Refs #131. Related to #91.

Signed-off-by: GUOGUO <xwdong1998@163.com>
Agent-assisted-by: Codex / gpt-6.1-sol [effort=xhigh] (author)
Generate the shared knowledge projection with the native builder from source 672b4b5.

Refs #131. Related to #91.

Signed-off-by: GUOGUO <xwdong1998@163.com>
Agent-assisted-by: Codex / gpt-6.1-sol [effort=xhigh] (author)
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-26T08:49:07.011203Z 7b3e2e4 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for the bounded knowledge-consolidation scope, with one nonblocking table-format correction below. Inspected head 55dd7da0db83905d8b2cb205d505b5aae042ff80, base 28ddd06a48128c187f093763c7f52ffdfd04a5f0.

The six cases retain their carried source scopes: training-only Alpa limits, historical negative graph evidence, independent LMCache pin, and AFD eager dense-FFN boundaries. Existing owners are reused; the other 58 generated records only refresh their source pin. The seven fixtures test retrieval/resource/revision contracts, not semantic selection or abstention, so this does not grant the broader #131 acceptance.

All four Ready jobs and current policy status are green. Verification was static source/ledger/projection/test inspection and CI-status inspection; no independently run tests, generator, fresh primary-paper audit or model/hardware experiments.

Comment thread docs/searchspace-matrix.md Outdated
@Dong1017 Dong1017 added the area:knowledge Knowledge line: packs, skill library, knowledge admission label Oct 24, 2026 — with ChatGPT Codex Connector
Combine the evidence provenance cells so the existing maturity and unevaluated-selection qualification remains visible in the six-column table.

Signed-off-by: GUOGUO <xwdong1998@163.com>
Agent-assisted-by: Codex / gpt-6.1-sol [effort=xhigh] (author)

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM at 5349fa3359efd3ff083181fa94ef3623e7803433 for the previously reviewed knowledge-consolidation scope. The delta from 55dd7da0db83905d8b2cb205d505b5aae042ff80 is exactly the one-line table correction: six cells now match the header, provenance stays under Evidence and the complete candidate / selection-not-evaluated qualification is visible under Maturity. The prior nonblocking P3 is cleared; no new finding.

All four Ready jobs and current policy status succeed at this head. No source knowledge, projection, tests or runtime changed. This was static delta/table/CI-status verification, not independently executed tests or efficacy validation; broader #131 acceptance remains separate.

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Integration follow-up at unchanged head 5349fa3359efd3ff083181fa94ef3623e7803433: after #130 merged, main is e6bedfa73618af3bb59249924d0fb3201439c021 and GitHub now reports this PR as conflicted / not mergeable. The earlier scoped LGTM still describes the inspected content, but is not clearance of a resolved integration tree; existing green CI predates this main change.

Please reconcile with current main, preserving both sets of canonical guidance and navigation/matrix changes, then regenerate the combined knowledge projection with the existing generator rather than selecting one side's generated files. Refresh any head-bound declaration and obtain applicable CI on the resulting head. The follow-up review can be limited to that integration delta and affected contracts.

This is an integration prerequisite, not a new source-fidelity finding or an expanded efficacy/acceptance requirement. Read-only metadata review; no tests or experiments run.

Preserve the existing decision cases and current owner navigation before refreshing their shared source bindings.

Signed-off-by: GUOGUO <xwdong1998@163.com>
Agent-assisted-by: Codex / gpt-6.1-sol [effort=xhigh] (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>
Agent-assisted-by: Codex / gpt-6.1-sol [effort=xhigh] (author)
Preserve canonical source and validation boundaries.

Signed-off-by: GUOGUO <xwdong1998@163.com>
Agent-assisted-by: Codex / gpt-6.1-sol [effort=xhigh] (author)
Preserve canonical source and validation boundaries.

Signed-off-by: GUOGUO <xwdong1998@163.com>
Agent-assisted-by: Codex / gpt-6.1-sol [effort=xhigh] (author)

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inspected head a19c4043ccd699fb145e6d274a7ba5fe2a2347ba, reported base 65147a03e465dc838770474736a620e04d72337e.

LGTM for the scoped integration refresh. The previously reviewed root procedure, metadata, six-case guide and fixture are unchanged. Navigation/matrix insertions retain the corrected six-column candidate/selection-not-evaluated qualification, with current-main content preserved. Combined source e2d3a597038ae5a3dca35858b72276545eafdad1 → head is projection-only; the other 58 records differ only in canonical provenance. No actionable integration finding.

GitHub reports mergeable; all four Ready jobs and per-head policy status pass. The prior integration prerequisite is cleared for this head/base.

Static delta/projection/CI inspection, not independently executed tests or generator regeneration. Earlier incomplete local fixture execution remains inconclusive; retrieval contracts do not establish semantic selection, abstention or agent efficacy. #131 acceptance and merging remain separate.

The aggregate declaration workflow has failures elsewhere; individual policy success is not an all-workflows-green claim.

Preserve the foundational mechanism decisions with all current main knowledge increments.
Keep main generated projections for regeneration from this combined source commit.

Agent-assisted-by: Codex / gpt-6.1-sol [effort=xhigh] (author)
Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>
Refresh discovery records from the actual combined source revision.
Verify committed source hashes and actual package resource and context reads.

Agent-assisted-by: Codex / gpt-6.1-sol [effort=xhigh] (author)
Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scoped LGTM for the integration delta at fad2e952ac7c8f2edbe991240fd0cd760c6578c7, reported base 5b939a95d7478b1d8bff013c477972903b7de9ba, compared with previously inspected a19c4043ccd699fb145e6d274a7ba5fe2a2347ba. No new actionable source/integration finding.

All nine optimization-search-policy owner blobs and the six-case fixture are unchanged. The matrix retains six columns and candidate / mechanism-selection-not-evaluated qualifications. Current-main canonical content is preserved. Combined source 136c648 to head is projection-only; other 59 records change provenance only. The prior resolved P3 remains fixed.

Exact-head Ready CI has four successful full-check/installed-package jobs; lint also passes. Fresh per-head policy: success, source https://github.com/InferMatrixLab/InferRSI/actions/runs/40271581200. Aggregate declarations failures/cancellations are separate and are not claimed green. Head and mergeability were refreshed before posting (mergeable=true); recheck required gates and the resulting integration tree before any merge.

The historical fixture execution limitation remains INCONCLUSIVE; software/source checks do not establish semantic selection or task-efficacy acceptance.

Read-only incremental source/blob/projection/caller/test-assertion and existing discussion/CI inspection. No PR-code execution, generator rerun, model/GPU experiment, thread resolution or Issue acceptance. This records the new inspected head without reopening resolved findings or repeating the upstream audit.

Reviewer: dot / OpenAI assistant; exact runtime model identifier is not exposed.

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM within this PR's accepted source/software scope. Meaningful CI-blocker clearance on unchanged head fad2e952ac7c8f2edbe991240fd0cd760c6578c7: Ready CI and declarations rerun are successful; exact-head knowledge-model-policy is success. This updates the prior scoped review rather than reopening resolved findings.

Integration inspected against actual main db82ec8346364667b90f4fa97fdb190150141cf0, separately from the PR API's base snapshot 5b939a95d7478b1d8bff013c477972903b7de9ba. The intervening #154/#118 changes (18 paths) do not overlap this contribution. New evaluation-planner mapped canonical directories are not changed by this PR; its source delivery is not thereby covered by a real efficacy evaluation. No new actionable integration finding or unresolved review blocker was identified.

Verification limits: incremental source/integration/prior-thread and CI inspection, no new local tests or model/hardware execution. Existing source-fidelity/packaging conclusions retain their original limits; this is not goal/campaign acceptance or issue closure. Recheck exact head, applicable checks and integration immediately before any merge, especially after another knowledge PR changes shared matrices/projections. Historical red runs remain historical, not rewritten evidence.

Preserve both canonical knowledge histories before rebuilding generated records.

Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>
Build the native projection from canonical source 732769f.

Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for repaired source/package integration at 7b3e2e499975e24afabc0f8193bcf5c786477262, incrementally compared with fad2e952ac7c8f2edbe991240fd0cd760c6578c7.

No optimization-search-policy owner resources or six-case fixtures changed old→new. The original contribution and corrected six-column maturity qualification remain intact; the resolved P3 remains fixed. Source 732769fd… → head is generated-only; the other 59 records change provenance only. Root/metadata/guide hashes were independently recomputed against projections, and main #134/#142 additions remain preserved. Prior conflict blocker is cleared (mergeable=true). The historical fixture timeout remains INCONCLUSIVE; no new efficacy claim is inferred.

All four Ready jobs and current policy succeed. Separate Skill Eval publisher fails HTTP 403; execution skipped and this contribution is not covered by the maintained evaluation map.

Outstanding integration decision: #156 is an overlapping alternative #131 implementation, including the same guide path with a different case set. Select/reconcile intentionally; these scoped LGTMs do not recommend merging both independently.

Inspected actual main 0c78a7e257dfaa3573b77581a39539e8a3a5da4c; API base a4a0fbd62defe2bc53ab090e1fa50bd7139663a6 differs only in four disjoint Flare paths. Static/delta/link/hash and remote-CI verification only; no generator/test rerun, model/device experiment, merge or issue acceptance.

Copy link
Copy Markdown
Collaborator Author

Integration follow-up only: head 7b3e2e499975e24afabc0f8193bcf5c786477262 currently has merge conflicts (mergeable=false / dirty) against main 5c6a8f6af7ecd8fa71cedd31889820eb1546aa50. Please reconcile with main while preserving other contributors' changes, regenerate only derived artifacts required by the existing source-binding contract, and run the affected/package checks on the resulting head before requesting merge.

This is a new integration-state update after the earlier conflict-clearance review, not another source review or a request to start a duplicate exact-head review. Specific conflicting files have not been independently reproduced here; no file-level conflict attribution is asserted.

@Dong1017 Dong1017 added the skill-eval:execute Owner-authorized paid Skill Eval opt-in; cumulative USD 200 per PR. Not a quality or merge approval. label Oct 29, 2026 — with ChatGPT Codex Connector
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Skill evaluation

Retained baseline → candidate

Strict aggregate outcomes are unchanged by component explanations; unavailable is not failure.

optimization-search-policy / case-05 · repeat 1
Component changes: diagnosis: FAIL → PASS; safe_next_check: FAIL → PASS.

  • Baseline: FAIL; diagnosis=capacity; next_check=inspect_allocator_accounting; citations=E4; reuse_authorized=false; components: schema=PASS, diagnosis=FAIL, citation=FAIL, safe_next_check=FAIL, no_invariant_violation=PASS; Reasons: Diagnosis does not match the recorded observations. Next read-only check does not match the recorded observations. Extra/nondecisive citation present; strict citation check fails. Required decisive citation missing.
  • Candidate: FAIL; diagnosis=readiness; next_check=check_receive_completion; citations=E2, E4; reuse_authorized=false; components: schema=PASS, diagnosis=PASS, citation=FAIL, safe_next_check=PASS, no_invariant_violation=PASS; Reasons: Extra/nondecisive citation present; strict citation check fails.
    Coverage: one synthetic diagnostic case per pair; not a production trace, engine benchmark or general efficacy result.

Showing 1 of 1 bound pairs; discovery excluded. Uncovered/unsupported changes: 65; gaps never count as passes.

Automatic execution / cumulative PR budget: {"daily_day":"2026-10-29","daily_remaining_usd":"528.99","label":"skill-eval:execute","reason":"Plan has no supported pairs or provider/bounds are stale","remaining_usd":"37.28","reserved_or_unresolved_usd":"162.72","spent_or_legacy_usd":"0","state":"blocked"}

Advisory: obtain missing evidence

Retained evidence availability: available. No approval or merge authorization is issued.

  • Uncovered/unsupported scope needs a case or explicit reviewer disposition.
  • optimization-search-policy: existing failure remains; fix or explicitly disposition it.

RETAINED RESULTS

Outcomes: {"fix":0,"inconclusive":0,"missing_pair":0,"regression":0,"unchanged_correct":0,"unchanged_incorrect":1}
Discovery is reported separately in retained evidence.

Selected comparisons: 1 · Estimated task calls: 2.
Uncovered/unsupported changes: 65. Gaps never count as passes.

Applicable only while these revisions and inputs match the current PR.
Paid execution requires trusted label authorization and cumulative PR reservation. Unknown accounting blocks new paid calls; planning is not execution.
Pipeline evidence only; offline fixed outputs do not establish skill efficacy.
Evidence and logs · Retention: 30 days.
Outcomes are historical; verifiable only before Unix time 1795837888 while the linked artifact is available. Expired evidence is unverified.

Plan bindings, budget, frozen exam and advisory details

Unchanged-correct is valid neutral evidence, not incremental benefit.
Content/source review and authorship admission remain separate.
Trusted regrading checks retained answers, not model/hardware provenance or a hostile executor; this is not an efficacy certification.

Per-case outcomes (discovery excluded; regressions and gaps first):

[{"case":"case-05","outcome":"unchanged_incorrect","repeat":1,"skill":"optimization-search-policy"}]

Showing at most 20 of 1 pairs; the evidence link contains all outcomes, including omitted regressions/gaps.

Frozen comparison optimization-search-policy: 734f18126b5b57c72cb16ddce31d51aba99bf2b33dc475d68808839b40ed9248
Case case-05 answer SHA-256 (baseline / candidate): 721357bb9d5dc31d4d6a68a3578cc2964e3fa2f8e2393f25787e67d186eb1a0f / 3f16ddc0c54a0b1300e755d15b5af402ceedc387c20507beca9398b226bc1b49
Plan SHA-256: 5b110389eec2ac85f7fd5e2d4fb91e715a67f9e4c67906a412b706664bd3e16a
Base: a4a0fbd62defe2bc53ab090e1fa50bd7139663a6 · Head: 7b3e2e499975e24afabc0f8193bcf5c786477262
Trusted implementation: a0a76b33de93bc131af5849b519b58ae077aa6d2

Model/settings: {"effort":"not_applicable","name":"claude-opus-5-5"}
Budget: {"comparisons":1,"estimated_calls":2,"limits_per_comparison":{"campaign_seconds":600,"max_task_calls":2,"repeats":1,"timeout_seconds":180,"token_stop_new_trials":200000},"provider_config_sha256":"b28d80f8194fbe0f5a9f01515efb24b218979fd45f702776560da276f4b30d04","reservation":{"amount":"20.34","currency":"USD","group":"default","limit":"200","per_request":"10.17","pricing_sha256":"2a82453ec18b75b92e12f9a9c18e6b57811a3ef593c31e4639b839df218cac39","requests":2}}

Skill Comparison Cases Estimated calls
optimization-search-policy update case-05 (holdout) 2

Frozen exam hashes: {"cases":"a0dd4e1615402a3a629f925c15435470095c27eeac54c4a2080c2d874abeb031","grader":"e25cfee943052aaefbddf3f545fe6d56d95ab1edb9723cd9353feeabc954ea9a","schema":"46ed2e63dadd5f9fa60107ff58b020554b42f47472ca80b4de9add79d92f8449"}

Provider/settings: {"client":{"api_version":"2023-06-01","endpoint":"https://yibuapi.com/v1/messages","kind":"anthropic-messages-full-context-v1","max_output_tokens":4096,"max_read_bytes":32768,"max_request_bytes":262144,"max_requests":1,"max_tool_calls":12,"provider":"yibuapi"},"limits":{"token_stop_new_trials":200000},"model":{"effort":"not_applicable","name":"claude-opus-5-5"},"pricing":{"currency":"USD","expires_at":1793271228,"group":"default","max_input_tokens":1000000,"model":"claude-opus-5-5","rate_evidence":"https://github.com/InferMatrixLab/InferRSI/issues/215#issuecomment-6467531341","rates":{"cache_read":"0.4","cache_write":"10","input":"8","output":"40"},"request_fee":"0","unit":"per_million_tokens"},"spend_limit":{"amount":"200","currency":"USD"}}

Uncovered / unsupported paths (65)
  • uncovered: "docs/reference-matrix.md"
  • uncovered: "docs/searchspace-matrix.md"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-aclgraph-capture-replay.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-device-health.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-distributed-parallel.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-graph-mode.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-model-adaptation.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-profiling.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-tuning.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-optimization-casebook.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-performance-analysis.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascendc-tiling-design.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/cann-attribution-overlap.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-device-memory-lifecycle.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-pinned-host-memory-lifecycle.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/assess-serving-preemption.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/assess-split-backward-schedule.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-h3-runtime-residency.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-llama-runtime-state.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-sd-cpp-residency.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-sglang-diffusion-lifetimes.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-vllm-kv-lifetimes.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/bound-windowed-attention-replay.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/budget-speculative-verification.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/calibrate-serving-simulation.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/cgroup-resource-controls.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/checkpoint-recovery-consistency.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/choose-parallel-training-plan.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/cite-check-downstream-pr.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/claim-contract.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/comfy-ui.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/competitor-scan.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/dcp-async-save-performance.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/dense-performance-evidence.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/diagnose-vllm-admission-pressure.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/diffusers-invocation-state.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/draft-acceptance-issue.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/evaluate-elastic-serving-placement.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/generation-cache-lifecycle.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/host-memory-management.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/host-overhead-optimization.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/inference-benchmark-design.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/inference-optimization-casebook.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/kernel-writing-guide.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/kv-tier-management.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/lazy-evaluation-stream-lifetimes.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/npu-skill-factory.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/os-process-placement.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/performance-ceiling-analysis.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/plan-activation-residency.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/repository-evolution-analysis.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/rsi-evidence.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/skill-evolution-playbook.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/storage-io-measurement.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/training-input-pipeline.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/training-resource-budget.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/weight-loading-optimization.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/cuda/skills/audit-afd-p2p-exchanges.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/cuda/skills/diagnose-attention-bottlenecks.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/cuda/skills/host-overhead-profiling.json"
  • uncovered: "src/inferrsi/data/repo_packs/inferrsi/knowledge/cuda/skills/reason-hopper-kernels.json"
  • uncovered: "src/inferrsi/data/skill_library/README.md"
  • uncovered: "tests/test_foundational_mechanism_decisions.py"

Presentation implementation: 5a46ad5a9e5bb93ec8111d47d97c7e341c476710. Original execution plan and frozen verifier bindings are retained; this report-only refresh starts no model execution or budget reservation.

@Dong1017 Dong1017 removed the skill-eval:execute Owner-authorized paid Skill Eval opt-in; cumulative USD 200 per PR. Not a quality or merge approval. label Oct 29, 2026
@Dong1017 Dong1017 added the skill-eval:execute Owner-authorized paid Skill Eval opt-in; cumulative USD 200 per PR. Not a quality or merge approval. label Oct 29, 2026 — with ChatGPT Codex Connector
@Dong1017 Dong1017 removed the skill-eval:execute Owner-authorized paid Skill Eval opt-in; cumulative USD 200 per PR. Not a quality or merge approval. label Oct 29, 2026
@Dong1017 Dong1017 added the skill-eval:execute Owner-authorized paid Skill Eval opt-in; cumulative USD 200 per PR. Not a quality or merge approval. label Oct 29, 2026 — with ChatGPT Codex Connector
@Dong1017 Dong1017 removed the skill-eval:execute Owner-authorized paid Skill Eval opt-in; cumulative USD 200 per PR. Not a quality or merge approval. label Oct 29, 2026

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Live diagnostic evidence at7b3e2e499975e24afabc0f8193bcf5c786477262 versus basea4a0fbd62defe2bc53ab090e1fa50bd7139663a6: https://github.com/InferMatrixLab/InferRSI/actions/runs/40814444430 completed a valid old/new pair under the separately frozen full-context harness (trusted a0a76b3).

Case05 observations: E1 cache hit; E2 receive pending; E3 identity match; E4 allocator denied. Frozen policy prioritizes readiness and expects onlyE2 as decisive citation, check_receive_completion, and refusal of immediate reuse.

Actual old answer: capacity / [E4] / inspect_allocator_accounting / reuse=false.
Actual new answer: readiness / [E2,E4] / check_receive_completion / reuse=false.

Strict score remains FAIL→FAIL (unchanged_incorrect). New knowledge condition aligns diagnosis and next check with the frozen readiness-first policy, but extraE4 fails the citation contract. Both answers pass schema and reuse-safety assertions. This is decision-correct with citation-contract failure in the new condition, not an unsafe reuse decision; the frozen score is not rewritten.

Interpretation: pending receive and allocator denial can coexist. This synthetic policy test does not establish a unique physical root cause, general efficacy or that the C1 text alone caused the difference (the complete bundles differ). One pair and65 uncovered paths do not clear overall PR acceptance or existing integration prerequisites. No new production-benchmark requirement is imposed on this source-only contribution.

Independent trusted regrading, report equality, all frozen input/controller/resource hashes and full manifests verified. Two provider responses/submissions completed,104261 input and5912 output tokens. Conservative observed-usage tariff upper estimate USD1.279090 is not an invoice; fullUSD20.34 reservation retained. Earlier failed attempts are preserved under #235; no refunds or blind retries. Plan5b110389eec2ac85f7fd5e2d4fb91e715a67f9e4c67906a412b706664bd3e16a.

Dong1017 commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator Author

Closed without merge; evaluation case retained

The repository owner explicitly confirmed closure without merge on 2026-10-29. This PR was closed without merge at 09:37:51 UTC. Related #131 is closed, but its timeline provides no evidence-backed acceptance explanation; that state alone does not prove this PR's deliverables were completed. #156 was also closed without merge. No replacement delivery was verified, and the added references/foundational-mechanism-decisions.md is absent from current main a4ed530923fe59e9e5b0e9f44e1bf3fd303b84c5. This is therefore not a duplicate/superseded/completed-delivery claim. The branch and discussion remain available for future scoped reuse.

At this assessment, head is 7b3e2e499975e24afabc0f8193bcf5c786477262; integration conflicts remain. Earlier source/software LGTMs do not clear the current integration state.

Retain this as a SkillEval-assisted review case with partial improvement and an unresolved strict check, not an example of automatic efficacy rejection:

  • Actual retained bot report: baseline capacity/E4/allocator check → candidate readiness/E2+E4/receive check. Diagnosis and next-check components improve; the candidate still fails the frozen exact-citation rule. Both refuse immediate reuse.
  • Aggregate remains FAIL→FAIL; one synthetic case and 65 uncovered changes do not establish overall ineffectiveness, harm, or a unique causal contribution from one text change.
  • Independent evidence review records these limits. SkillEval was advisory, not a required automatic merge gate. New-execution admission being blocked is also distinct from rejecting this PR's content.

Closure is a human scope/prioritization decision informed by the available evidence. Do not relabel the historical result as a general negative efficacy result or change the frozen grading to manufacture a pass. Broader evaluation/report usability work continues in #253.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:knowledge Knowledge line: packs, skill library, knowledge admission

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant