Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
1007bdd
feat: add recurrent KV finalization source guidance
Dong1017 Oct 23, 2026
5948322
chore: generate recurrent KV knowledge projection
Dong1017 Oct 23, 2026
6475bae
fix: bound recurrent KV source ledger to actual lines
Dong1017 Oct 23, 2026
863f680
chore: regenerate recurrent KV projection after ledger fix
Dong1017 Oct 23, 2026
5cc79d4
Merge main into recurrent KV knowledge contribution
Dong1017 Oct 24, 2026
cfe6ed0
chore: regenerate recurrent KV knowledge from combined sources
Dong1017 Oct 24, 2026
fc0fda5
Merge current main into recurrent KV knowledge contribution
Dong1017 Oct 24, 2026
04ca1cd
Regenerate shared knowledge from combined source
Dong1017 Oct 24, 2026
876e45a
Merge current main into recurrent KV knowledge source
Dong1017 Oct 26, 2026
2971186
Regenerate recurrent KV knowledge from merged canonical source
Dong1017 Oct 26, 2026
0115f1b
Reconcile PR #121 with current canonical main
Dong1017 Oct 26, 2026
3254e9a
Regenerate PR #121 knowledge from combined source
Dong1017 Oct 26, 2026
e9d26f1
Reconcile recurrent KV candidate with current sources
Dong1017 Oct 27, 2026
e449774
Regenerate knowledge from the merged recurrent KV source
Dong1017 Oct 27, 2026
51660a7
Normalize default recurrent KV reference metadata
Dong1017 Oct 27, 2026
73cf73e
Regenerate knowledge after reference metadata normalization
Dong1017 Oct 27, 2026
d8a54eb
Merge current main into recurrent KV knowledge
Dong1017 Oct 28, 2026
400288b
Regenerate recurrent KV knowledge from merged canonical source
Dong1017 Oct 28, 2026
13b5651
Merge current main while retaining verified skill discovery
Dong1017 Oct 28, 2026
8819925
Refresh recurrent KV skill coverage from merged sources
Dong1017 Oct 28, 2026
1bdae0b
Correct the combined local profile inventory count
Dong1017 Oct 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/knowledge-base.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ separate work, not an implemented capability of this knowledge base.
```text
src/inferrsi/data/skill_library/
README.md Library navigation
skill-index.json Stable IDs → categorized paths/backend (64 skills)
skill-index.json Stable IDs → categorized paths/backend (66 skills)
hardware/{ascend,cuda,host}/<id>/ Hardware/platform-specific complete skills
frameworks/<framework>/<id>/ Framework-owned complete skills
repositories/<repo>/ Engine entry, sub-skills and focused complete skills
Expand Down Expand Up @@ -171,18 +171,18 @@ Complete procedures, sidecars, references and cases are packaged, with 13 Ascend
repositories, methods or workflows. The 21 engine and research-router skills
use the same ownership layout under `repositories/<engine>/` and
`workflows/inference-engine-research/`, including their sub-skills and references.
The library contains 94 `SKILL.md` files, all in English.
The library contains 96 `SKILL.md` files, all in English.
The collection overview, import notes and source manifest belong to the research
workflow's references; Strata design-space analysis belongs to Strata. The library
root contains only navigation and the registry.

The [generated catalog](../src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md)
projects the 64 registered skills into the contribution catalog. Its records
projects the 66 registered skills into the contribution catalog. Its records
use `inferrsi.<backend>.skills.<name_with_underscores>` IDs, with `ascend` and
`cuda` for scopes that depend on those backends and `common` for the remaining skills.
Each `skill_documents.original` points to the full packaged procedure. The
`skills` CLI retains the original hyphenated skill IDs and defaults to this
64-ID registry; engine procedures remain available through library navigation
66-ID registry; engine procedures remain available through library navigation
and catalog document pointers.

The registered [h3.c runtime audit](../src/inferrsi/data/skill_library/repositories/h3-c/audit-h3-runtime-residency/SKILL.md)
Expand Down
2 changes: 1 addition & 1 deletion docs/reference-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ Reading rules:
| **C4 / NVIDIA Resiliency Extension** @ `49af0b03f14550d36a60dcfebd573848b5fd8435` (Apache-2.0 source headers; Megatron-Core lineage) | local `base_manager.py` complete-iteration selection/save/finalizer and `local_manager.py` dirty/final file lifecycle | mined (contract only) — CUDA D2H synchronized before async request returns; no NPU qualification | mined — snapshot/replica lifetime constraints → `checkpoint-recovery-consistency` | mined — clean/dirty publication, complete-rank generation selection and incomplete-save rejection; no power-loss proof | mined — all-rank availability/finalization and replica failure domains, not transport throughput | out-of-scope |
| **CR1 / DeepSpeed ZeRO-Infinity** @ `5f266e26d216a705d9c64cac88fb813cefdbac4b` (Apache-2.0) | complete short `offload_config.py`, official ZeRO-3 live offload-field docs; no runtime/checkpoint audit | mined — parameter stage-3 and optimizer stage-1/2/3 boundaries → `training-resource-budget` | mined — CPU optimizer budget, pinned RAM, element-count residency and gated core share | mined — NVMe buffer pools and tiled read/compute/write overlap intent; buffer-size units unresolved in approved excerpt | out-of-scope — CPU/device transfer is not network coverage | mined — CPU resource-budget qualification; no privileged changes |
| **CR5 / Ascend MindSpeed adaptive memory** — `core_r0.8.0`, page history `53193db8` (full pin and license unresolved) | original GitCode historical `docs/features/adaptive-memory.md` body directly read; implementation unavailable | mined (docs-only case) — spare-NPU-memory versus recompute/swap decision → `training-resource-budget` historical case; changed-layer comparison rejected as controlled gain | mined (docs-only) — CPU swap and allocator/affinity prerequisites | out-of-scope | out-of-scope — NPU/CPU copies not network evidence | mined (historical conditions) — incompatible allocator features and CPU-affinity prerequisites; no current compatibility claim |
| InferMatrixLab/vllm-rlt (legacy ThinkFlowLab/vllm-rlt) @ `4a350344027960a48384626aa646f1765fda6d4f` (root Apache-2.0; model attribution in NOTICE) | Fixed-pin bounded depth-aware KV manager/runner/docs/test reads; [intake](reviews/issue91-vllm-rlt-intake.md); descriptor-only / intake-plan | located-not-mined — synchronous early-exit LAST_EXITED propagation versus SHARED and bookkeeping-only marking; no operating skill or runtime qualification | out-of-scope — device KV does not establish host-RAM coverage | out-of-scope — no storage mechanism extracted | out-of-scope — inspected KV copies are local tensor copies; no transport mechanism extracted | out-of-scope — no OS mechanism extracted |
| InferMatrixLab/vllm-rlt (legacy ThinkFlowLab/vllm-rlt) @ `4a350344027960a48384626aa646f1765fda6d4f` (root Apache-2.0; model attribution in NOTICE) | Bounded manager/runner/scheduler/Ouro/host-metadata/Torch-consumer chain; [candidate](../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md), [source map](../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md), [construction/cases](reviews/issue91-vllm-rlt-finalization.md); historical [intake](reviews/issue91-vllm-rlt-intake.md) retained | source-guidance — synchronous LAST_EXITED copying versus SHARED, written-prefix validity, later deeper attention and normal release/wait/free; candidate, native runtime and efficacy not_run | out-of-scope — host bookkeeping does not establish host-RAM coverage | out-of-scope — no storage mechanism extracted | out-of-scope — local tensor copies; no transport mechanism extracted | out-of-scope — no OS mechanism extracted |
| vllm-project/afd-plugin @ `df8d6a0c403c3ca1b4ed9cf1687e7204908bebec` (Apache-2.0 selected Python readset; retained MIT CAM subset notice) | Fixed-pin topology/factory/runner/metadata/DeepSeek dense proxy and P2pNccl reads; [full ledger](../src/inferrsi/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/references/source-notes.md); [historical intake](reviews/issue91-afd-plugin-intake.md) retained | mined (bounded source-only candidate) — role/world/subgroup rank ledger and eager dense-FFN per-stage/per-peer payload ownership → [audit-afd-p2p-exchanges](../src/inferrsi/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md); native/backend/model validation not_run | out-of-scope — no host-memory mechanism extracted | out-of-scope — no storage mechanism extracted | source call chain mined — actual NCCL-backed send/recv consumer identified; underlying completion, graph and multi-host/physical transport unqualified; no Network coverage credit | out-of-scope — no OS mechanism extracted |

Wave-8 implementation provenance (Dot bundles integrated 2026-10-15):
Expand Down
47 changes: 47 additions & 0 deletions docs/reviews/issue91-vllm-rlt-finalization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# vLLM-RLT synchronous KV finalization

Related to #91. This is one source-derived candidate capability, extending the descriptor-only [intake](issue91-vllm-rlt-intake.md) without replacing its historical evidence.

## Construction record

The source anchor is `InferMatrixLab/vllm-rlt@4a350344027960a48384626aa646f1765fda6d4f`, tree `4c2cab1f497513ea0590060eb53ee0b179a5bf70`. The approved scope is synchronous native Ouro with host `KVMetadata`: LAST_EXITED propagation versus SHARED, later attention consumption, and normal release/wait/free order. Asynchronous preemption/speculation, device-owned metadata, transfer/prefix ownership, capacity estimation and whole-engine extraction are excluded.

Portable AREX `distill-ml-knowledge` scope → ground → construct → verify informs the direct, task-agnostic source construction. The existing `extend-repo-skill` ownership check found no RLT operating owner; the separately approved `audit-recurrent-kv-finalization` capability is the only new node. InferRSI's existing typed metadata, canonical library, registry and readers are the packaging contract. No DisCo managed import, extra router, dependency installation or evaluation framework is introduced.

The grounded evidence is the intake's bounded manager/runner/test/license material plus the fixed caller, request, metadata, Ouro consumer and Torch dispatch tuple. The [source notes](../../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-notes.md) explain the call chain and exclusions; the [file ledger](../../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/references/source-identity.json) records exact Git blobs, SHA256, ranges and carried versus added acquisition. The original Apache-2.0 LICENSE and model/configuration NOTICE bytes are retained. No implementation or model weights are copied.

```mermaid
flowchart LR
A["Pinned manager, scheduler, runner, Ouro and metadata sources"] --> B["One candidate procedure + source ledger + notices"]
B --> C["Existing registry and library navigation"]
C --> D["skills / checkout reader: complete local resources"]
C --> E["Stock projection from committed canonical source"]
E --> F["catalog / package reader"]
```

The two rules are narrow: copy the final executed recurrent-layer K/V into skipped LAST_EXITED planes before a later deeper history read; retain the last completion dependency until runner release precedes cache-reference reclamation. The procedure preserves allocated-capacity, written-validity, copied-data and completed-work distinctions. SHARED is a single-plane control; bookkeeping-only `mark_finalized` is not a copying substitute. Backend placement is navigation, not qualification.

## Proposed representative and regression cases

These are candidate inputs and expected discriminators, not results. Native tests use the original package and real Torch tensors. An absent/new skill comparison can use the same questions, without changing the engine, inputs or expected source contract.

| Case | Positive / negative discriminator | Grounded input |
|---|---|---|
| Executed-plane regression | Preserve shallower/executed K/V; deeper propagation is a physical copy, not an alias | Upstream `test_last_exited_copies_every_layer_and_preserves_executed_depths`, 38–58 |
| Later deeper attention | Finalized earlier tokens remain readable as later tokens reach deeper planes across a block boundary | `test_early_exit_then_later_token_loops_deeper_across_block_boundary`, 61–77 |
| Premature finalize / prefix gap | Allocated capacity or a sparse later write must not make missing history valid | `test_validation_happens_before_mutation`, 97–107; `test_out_of_order_writes_only_expose_complete_prefix`, 116–127 |
| Bookkeeping sentinel | `mark_finalized` preserves deeper sentinels; `finalize_token` performs the copy | `test_mark_finalized_records_deeper_planes_without_copying`, 216–233 |
| SHARED control | One plane serves every depth; do not expect LAST_EXITED isolation or extra propagation | `test_shared_layout_exposes_one_plane`, 168–174; `test_mark_finalized_is_a_no_op_for_shared_layout`, 245–258 |
| Reused-page regression | A new request cannot consume stale old-request bytes merely because pages are reused | `test_block_reuse_never_exposes_stale_request_history`, 80–94 |
| Retirement investigation | Identify the latest event and release wait before scheduler finish/free; recording alone is insufficient | Runner 1331–1375; engine 626–640; scheduler 181–193 |
| Unsupported-path counterexample | Refuse to qualify async/device-owned metadata from host-prefix checks or synchronous call order | README 70–75; metadata 45–54; manager 716–727 |

All test names above are in the pinned upstream `tests/test_kv_cache.py`. Its manager tuple is CPU/float32/Torch attention, 2 layers, 1 KV head, head dimension 4, 18 blocks, block size 2 and 3 loops. The package initializer and test conftest require real Torch. The available construction environment lacks Torch and other declared CPU dependencies, so upstream import and native CPU tests are **not_run**; no classes, tensors or formulas were substituted. This is a concrete execution gap, not a correctness failure.

## Verification boundaries and handoff

Schema/frontmatter, registry navigation, source identity/ranges, local resource links, retained notices, checkout reads and deterministic catalog generation are content/package checks. Matching package reads and source hashes must be checked on the real generated child of the committed canonical source. They establish access and provenance, not RLT runtime correctness.

Native CPU/CUDA, model output equivalence, quality, performance and agent efficacy remain **not_run / not evaluated**. CPU cache tests cannot establish CUDA completion. The existing asynchronous-output/cleanup warnings remain unresolved. Candidate maturity and `empirical_validation=not_run` are preserved independently of software checks or merge status.

Author: Codex / gpt-6.1-sol / xhigh. Independent review, final source/generated commit identities and actual check receipts belong to the revision-scoped PR. Governing tracking Issue closure is not claimed.
2 changes: 1 addition & 1 deletion docs/searchspace-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ System-level: scheduler, DVFS, process isolation, environment variables, driver
| Distributed recovery qualification | RAM / SSD / Network / Verify | Low | C3/C4 contracts extracted; replica transport, whole-state completeness, world-size changes and actual failure recovery remain unvalidated |
| Cross-resource applicability and historical evidence | Accelerator / RAM / OS | Low | DeepSpeed buffer_size units/dispatch outside inspected config; MindSpeed full pin/license/implementation/current compatibility unresolved; CR6 is expert methodology, not measured gains |
| OS/System isolcpus and scheduler priorities (nice / ionice / SCHED_FIFO / RR) | OS / Optimize | Low | cgroup v2 controls are now sourced (`cgroup-resource-controls`); the scheduler-priority class remains without a source and deferred |
| vLLM-RLT recurrent-depth KV finalization | Accelerator / KV correctness / Verify | Intake only; not coverage | `vllm_rlt` remains descriptor-only; canonical owner corrected. Follow the [bounded early-exit intake](reviews/issue91-vllm-rlt-intake.md) before extraction; tensor copying, written-position bookkeeping and device completion remain distinct, with no runtime qualification |
| vLLM-RLT recurrent-depth KV finalization | Accelerator / KV correctness / Verify | Source-guidance candidate; native/efficacy not_run | [One operating candidate](../src/inferrsi/data/skill_library/repositories/vllm-rlt/audit-recurrent-kv-finalization/SKILL.md) traces synchronous host-KVMetadata exit/copy/validity → later depth-specific attention → normal release/wait/free. LAST_EXITED planes, SHARED single plane, bookkeeping and CUDA completion remain distinct; DeviceKVMetadata/async/speculation/capacity/transport excluded. [Intake](reviews/issue91-vllm-rlt-intake.md) retained |
| AFD P2pNccl completion, graph and transport qualification | Accelerator / Communication / Verify | Candidate audit delivered; native qualification gap remains | [audit-afd-p2p-exchanges](../src/inferrsi/data/skill_library/repositories/afd-plugin/audit-afd-p2p-exchanges/SKILL.md) covers rank and peer/stage ownership at its pin. The `afd_plugin` runtime descriptor remains unverified; underlying completion, DBO/graph and physical/multi-host transport are unqualified. Historical [intake](reviews/issue91-afd-plugin-intake.md) retained; no new Network coverage |

**Resolved gaps since initial matrix (all high/medium priority cleared):**
Expand Down
Loading
Loading