diff --git a/docs/knowledge-base.md b/docs/knowledge-base.md index 3953514ad..77155c903 100644 --- a/docs/knowledge-base.md +++ b/docs/knowledge-base.md @@ -14,7 +14,7 @@ separate work, not an implemented capability of this knowledge base. ```text src/inferrsi/data/skill_library/ README.md Library navigation - skill-index.json Stable IDs → categorized paths/backend (55 skills) + skill-index.json Stable IDs → categorized paths/backend (57 skills) hardware/{ascend,cuda,host}// Hardware/platform-specific complete skills frameworks/// Framework-owned complete skills repositories// Engine entry, sub-skills and focused complete skills @@ -168,18 +168,18 @@ Complete procedures, sidecars, references and cases are packaged, with 13 Ascend repositories, methods or workflows. The 21 engine and research-router skills use the same ownership layout under `repositories//` and `workflows/inference-engine-research/`, including their sub-skills and references. -The library contains 76 `SKILL.md` files, all in English. +The library contains 78 `SKILL.md` files, all in English. The collection overview, import notes and source manifest belong to the research workflow's references; Strata design-space analysis belongs to Strata. The library root contains only navigation and the registry. The [generated catalog](../src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md) -projects the 55 registered skills into the contribution catalog. Its records +projects the 57 registered skills into the contribution catalog. Its records use `inferrsi..skills.` IDs, with `ascend` and `cuda` for those hardware-specific scopes and `common` for the remaining skills. Each `skill_documents.original` points to the full packaged procedure. The `skills` CLI retains the original hyphenated skill IDs and defaults to this -55-ID registry; engine procedures remain available through library navigation +57-ID registry; engine procedures remain available through library navigation and catalog document pointers. Use `inferrsi knowledge search --repo-type inferrsi --limit 100` in an ordinary diff --git a/docs/reference-matrix.md b/docs/reference-matrix.md index f1906b3b2..7023f20eb 100644 --- a/docs/reference-matrix.md +++ b/docs/reference-matrix.md @@ -335,3 +335,16 @@ The generated abstract is a projection, not independent corroboration. The existing Diffusers offload card retains its earlier source pin. This bounded invocation readset does not establish caller serialization, general thread safety or exception cleanup. The generated abstract adds discovery, not corroboration. + +## DeepInfer source extraction + +| Source / revision | Inspected scope | Reuse, extraction and limits | +|---|---|---| +| deepseek-ai/DeepInfer @ `8b16149741a4935e1dec8cea27853f273b374c5b` (Apache-2.0; direct vendors retain separate licenses) | Bounded DFlash producer/verification budget, scheduler confidence/round counters, worker ZOS metadata repair, YoCo boundary/full-layout KV, SWA metadata/ring sizing and released serving hooks; selected test/harness source | Two source-derived candidates: [verification budget](../src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md), [bounded query replay](../src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md); reuse [host scheduling case](../src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md) and [capability audit](../src/inferrsi/data/skill_library/methods/repository-evolution-analysis/references/capability-presence-audit.md). Their F labels mean pinned CODE, P means proposed procedure; test/harness reading is not execution. Native/model/device/sampling/quality/performance not_run. | + +The source-reading bundle preserves six direct vendor snapshots but omits nine +media assets and ten nested gitlinks. Parent-code observations do not certify +those dependencies, a build-complete checkout or a complete serving system. +The three peer cards summarize related source observations; generated abstracts +point to the complete procedures. Neither independently corroborates or promotes +the candidate mechanisms. diff --git a/docs/searchspace-matrix.md b/docs/searchspace-matrix.md index 16a8975b7..1e3ae3110 100644 --- a/docs/searchspace-matrix.md +++ b/docs/searchspace-matrix.md @@ -357,3 +357,17 @@ does not certify backend compatibility. No measured or green coverage is credited. This adds no SSD, Network or OS capability and does not retest the existing offload mechanism. `common` indexes framework guidance without certifying backend compatibility. + +## DeepInfer bounded source guidance + +| Resource | Stage / named mechanism | Canonical owner | Evidence boundary | +|---|---|---|---| +| Accelerator | Diagnose / Configure / Verify: configured-width DFlash drafts versus budgeted verification rows; lagged CPU totals and GPU ZOS per-request metadata correction | [budget-speculative-verification](../src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md) | Source constraints and cost objective, not sampling-distribution proof, dispatch qualification or measured gain | +| Accelerator / RAM / Host | Diagnose / Configure / Verify: retained query dependency closure, separately preserved full-layout KV, SWA valid-state bounds and nested ring-sizing gates | [bound-windowed-attention-replay](../src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md) | Model-specific boundary/rollback/cache contract; native equivalence and resource measurements unrun | +| RAM / Host | Observe / Diagnose / Verify: host work overlap, one-step retained state, repair and drain obligations | [Existing host-overhead owner case](../src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md) | Source-level schedule, no reproduced host-gap reduction or complete-service latency result | + +These rows add source-derived procedure guidance; native execution and measured +gains remain unverified. They do not add measured resource coverage: +local host scheduling and source references to collectives do not establish new +SSD, Network or OS-control coverage. Backend and production-service scope remain +separate from the public release's inspected code paths. diff --git a/src/inferrsi/catalog.py b/src/inferrsi/catalog.py index d27c9fe9a..82bc78c30 100644 --- a/src/inferrsi/catalog.py +++ b/src/inferrsi/catalog.py @@ -16,7 +16,9 @@ "dependency": ( "flashinfer", "flashattention", "pytorch", "triton", "diffusers", "transformers" ), - "peer": ("sglang", "tensorrt_llm", "lmdeploy", "xdit", "cache_dit", "qwen3_tts_engine"), + "peer": ( + "sglang", "tensorrt_llm", "lmdeploy", "xdit", "cache_dit", "qwen3_tts_engine", "deepinfer" + ), } KNOWLEDGE_REPOS = tuple( dict.fromkeys(repo for repos in KNOWLEDGE_SCOPES.values() for repo in repos) diff --git a/src/inferrsi/data/knowledge_packs/README.md b/src/inferrsi/data/knowledge_packs/README.md index 8f24e6859..12415f8e5 100644 --- a/src/inferrsi/data/knowledge_packs/README.md +++ b/src/inferrsi/data/knowledge_packs/README.md @@ -8,7 +8,7 @@ experiment; repository conventions stay with their owning repository. | --- | --- | --- | | Contribution / `contribution` | vLLM, vLLM-Omni, vllm-ascend; legacy repository packs also retain imported Strata, SGLang and SGLang-Omni procedures | L2C blind attempt → comparison → scoped skill; separate performance evidence/candidates | | Dependency / `dependency` | FlashInfer, FlashAttention, PyTorch/Inductor, Triton, Diffusers, Transformers | API/capability constraints and downstream impact; no blind coding session required | -| Peer technique / `peer` | SGLang, TensorRT-LLM, LMDeploy, xDiT, Cache-DiT | Technique, hardware, reported benefit, costs, quality risks and a reproduction hypothesis | +| Peer technique / `peer` | SGLang, TensorRT-LLM, LMDeploy, xDiT, Cache-DiT, Qwen3-TTS engine, DeepInfer | Technique, hardware, reported benefit, costs, quality risks and a reproduction hypothesis | This is an acquisition policy, not a ranking or fixed dependency graph. Cache-DiT, for example, can also be a direct dependency of vLLM-Omni. Its entry @@ -117,6 +117,20 @@ speedup numbers whose conditions or isolated contribution are unclear. | peer | xdit | [xDiT parallelism: separate current-state attention from stale-state pipelines](peer/xdit/cuda/diffusion/parallelism-freshness-and-topology.json) | | peer | cache_dit | [Cache-DiT residual reuse needs model- and schedule-specific calibration](peer/cache_dit/common/diffusion/residual-cache-calibration.json) | +## DeepInfer extraction: 2026-10-20 + +Three source-derived candidates at the same immutable DeepInfer revision. +Owner review on 2026-10-22 corrected query/KV ownership, nested feature gates, +verification budgeting and release-scope attribution. Original access dates +are retained; newly inspected sources carry their own access date. No native, +model, service or performance validation is claimed. + +| Tier | Source | Card | +| --- | --- | --- | +| peer | deepinfer | [DeepInfer zero-overhead scheduling: defer the previous step's post-process to overlap host prep with device work](peer/deepinfer/common/llm/zero-overhead-deferred-scheduling.json) | +| peer | deepinfer | [DeepInfer verification budget: spend verification rows by marginal acceptance, not by fixed draft width](peer/deepinfer/common/llm/speculative-verification-budget.json) | +| peer | deepinfer | [DeepInfer bounded replay: compact query rows with a derived dependency closure and separately preserved historical state](peer/deepinfer/common/llm/bounded-windowed-replay.json) | + ## Maintaining the tiers 1. Search for an existing ID and topic before adding a card. Keep one canonical diff --git a/src/inferrsi/data/knowledge_packs/peer/deepinfer/common/llm/bounded-windowed-replay.json b/src/inferrsi/data/knowledge_packs/peer/deepinfer/common/llm/bounded-windowed-replay.json new file mode 100644 index 000000000..eaf659ee7 --- /dev/null +++ b/src/inferrsi/data/knowledge_packs/peer/deepinfer/common/llm/bounded-windowed-replay.json @@ -0,0 +1,171 @@ +{ + "schema_version": 1, + "id": "deepinfer.common.llm.bounded_windowed_replay", + "tier": "peer", + "repo": "deepinfer", + "backend": "common", + "component": "llm", + "scenario": "llm.*", + "title": "DeepInfer bounded replay: compact query rows with a derived dependency closure and separately preserved historical state", + "status": "imported", + "summary": "A decoder stack split at an explicit producer/consumer boundary can replay only a retained suffix if the boundary captures the full-length state the consumer needs. The retain arithmetic is small and transferable; which layers are safe to shorten is a per-model dependency question and must be derived, not assumed from per-layer windowing.", + "tags": [ + "deepinfer", + "sliding_window_attention", + "prefix_replay", + "kv_cache", + "metadata", + "correctness", + "transfer_hypothesis" + ], + "sources": [ + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/input_metadata/prepare_tensor_input_v4.cpp", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "93a6b1dd533db116d2f7540c78d6bb5419bdb8c1d2063b031ab8b1a8cc138fe2" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/input_metadata/yoco.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "882a33de3da0fba81644adde9e9e1a90ec3d3ff34a8f3ad4dda18faafebd6460" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/environ.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "fe2c531989c419f0b51b169a07f5ed826dcec7f95c0c5bc3a351805923c4f218" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/models/v4_modules/yoco.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "7dc8ad7b6ca4902bcef67b1cbc4804592995c5b6af9ebaa00b59cc338c1aa6ec" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/models/v4_modules/attn.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "cb228c8fd4c76200e2ec266c51872107cfe7b384fdc88458b3d1c0ff101f3116" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/cache_engine.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "ca3d80120fe6bc50971b180278dd575d585e1a6c7a3c2174a8a6fd6213454161" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/llm_engine.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "a636d60ca5711f6b7c21e68370c3bd9c2de6fc3f4439a7429ca4f572f0edeebe" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/model_metadata_utils.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-22", + "content_sha256": "1ef7803d2b0083d025de676c841db9a317235b881fb9ebf6781ec76d49d7b22c" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/model_runner.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-22", + "content_sha256": "9e2e18d136c9e0c9f23905ff994e9d23afaa94aa7a133d425e7fc1fc069033ef" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/models/decoder_layer.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-22", + "content_sha256": "2189fa14c0665a3a211b47c5440748273fd92b39a68268a0080530d3c95b7297" + } + ], + "applicability": { + "hardware": [ + "Backend-independent retain rule; the reference asserts support only for NVIDIA SM100 and Ascend 950" + ], + "software": [ + "deepseek-ai/DeepInfer at 8b16149741a4935e1dec8cea27853f273b374c5b; the model's window sizes and layer boundary come from its config and do not transfer" + ], + "conditions": [ + "The decoder has an explicit boundary: earlier layers run over the full prefix, later consumer layers run compact query rows while reading all required preserved state", + "A dependency closure shows each retained query's inputs are available from retained producer outputs or explicitly preserved historical state", + "Full-length consumer KV is written at the boundary, independent of the compact query", + "The rollback depth is known and floored at the speculation width plus one" + ] + }, + "claims": [ + { + "kind": "source_observed", + "description": "The retain length is window plus backtracking depth, and the backtracking depth cannot be configured below the speculative width plus one, so a full-width rollback still leaves a full window." + }, + { + "kind": "source_observed", + "description": "The retain check is guarded by the truncation mode: the metadata entrypoint returns early when the mode is off, so the keep-length assertions apply only when truncation is enabled. The keep length has no default value and is not derived from the window, so enabling truncation without setting it fails rather than silently clipping. The same condition is checked at model-metadata construction and at worker setup." + }, + { + "kind": "source_observed", + "description": "The SWA path's visible keys intersect the sliding window start, retained rectangle and valid-state start; recompute clears SWA state below the last bound. This does not constrain all main/index historical KV to the compact query suffix. An empty SWA visible region is represented explicitly." + }, + { + "kind": "source_observed", + "description": "Two modes exist: a mask mode that keeps every query and only narrows visibility, and a compact mode that removes clipped rows. Only the second saves compute, and the first is therefore the natural equivalence reference." + }, + { + "kind": "source_observed", + "description": "The boundary returns both a compact copy and the original full hidden and residual; the caller stores the full one and continues with the compact one. The full-length consumer KV is written from the full hidden in a separate pass, so the persistent cache stays complete while the query is compact." + }, + { + "kind": "source_observed", + "description": "For a positive window the inference ring is window plus rollback; otherwise it is zero. Both kvstore-ring additions are gated by the store-extra-state flag. Within that branch the store-granularity margin is added, plus the keep margin only when truncation mode is not off. The keep margin therefore requires both gates." + }, + { + "kind": "source_observed", + "description": "Prompt logprobs are not reconstructable from a clipped region; the engine forces them off with a warning rather than rejecting the request, so the request still runs with per-token prompt logprobs disabled." + } + ], + "downstream_impact": [ + "The retain arithmetic and the visible-region formula are reusable in any decoder that can derive the same boundary dependency; the boundary index, window sizes and ring lengths must be re-derived per model.", + "Query-row compaction and historical-key retention are separate decisions. A consumer may keep full-layout keys in a compressed or shared index while computing over fewer query rows, so compaction is not evidence that historical keys were dropped.", + "Prompt logprobs become unavailable for the clipped region, whether by being forced off or by rejection; which of the two is a policy choice, not a consequence of the mechanism.", + "The mask/compact split is a reusable testing pattern: a compute-neutral diagnostic mode isolates the visibility math from the gather." + ], + "recommendations": [ + "Derive the producer/consumer dependency closure before applying window arithmetic; one layer's finite window does not determine the whole stack's dependency horizon. Preserved historical state remains a valid input.", + "Enforce the retain length at every entry, not once, because the value can arrive through an environment variable.", + "Do not report the diagnostic mode as an optimization; it is compute-neutral by construction.", + "State whether historical keys are preserved, compacted or compressed, separately from the query-compaction decision.", + "State the anchor explicitly. Preemption-and-recompute moves the retained rectangle, so bit-identical results across preemption are not available." + ], + "validation_plan": [ + "Dependency closure: for every consumer layer, list what it reads and where it was produced.", + "Full-length KV write exists at the boundary and is independent of the compact query.", + "Metadata equivalence between the two modes at chunk boundaries: visible-key sets, store slots and compressed-key ranges.", + "End-to-end equivalence on a fixed teacher-forcing trajectory, comparing logprobs and top-k element-wise.", + "A dependency-probe run that keeps all historical keys and shortens only the query, to separate the two decisions.", + "Chunk sizes that do and do not align with store granularity, including a fully clipped chunk.", + "Preemption count during the comparison run, which must be zero for the result to be interpretable." + ], + "limitations": [ + "No equivalence run was performed; the metadata argument is from the shared formula, not from a measurement.", + "Applies to architectures with a demonstrated boundary dependency, not to arbitrary sliding-window stacks.", + "Window sizes, boundary-layer index and ring lengths are specific to one model and must not be carried across.", + "Query-row compaction does not imply dropping historical keys.", + "Prompt logprobs are not reconstructable from a clipped region.", + "The truncation path is off by default in the source." + ], + "verification": { + "locally_verified": false, + "evidence_level": "source_observed" + } +} diff --git a/src/inferrsi/data/knowledge_packs/peer/deepinfer/common/llm/speculative-verification-budget.json b/src/inferrsi/data/knowledge_packs/peer/deepinfer/common/llm/speculative-verification-budget.json new file mode 100644 index 000000000..2cbfc776a --- /dev/null +++ b/src/inferrsi/data/knowledge_packs/peer/deepinfer/common/llm/speculative-verification-budget.json @@ -0,0 +1,160 @@ +{ + "schema_version": 1, + "id": "deepinfer.common.llm.speculative_verification_budget", + "tier": "peer", + "repo": "deepinfer", + "backend": "common", + "component": "llm", + "scenario": "llm.*", + "title": "DeepInfer verification budget: spend verification rows by marginal acceptance, not by fixed draft width", + "status": "imported", + "summary": "A confidence head predicts per-position acceptance; a cumulative non-increasing rate vector plus a fitted step-cost model selects how many draft rows to verify per step. The objective is expected accepted tokens per unit time, so fewer verified drafts may accept fewer tokens while still raising throughput. Draft compute is unchanged.", + "tags": [ + "speculative_decoding", + "mtp", + "verification", + "cost_model", + "acceptance_rate", + "transfer_hypothesis" + ], + "sources": [ + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/worker.py#L118-L219", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-22", + "content_sha256": "ad32e595d9ab289dff1a7709136431ac60dae709e9e5123ae42eb0ebf7c30cbb" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/mtp_varlen.hpp", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "164f45db4cccd6d6909dd2846e7dfffb57754b108af679443bf4bd71ec33785a" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/environ.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "fe2c531989c419f0b51b169a07f5ed826dcec7f95c0c5bc3a351805923c4f218" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/sampling_params.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "f800034efdebe1b21620660a51fa9122d3d7f138d8e60cd738f33d643c3116f8" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/models/mtp/dflash.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "12ebfb9c793062844d40e0fab17164d5365d965c318f8b9f0f4adf51b3afb684" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/layers/tilelang_ops/fused_sample.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "f43ac934284d15f708a0aa07aaa8fff7daa257984e311ca554bd9dbea89e5e1d" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/layers/sampler.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-22", + "content_sha256": "81205a12e860041ffa904321b852db2d048bc038bb07d494dbb78b44b522d9a9" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/scheduler.hpp", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-22", + "content_sha256": "c14de4d5341170227b82ddf2d146d9c6a337937885a6886237aab5a7238da5db" + } + ], + "applicability": { + "hardware": [ + "Backend-independent procedure; the reference implementation targets NVIDIA SM100 and Ascend 950 only" + ], + "software": [ + "deepseek-ai/DeepInfer at 8b16149741a4935e1dec8cea27853f273b374c5b; requires a model with a trained confidence head" + ], + "conditions": [ + "Multi-token speculative decoding with a draft head", + "Verification cost grows with query rows", + "Acceptance varies materially by request and draft position", + "A measured (rows, step_time) curve is obtainable at warmup", + "The per-position confidence feeding a verification decision at position i is independent of confidence at positions greater than i" + ] + }, + "claims": [ + { + "kind": "source_observed", + "description": "Two queues of (accept_rate, request) pairs are sorted descending by rate with index tie-breaking, and rows are granted one at a time, so every grant is a prefix over draft positions." + }, + { + "kind": "source_observed", + "description": "The selection maximizes expected accepted tokens over estimated step time, where step time is a graph-size baseline plus a non-negative fitted slope times graph size plus a sampler term estimated per request count. Because the numerator is an expectation over accepted prefixes, reducing verified rows can reduce accepted tokens while still raising the ratio." + }, + { + "kind": "source_observed", + "description": "The search starts with one row per scheduled request and asserts best_bsz >= num_seqs. This is a row-allocation contract, not a measured speed comparison with plain decode." + }, + { + "kind": "source_observed", + "description": "The inspected DFlash decode producer samples its configured full width in both paths; variable-length scheduling budgets verification rows. This conclusion comes from sampler.py and DFlash's fixed-width metadata/forward/sample chain, not the non-budgeted verification-length branch." + }, + { + "kind": "source_observed", + "description": "The budgeted path is opt-in. DEEPINFER_ENABLE_MTP_VARLEN defaults false while DEEPINFER_USE_NEXTN_DECODE defaults 5, so fixed-width speculation is the default and the budget must be enabled." + }, + { + "kind": "source_observed", + "description": "The source requires later-position confidence not to affect earlier verification decisions. CPU scheduling takes the total budget from N-2 and reads the latest CPU accept_rates for per-request lengths; when a forward is in flight those rates lag. The overlapping ZOS worker redistributes fixed total_q using the previous sampler's GPU confidence before the main forward, then synchronizes corrected metadata before sampling. This source design is not a runtime sampling-correctness proof." + }, + { + "kind": "source_observed", + "description": "The confidence-head temperature and the request sampling temperature are separate values with separate clamps. The former scales the confidence head's output; the latter is a SamplingParams field. Neither implies the other." + }, + { + "kind": "source_observed", + "description": "The reject path is gated on draft logprobs being supplied rather than on request temperature, so a request configured as greedy still goes through rejection. This reading is from the gating logic; no run confirmed it end to end." + } + ], + "downstream_impact": [ + "vLLM-family engines with EAGLE-style multi-token verification could adopt the cumulative-rate plus cost-model framing; whether their verification cost has the same shape is unverified.", + "The per-position independence requirement is the part most likely to be lost in transfer: any alternative confidence timing must be justified against it, not merely measured for speed.", + "The two-round lag is a consequence of that requirement under a pipelined scheduler, so it cannot be simplified away by reading an earlier round without re-deriving the property.", + "Ascend: no separate arithmetic is established; cross-backend rejection parity is unproven in the source itself." + ], + "recommendations": [ + "Report verification rows and accepted tokens as separate measurements. A lower accepted length is an expected outcome of budgeting, not a defect.", + "Validate sampling correctness against an independently specified rejection rule rather than by comparing budget-on and budget-off token streams, which do not have to agree.", + "Include the draft forward in any throughput number, or the reported gain overstates the effect.", + "Do not apply this to a separate draft model whose forward cost scales with the number of drafts; the budget would need to throttle drafting too.", + "Do not use acceptance uniformity alone as a stop condition. Near-zero acceptance can still favour a one-row budget by avoiding wasted verification; decide from the measured cost model." + ], + "validation_plan": [ + "Allocation invariants: every grant is a prefix over draft positions and at least one row per request.", + "Sampling correctness against a separately implemented rejection rule, on synthetic distributions with known acceptance probabilities.", + "Per-position independence probe: perturb confidence at positions after i and confirm the verify decision at i is unchanged.", + "Verification rows per step, accepted tokens per step, total step time and end-to-end throughput, each reported separately.", + "Acceptance-uniformity sweep including all-low, to test the cost model rather than to exclude the case.", + "Peak memory at fixed concurrency, because wide top-k verification pools can dominate memory." + ], + "limitations": [ + "No gain was reproduced; nothing here is a measured speedup.", + "The source's own efficiency figures are deployment documentation, not a controlled experiment.", + "Requires a trained confidence head; a substitute estimate needs its own calibration.", + "Budgeting can lower accepted length, so equal accepted length is not an available correctness criterion.", + "Cross-backend equality of the rejection comparison has no assertion in the source.", + "Numeric acceptance rates and warmup curves are deployment-specific and are not carried in this record." + ], + "verification": { + "locally_verified": false, + "evidence_level": "source_observed" + } +} diff --git a/src/inferrsi/data/knowledge_packs/peer/deepinfer/common/llm/zero-overhead-deferred-scheduling.json b/src/inferrsi/data/knowledge_packs/peer/deepinfer/common/llm/zero-overhead-deferred-scheduling.json new file mode 100644 index 000000000..f381dd0f3 --- /dev/null +++ b/src/inferrsi/data/knowledge_packs/peer/deepinfer/common/llm/zero-overhead-deferred-scheduling.json @@ -0,0 +1,140 @@ +{ + "schema_version": 1, + "id": "deepinfer.common.llm.zero_overhead_deferred_scheduling", + "tier": "peer", + "repo": "deepinfer", + "backend": "common", + "component": "llm", + "scenario": "llm.*", + "title": "DeepInfer zero-overhead scheduling: defer the previous step's post-process to overlap host prep with device work", + "status": "imported", + "summary": "Move the CPU-only post-process of step N into step N+1 so it overlaps the next forward. The reusable part is the staleness enumeration and the on-device repair, not the scheduler itself.", + "tags": [ + "scheduling", + "host_overhead", + "overlap", + "pipelining", + "stale_state", + "transfer_hypothesis" + ], + "sources": [ + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/llm_engine.py#L307-L314", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-22", + "content_sha256": "a636d60ca5711f6b7c21e68370c3bd9c2de6fc3f4439a7429ca4f572f0edeebe" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/parallel_utils/parallel_state.py#L89-L94", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-22", + "content_sha256": "3565b2449abff4df0df2f979061a73364886c93602218b43c2f86c70f2629801" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/worker.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "ad32e595d9ab289dff1a7709136431ac60dae709e9e5123ae42eb0ebf7c30cbb" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/zos.cpp", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "1cb70e0b4ed7181b95e19ef2cc3cc640d1cd8f24e874ce4bc34fe51d6f031b46" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/scheduler.hpp", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "c14de4d5341170227b82ddf2d146d9c6a337937885a6886237aab5a7238da5db" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/arg_utils.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "c28c478bd376165c073c6e0722bed8468599a2fd766ffc7aafcd4ef270af52c5" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/engine_type.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "9933d83f500e3cc14c5fef0dd2fbe5c5170dbbf24df217376b0a55c4fb4ab48c" + }, + { + "url": "https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/utils.py", + "type": "source_code", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "accessed_at": "2026-10-20", + "content_sha256": "8a0049b98ea6f7ef98555ad8b38b9e8dac2292c0d2a89c2090bbdb5e5ee7295a" + } + ], + "applicability": { + "hardware": [ + "Backend-independent pipelining idea; the reference implementation asserts support only for NVIDIA SM100 and Ascend 950" + ], + "software": [ + "deepseek-ai/DeepInfer at 8b16149741a4935e1dec8cea27853f273b374c5b; later revisions need reinspection" + ], + "conditions": [ + "Decode-heavy workload where the CPU post-process is exposed between device kernels", + "At least one extra step of retained metadata, state slots and KV blocks is affordable", + "Distributed step-type votes can run on a host-side backend, not a device collective", + "The entrypoint has a drain path, because callers see results one step late" + ] + }, + "claims": [ + { + "kind": "source_observed", + "description": "The reorder is forward(N) -> post-process(N-1) -> sample(N) -> defer post-process(N). Prompt steps never defer; only decode steps, and only when DEEPINFER_ZERO_OVERHEAD_SCHEDULING is set." + }, + { + "kind": "source_observed", + "description": "Three quantities become one step stale: per-request query length under adaptive speculation, per-request key length after rejected drafts, and input token ids for rows continuing from the previous batch. Each is repaired on device from a remap of the previous batch." + }, + { + "kind": "source_observed", + "description": "The scheduler releases the GIL; the standard ZOS engine constructor selects its CPU data-parallel group, created with Gloo, and the scheduler's Gloo vote uses a CPU tensor. This is a constructor/code path observation, not proof that every scheduler branch avoids device synchronization." + }, + { + "kind": "source_observed", + "description": "The default depends on how the engine object is built. The environment variable defaults false, while EngineArgs sets it with setdefault from a per-engine-type property that is true for NORMAL and COMPLETION and false for PROMPT. A README claim of 'enabled by default' and a false variable default can both be true of different entrypoints." + }, + { + "kind": "source_observed", + "description": "Forward and sample launches use a torch-visible synchronization debug guard. Event.synchronize is outside its coverage, another thread's allow scope can lift enforcement, and an environment setting disables it. Guard presence alone does not prove synchronization-free execution." + } + ], + "downstream_impact": [ + "The host-overhead profiling procedure measures exposed preparation; this source case illustrates moving that preparation rather than shrinking it, without a reproduced gain.", + "A vLLM or vLLM-family engine could adopt the reorder only if its scheduler is device-free and it can enumerate the stale quantities; neither is established here.", + "Ascend: the mechanism is arithmetic and metadata, but the reference asserts unsupported combinations rather than degrading, so the transfer hypothesis needs its own gate check." + ], + "recommendations": [ + "Treat the staleness list as the deliverable, not the code. An engine that cannot enumerate what becomes stale cannot safely pipeline.", + "Confirm the drain path before measuring anything; without it tokens are silently lost at the end of a request.", + "Preserve the standard ZOS constructor's host DP-group ordering, or re-establish the ordering for a custom construction before measuring overlap." + ], + "validation_plan": [ + "Profile with named regions before and after, reporting the idle window separately from kernel time.", + "Compare a fixed-seed single request token-for-token between the pipelined and direct orders.", + "Report the fraction of steps where the remap found overlap, split by prompt and decode.", + "Measure peak memory, since one extra step of state is retained." + ], + "limitations": [ + "No DeepInfer build or run was performed; nothing here is a measured gain.", + "The reference is an offline engine for one model; transfer to a serving engine with continuous batching is unproven.", + "Scheduler stale-output guards, sample-event synchronization and deferred metadata release exist. Their presence does not establish complete in-flight KV/state reuse safety; a reachable preemption race was neither proved nor reproduced, so it remains a validation question rather than a known defect.", + "Numeric timer thresholds and any reported throughput belong to the source deployment and are not reproduced." + ], + "verification": { + "locally_verified": false, + "evidence_level": "source_observed" + } +} diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md b/src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md index dd53c5678..e4bf4b92f 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md @@ -1,6 +1,6 @@ # InferRSI skill discovery index -Generated abstracts for all 55 registered skills in `data/skill_library/`. +Generated abstracts for all 57 registered skills in `data/skill_library/`. Their complete procedures, references and cases ship in the same package. `skill-index.json` maps stable skill IDs to categorized directories. The reference and searchspace matrices @@ -15,7 +15,7 @@ These are knowledge abstracts (`--kind knowledge`), with namespaced package IDs distinct from explicitly selected `checkout_ref` skills. Ascend/CUDA procedures use their corresponding backend; shared methods use `common`. -Source snapshot: [b5178de4f8e279b301614dd8a613ebd1c1985984](https://github.com/InferMatrixLab/InferRSI/tree/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library). +Source snapshot: [a704774293ea320b0a2f1f5229d61d7b307fc089](https://github.com/InferMatrixLab/InferRSI/tree/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library). ```bash inferrsi knowledge search 'profiling' --repo-type inferrsi --limit 50 @@ -29,7 +29,7 @@ contain the current input paths and bytes (Git checkout filters are respected). ```bash python tools/build_skill_index.py --source-revision -python tools/build_skill_index.py --source-revision b5178de4f8e279b301614dd8a613ebd1c1985984 --check +python tools/build_skill_index.py --source-revision a704774293ea320b0a2f1f5229d61d7b307fc089 --check ``` `--check` detects missing, stale and unexpected generated records without writing. @@ -53,6 +53,8 @@ References and cases ship with their owning skill, not as duplicate knowledge ab | [audit-sd-cpp-residency](common/skills/audit-sd-cpp-residency.json) | common | candidate | | [audit-sglang-diffusion-lifetimes](common/skills/audit-sglang-diffusion-lifetimes.json) | common | candidate | | [audit-vllm-kv-lifetimes](common/skills/audit-vllm-kv-lifetimes.json) | common | candidate | +| [bound-windowed-attention-replay](common/skills/bound-windowed-attention-replay.json) | common | candidate | +| [budget-speculative-verification](common/skills/budget-speculative-verification.json) | common | candidate | | [calibrate-serving-simulation](common/skills/calibrate-serving-simulation.json) | common | candidate | | [cann-attribution-overlap](ascend/skills/cann-attribution-overlap.json) | ascend | candidate | | [cgroup-resource-controls](common/skills/cgroup-resource-controls.json) | common | candidate | diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-aclgraph-capture-replay.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-aclgraph-capture-replay.json index a5ad8f354..8e4346d82 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-aclgraph-capture-replay.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-aclgraph-capture-replay.json @@ -21,8 +21,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json", @@ -41,7 +41,7 @@ "src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md": "718be35dc6d93a97e356ed645df899d43e2c37944f4dece8e167efef810fefea", "src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json": "e4c02d4ae85e473947e024c38aed121ca8e5cfbfe1091c5792ce58a349248187", "src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/references/source-notes.md": "6ffbfb68cc7e04923b4fbca2a73a7555b7c3b45a6f40819bb5f3213ef05cf88d", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "ascend-aclgraph-capture-replay", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-device-health.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-device-health.json index ac308d9d8..39f68106c 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-device-health.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-device-health.json @@ -23,8 +23,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/metadata.json", @@ -41,7 +41,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md": "03829039860ece35519f2e4c339a48b5014ec25e07a2c150a3efb6a3a4e35b4a", "src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/metadata.json": "2c23209ef418287f1023f53eba4eda66551f09ab102a61c879df27c25365e757", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "ascend-device-health", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-distributed-parallel.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-distributed-parallel.json index 08be7ad9a..9d7a98eab 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-distributed-parallel.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-distributed-parallel.json @@ -24,8 +24,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/metadata.json", @@ -42,7 +42,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md": "4252cc6f984600393b277c45f8c3ef84f5dd0250c13d5b80a2e1b35d3531c520", "src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/metadata.json": "843c0a93bf3e61c107c432e05b23a7466ed07b7c061142fed1e6128ca22734cb", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "ascend-distributed-parallel", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-graph-mode.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-graph-mode.json index 90eb9bc11..1d4192e9d 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-graph-mode.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-graph-mode.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/metadata.json", @@ -40,7 +40,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md": "9203cc5a14a1d65df5fbf275734b93be023e06527459b2f79864f28a338cb44a", "src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/metadata.json": "b0bfa30de94dcda06e2289f5d4a2e44a473f40fb871059e4580f33588f2a51dc", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "ascend-graph-mode", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-model-adaptation.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-model-adaptation.json index e0ab0efa1..aa6e71bcb 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-model-adaptation.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-model-adaptation.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/metadata.json", @@ -40,7 +40,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md": "d9e81a63981819fac8f5a21dca9c90f86e26788f58339d3a9c8e66319c178472", "src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/metadata.json": "4c6d0c901f0b3858c7dc337c86d892194e4102c8b349932a4caf1a811c99b1ed", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "ascend-model-adaptation", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-profiling.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-profiling.json index f73d98da1..9cfde27e6 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-profiling.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-profiling.json @@ -23,8 +23,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/metadata.json", @@ -41,7 +41,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md": "401981cf7828edcab0c9ea882f7ecd83badfa3c2a95006e912cf99d67b3d6a12", "src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/metadata.json": "e45210a4df7b1ee183f06573dbc36db5ede182f11ab1cfad5ae22f523b89c557", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "ascend-operator-profiling", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-tuning.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-tuning.json index 733268b6d..a1bc91ad4 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-tuning.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-operator-tuning.json @@ -24,8 +24,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/metadata.json", @@ -42,7 +42,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md": "38b151bc6b4e9734c6fa41efb7a30343559400c0c75bbdd26bd8a968b8ea4592", "src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/metadata.json": "ab7dedfa873b8572db277e43536e6ce04c68ac987aee189ffd5861e4e9bb2d6d", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "ascend-operator-tuning", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-optimization-casebook.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-optimization-casebook.json index d33da0adc..877b9014c 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-optimization-casebook.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-optimization-casebook.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/ascend-optimization-casebook/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/ascend-optimization-casebook/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/ascend-optimization-casebook/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/ascend-optimization-casebook/metadata.json", @@ -40,7 +40,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/ascend/ascend-optimization-casebook/SKILL.md": "647f5e9a288f2c4e1617ba53df59d9d827df749eb4fe86182d21a9c41175caf8", "src/inferrsi/data/skill_library/hardware/ascend/ascend-optimization-casebook/metadata.json": "b6c4b9a73862b4434038f8e1bc24073864ee334014598fb998af592acb54bcb4", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "ascend-optimization-casebook", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-performance-analysis.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-performance-analysis.json index a6c42e35f..14a541b7e 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-performance-analysis.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascend-performance-analysis.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/ascend-performance-analysis/metadata.json", @@ -40,7 +40,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md": "955a7b4206e4ea3a73e9aa8723df1ad5f823f950520441bcfdf68d8ba07ca6c7", "src/inferrsi/data/skill_library/hardware/ascend/ascend-performance-analysis/metadata.json": "ce1ab797865799a58a1bb0ce5ee949a061283b6d5f0353c29ab6c7a95a3a12f1", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "ascend-performance-analysis", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascendc-tiling-design.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascendc-tiling-design.json index 755d70f30..cb0a50977 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascendc-tiling-design.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/ascendc-tiling-design.json @@ -24,8 +24,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/ascendc-tiling-design/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/ascendc-tiling-design/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/ascendc-tiling-design/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/ascendc-tiling-design/metadata.json", @@ -42,7 +42,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/ascend/ascendc-tiling-design/SKILL.md": "93aa1b775ced304cebfe6c4dec0503ddb2260ecb3cb01edc17ce329446c5ce2b", "src/inferrsi/data/skill_library/hardware/ascend/ascendc-tiling-design/metadata.json": "04dd3e8961d5c67105b11ff52f1354f34124d9af1b03f9da08cba67cb9f9d463", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "ascendc-tiling-design", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/cann-attribution-overlap.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/cann-attribution-overlap.json index 8f841a39f..8cfa7b876 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/cann-attribution-overlap.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/cann-attribution-overlap.json @@ -24,8 +24,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/cann-attribution-overlap/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/cann-attribution-overlap/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/cann-attribution-overlap/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/cann-attribution-overlap/metadata.json", @@ -42,7 +42,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/ascend/cann-attribution-overlap/SKILL.md": "0e6f094dc554abea12e07def50f20b233b47127779a8aefad595b4ab1efd24c2", "src/inferrsi/data/skill_library/hardware/ascend/cann-attribution-overlap/metadata.json": "64b83aae3b9be5ada75e6111114a93bf55173d9df913b4ea8bafd2b4cea23018", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "cann-attribution-overlap", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-device-memory-lifecycle.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-device-memory-lifecycle.json index 00fab7225..15afce5b2 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-device-memory-lifecycle.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-device-memory-lifecycle.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/metadata.json", @@ -42,7 +42,7 @@ "src/inferrsi/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/SKILL.md": "8428df7157af024fa17cfe0b3818ed54bd9f468306dda46eb8d4f0edf4f3f230", "src/inferrsi/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/metadata.json": "7ee44d25214f4a6bbeeea4bec61941b2ec2ea469e595865e9346a7589576bd16", "src/inferrsi/data/skill_library/hardware/ascend/torchnpu-device-memory-lifecycle/references/source-notes.md": "80cba4d3a884e7ee2618326cf74b9f9c3a34e4d9cf443707f6bbe673f2d5ba65", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "torchnpu-device-memory-lifecycle", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-pinned-host-memory-lifecycle.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-pinned-host-memory-lifecycle.json index 585da4e5b..6756403e9 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-pinned-host-memory-lifecycle.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/ascend/skills/torchnpu-pinned-host-memory-lifecycle.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md", "src/inferrsi/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/metadata.json", @@ -42,7 +42,7 @@ "src/inferrsi/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/SKILL.md": "23e93ea7c98f99743509fdc04139d78ed2e7123c63e80b366c6beaba7eb70d0d", "src/inferrsi/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/metadata.json": "17f96ff8d36a844d4a62a06975ad918bb8554f343543a3e7ea23e2d98c37fd0b", "src/inferrsi/data/skill_library/hardware/ascend/torchnpu-pinned-host-memory-lifecycle/references/source-notes.md": "435193590b4a2ba32e2e4c6fa9d7e8f889528d412cbbb48e35ec5433f7dab384", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "torchnpu-pinned-host-memory-lifecycle", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/assess-serving-preemption.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/assess-serving-preemption.json index 32e49b709..34dd6a051 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/assess-serving-preemption.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/assess-serving-preemption.json @@ -16,8 +16,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/assess-serving-preemption/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/assess-serving-preemption/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/assess-serving-preemption/SKILL.md", "src/inferrsi/data/skill_library/methods/assess-serving-preemption/metadata.json", @@ -36,7 +36,7 @@ "src/inferrsi/data/skill_library/methods/assess-serving-preemption/SKILL.md": "0a5f48d9352c5bf460775ef47f9136829978811f4a16e84983e299782cae305d", "src/inferrsi/data/skill_library/methods/assess-serving-preemption/metadata.json": "4c52db9b143204496d9bc194d981322ad39a6f8cb399fb49abcb80d7b2924533", "src/inferrsi/data/skill_library/methods/assess-serving-preemption/references/source-notes.md": "f29fcecfba0490008a354c11b42403983fe165e56ff907ad61cc6040d2b6407f", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "assess-serving-preemption", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/assess-split-backward-schedule.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/assess-split-backward-schedule.json index 0796b38d5..cba8e7026 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/assess-split-backward-schedule.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/assess-split-backward-schedule.json @@ -16,8 +16,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/assess-split-backward-schedule/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/assess-split-backward-schedule/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/assess-split-backward-schedule/SKILL.md", "src/inferrsi/data/skill_library/methods/assess-split-backward-schedule/metadata.json", @@ -36,7 +36,7 @@ "src/inferrsi/data/skill_library/methods/assess-split-backward-schedule/SKILL.md": "1e64f28ea71672fb53c56f6108b87864738c6d03d6dcd32eedaef9e0625e96e3", "src/inferrsi/data/skill_library/methods/assess-split-backward-schedule/metadata.json": "858fdd88c6bd2f118bd1d527bb37d47bcc2722cc6140d52ee4657e646cdce080", "src/inferrsi/data/skill_library/methods/assess-split-backward-schedule/references/source-notes.md": "61b1609f71c37b61d136b8c4d39f793de87cffb1cf31c8ec737a1c3fa6fb1b4e", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "assess-split-backward-schedule", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-llama-runtime-state.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-llama-runtime-state.json index a66910c0f..a4c46e213 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-llama-runtime-state.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-llama-runtime-state.json @@ -23,8 +23,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md", "src/inferrsi/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/metadata.json", @@ -43,7 +43,7 @@ "src/inferrsi/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/SKILL.md": "a7bc715ab5c740e8e6296adf9d891f4ff7c68cdba2db0aeccd1f70f86f820eb1", "src/inferrsi/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/metadata.json": "c354138008eb577b93d2a3cf93f6ae56369439d0d70d799c47a63d8cb4ff8b75", "src/inferrsi/data/skill_library/repositories/llama-cpp/audit-llama-runtime-state/references/source-notes.md": "3d12a401e9decb55a2f4f3b2bfa0d360ef5110f63a0469dfe285654e78054190", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "audit-llama-runtime-state", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-sd-cpp-residency.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-sd-cpp-residency.json index faf64657a..55f528287 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-sd-cpp-residency.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-sd-cpp-residency.json @@ -23,8 +23,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/LICENSE", "src/inferrsi/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/SKILL.md", @@ -51,7 +51,7 @@ "src/inferrsi/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-identity.json": "7a324c05817f733fbe6a4b2b36317034c05dd5db06eb0be089546119d80060b1", "src/inferrsi/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/source-notes.md": "b67affbd7d237d82b7f51be49f1537e9820fe56ec70a71ee76809f9e599ca693", "src/inferrsi/data/skill_library/repositories/stable-diffusion-cpp/audit-sd-cpp-residency/references/validation-boundaries.md": "5dcf5fa7bf734fbe13214f57d2dfb40ff8215d9cd34e072a9e615bf103b82fc2", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "audit-sd-cpp-residency", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-sglang-diffusion-lifetimes.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-sglang-diffusion-lifetimes.json index e15843e5a..75b0ed288 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-sglang-diffusion-lifetimes.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-sglang-diffusion-lifetimes.json @@ -23,8 +23,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/LICENSE", "src/inferrsi/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md", @@ -45,7 +45,7 @@ "src/inferrsi/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/SKILL.md": "384d3ac7ee7084109e471cb7fa424b6476d7e2874db06ebfc8a522ba8a97c6b0", "src/inferrsi/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/metadata.json": "2080a5435a66ade85d2d8318003afd7f25e37aa7942ba6b468992115de7fa20b", "src/inferrsi/data/skill_library/repositories/sglang/audit-sglang-diffusion-lifetimes/references/source-notes.md": "d0fcfd1177ae2f351041c5b2abba7f5121cec133006d9e4dca8f74ddb567332d", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "audit-sglang-diffusion-lifetimes", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-vllm-kv-lifetimes.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-vllm-kv-lifetimes.json index 5c0c73788..d8193ff3b 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-vllm-kv-lifetimes.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/audit-vllm-kv-lifetimes.json @@ -21,8 +21,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md", "src/inferrsi/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/metadata.json", @@ -41,7 +41,7 @@ "src/inferrsi/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/SKILL.md": "313733a65cc48442b18dda1c4665139a689963a90e1032a2235bc6f40e3a8e06", "src/inferrsi/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/metadata.json": "d061abfaf80db3d1b5b34644e38281f9d623d1f49d1e17ef872ebd1a705d4b6c", "src/inferrsi/data/skill_library/repositories/vllm/audit-vllm-kv-lifetimes/references/source-notes.md": "3665b034594fc8cc767c408002f4e16f8803369242405027c7f37ce6c9ecebe9", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "audit-vllm-kv-lifetimes", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/bound-windowed-attention-replay.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/bound-windowed-attention-replay.json new file mode 100644 index 000000000..26494419b --- /dev/null +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/bound-windowed-attention-replay.json @@ -0,0 +1,135 @@ +{ + "schema_version": 1, + "id": "inferrsi.common.skills.bound_windowed_attention_replay", + "title": "bound-windowed-attention-replay", + "repo": "inferrsi", + "tier": "contribution", + "backend": "common", + "component": "skills", + "scenario": "skills.discovery", + "status": "imported", + "summary": "Bound replay or prefill query rows across a producer/consumer boundary whose retained inputs and separately preserved historical state satisfy the full dependency closure. Use when a model has shared hidden state or preserved-compressed KV and the retained region must be derived rather than assumed. Derives the retain rule from window plus rollback depth, separates query-row compaction from dropping historical keys, and names what becomes unrecoverable.", + "tags": [ + "deepinfer", + "sliding-window-attention", + "prefix-replay", + "kv-cache", + "metadata", + "correctness" + ], + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", + "paths": [ + "src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md", + "src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/metadata.json", + "src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/source-notes.md", + "src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/validation.md", + "src/inferrsi/data/skill_library/skill-index.json" + ], + "content_sha256": "4703beb6e5194ef2ff2c0b5ec69f64e35417a4bc0dfb50a10c80a920b77ef3b5" + } + ], + "index_type": "packaged-skill", + "skill_documents": { + "original": "data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md" + }, + "checkout_ref": "checkout:bound-windowed-attention-replay", + "source_files": { + "src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md": "4703beb6e5194ef2ff2c0b5ec69f64e35417a4bc0dfb50a10c80a920b77ef3b5", + "src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/metadata.json": "ce52c99752ddb771adfa8ae3fba53844f64219513b5e88126f62351de3bb3f9d", + "src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/source-notes.md": "fd917bd5ac80bee9c9f4525f8e0300f6d67dc144e2a16632af42a67f3d53d098", + "src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/validation.md": "60e71e70c5519bcdf6c171eec2be2f342e8801bedd746d6005f11adc646dfaff", + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" + }, + "skill_metadata": { + "name": "bound-windowed-attention-replay", + "description": "Bound replay or prefill query rows across a producer/consumer boundary whose retained inputs and separately preserved historical state satisfy the full dependency closure. Use when a model has shared hidden state or preserved-compressed KV and the retained region must be derived rather than assumed. Derives the retain rule from window plus rollback depth, separates query-row compaction from dropping historical keys, and names what becomes unrecoverable.", + "tags": [ + "deepinfer", + "sliding-window-attention", + "prefix-replay", + "kv-cache", + "metadata", + "correctness" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/deepseek-ai/DeepInfer", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "paths": [ + "csrc/engine/input_metadata/prepare_tensor_input_v4.cpp", + "deepinfer/engine/llm_engine.py", + "deepinfer/model_executor/input_metadata/yoco.py", + "deepinfer/model_executor/model_metadata_utils.py", + "deepinfer/model_executor/models/decoder_layer.py", + "deepinfer/model_executor/models/v4_modules/attn.py", + "deepinfer/model_executor/models/v4_modules/yoco.py", + "deepinfer/worker/cache_engine.py", + "deepinfer/worker/model_runner.py", + "deepinfer/environ.py", + "LICENSE" + ], + "readset": "references/source-notes.md" + } + ], + "applicability": { + "scope": "Decoders with an explicit producer/consumer boundary and a demonstrated dependency closure for compact query rows, including separately preserved historical state", + "prerequisites": "A derived dependency closure; a full-length KV write at the boundary; known rollback depth; control over metadata construction", + "hardware": "Backend-independent retain arithmetic; window sizes, boundary index and ring lengths come from the model config and do not transfer" + }, + "limitations": [ + "Source-derived candidate; no equivalence run was performed.", + "Applies to architectures with a demonstrated boundary dependency, not to arbitrary sliding-window stacks.", + "Window sizes, boundary-layer index and ring lengths are model-config specific and are not transferable.", + "Query-row compaction does not imply dropping historical keys; the two decisions are separate.", + "Prompt logprobs are unrecoverable for the clipped region; the reference forces them off with a warning.", + "The truncation path is off by default in the reference source." + ], + "provenance_readsets": [ + "references/source-notes.md", + "references/validation.md" + ], + "references": [], + "consumption_mode": "packaged" + }, + "applicability": { + "scope": "Decoders with an explicit producer/consumer boundary and a demonstrated dependency closure for compact query rows, including separately preserved historical state", + "prerequisites": "A derived dependency closure; a full-length KV write at the boundary; known rollback depth; control over metadata construction", + "hardware": "Backend-independent retain arithmetic; window sizes, boundary index and ring lengths come from the model config and do not transfer" + }, + "recommendations": [ + "Read the pinned SKILL.md and its prerequisites before choosing this procedure.", + "Read the packaged references and owner-local cases alongside the procedure." + ], + "limitations": [ + "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", + "The imported status describes this index, not the original skill; its declared status is candidate.", + "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", + "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", + "Source-derived candidate; no equivalence run was performed.", + "Applies to architectures with a demonstrated boundary dependency, not to arbitrary sliding-window stacks.", + "Window sizes, boundary-layer index and ring lengths are model-config specific and are not transferable.", + "Query-row compaction does not imply dropping historical keys; the two decisions are separate.", + "Prompt logprobs are unrecoverable for the clipped region; the reference forces them off with a warning.", + "The truncation path is off by default in the reference source." + ], + "verification": { + "locally_verified": false, + "evidence_level": "guidance" + } +} diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/budget-speculative-verification.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/budget-speculative-verification.json new file mode 100644 index 000000000..8c5bd6053 --- /dev/null +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/budget-speculative-verification.json @@ -0,0 +1,135 @@ +{ + "schema_version": 1, + "id": "inferrsi.common.skills.budget_speculative_verification", + "title": "budget-speculative-verification", + "repo": "inferrsi", + "tier": "contribution", + "backend": "common", + "component": "skills", + "scenario": "skills.discovery", + "status": "imported", + "summary": "Decide how many draft tokens to verify per decode step when acceptance varies by request and position. Use when an engine already runs a draft head with multi-token speculation and verification cost grows with query rows. Separates draft cost from verify cost, builds a cumulative-acceptance rate vector, fits a cost model, and allocates a per-step row budget.", + "tags": [ + "deepinfer", + "speculative-decoding", + "mtp", + "verification", + "cost-model", + "scheduling" + ], + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", + "paths": [ + "src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md", + "src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/metadata.json", + "src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/source-notes.md", + "src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/validation.md", + "src/inferrsi/data/skill_library/skill-index.json" + ], + "content_sha256": "f157562d1edc905cb4fc07246567e3c5b844efb9fbf5fe39f0d40553a1b1a307" + } + ], + "index_type": "packaged-skill", + "skill_documents": { + "original": "data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md" + }, + "checkout_ref": "checkout:budget-speculative-verification", + "source_files": { + "src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md": "f157562d1edc905cb4fc07246567e3c5b844efb9fbf5fe39f0d40553a1b1a307", + "src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/metadata.json": "d8f76d5a4294e3e951f095e5c5db6c48113a5f2fec12771209db0fb33ec631d9", + "src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/source-notes.md": "6fa955022f16cd598f44fd6521901d1cb868b06cec545cabe2b82769a641b5d6", + "src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/validation.md": "673d93f6700052974b340fab25b3438e0e6344960569ac9abec7d7720f662288", + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" + }, + "skill_metadata": { + "name": "budget-speculative-verification", + "description": "Decide how many draft tokens to verify per decode step when acceptance varies by request and position. Use when an engine already runs a draft head with multi-token speculation and verification cost grows with query rows. Separates draft cost from verify cost, builds a cumulative-acceptance rate vector, fits a cost model, and allocates a per-step row budget.", + "tags": [ + "deepinfer", + "speculative-decoding", + "mtp", + "verification", + "cost-model", + "scheduling" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/deepseek-ai/DeepInfer", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "paths": [ + "csrc/engine/mtp_varlen.hpp", + "csrc/engine/scheduler.hpp", + "deepinfer/worker/worker.py", + "deepinfer/environ.py", + "deepinfer/sampling_params.py", + "deepinfer/model_executor/models/mtp/dflash.py", + "deepinfer/model_executor/layers/sampler.py", + "deepinfer/model_executor/layers/tilelang_ops/fused_sample.py", + "LICENSE" + ], + "readset": "references/source-notes.md" + } + ], + "applicability": { + "scope": "Engines with multi-token speculative decoding whose verification forward cost grows with query rows", + "prerequisites": "A calibrated per-position acceptance signal; a measured (rows, step_time) curve; a handle on metadata construction", + "hardware": "Backend-independent procedure; the reference implementation targets NVIDIA SM100 and Ascend 950 only" + }, + "limitations": [ + "Source-derived candidate; no local efficacy validation and no reproduced gain.", + "The reference budgets verification rows only; draft compute is unchanged.", + "The budgeted path is off by default in the reference source.", + "Budgeting can lower accepted length; equal accepted length is not an acceptance criterion.", + "Per-position independence of the confidence used for verification is a correctness requirement, not only a scheduling convenience.", + "Cross-backend rejection-comparison parity is unproven; the reference has no equality assertion.", + "Numeric acceptance rates and warmup curves are deployment-specific and are not carried here." + ], + "provenance_readsets": [ + "references/source-notes.md", + "references/validation.md" + ], + "references": [], + "consumption_mode": "packaged" + }, + "applicability": { + "scope": "Engines with multi-token speculative decoding whose verification forward cost grows with query rows", + "prerequisites": "A calibrated per-position acceptance signal; a measured (rows, step_time) curve; a handle on metadata construction", + "hardware": "Backend-independent procedure; the reference implementation targets NVIDIA SM100 and Ascend 950 only" + }, + "recommendations": [ + "Read the pinned SKILL.md and its prerequisites before choosing this procedure.", + "Read the packaged references and owner-local cases alongside the procedure." + ], + "limitations": [ + "This generated abstract points to the complete packaged skill; external evidence and upstream repositories remain source links.", + "The imported status describes this index, not the original skill; its declared status is candidate.", + "Reading or indexing does not authorize execution, validate efficacy, promote a candidate or grant a license.", + "The backend categorizes the procedure's hardware dependency; it does not certify compatibility, and skills.discovery is a discovery scenario.", + "Source-derived candidate; no local efficacy validation and no reproduced gain.", + "The reference budgets verification rows only; draft compute is unchanged.", + "The budgeted path is off by default in the reference source.", + "Budgeting can lower accepted length; equal accepted length is not an acceptance criterion.", + "Per-position independence of the confidence used for verification is a correctness requirement, not only a scheduling convenience.", + "Cross-backend rejection-comparison parity is unproven; the reference has no equality assertion.", + "Numeric acceptance rates and warmup curves are deployment-specific and are not carried here." + ], + "verification": { + "locally_verified": false, + "evidence_level": "guidance" + } +} diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/calibrate-serving-simulation.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/calibrate-serving-simulation.json index fcc433765..40d299ee0 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/calibrate-serving-simulation.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/calibrate-serving-simulation.json @@ -16,8 +16,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md", "src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/metadata.json", @@ -36,7 +36,7 @@ "src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md": "ac773c6f1c110a664dfca28a3ae9129b10c07614e60ca3640dea776220e7b352", "src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/metadata.json": "553406b95177a38d73ea6f305c30f361765a437dfe8cfec86c26d4c00e1ac41c", "src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/references/source-notes.md": "9696d3c0ef96b6611787446bd6f96d4d6255ca8d107105de8c1d9af83edb9fa5", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "calibrate-serving-simulation", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/cgroup-resource-controls.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/cgroup-resource-controls.json index 52254def4..851cae750 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/cgroup-resource-controls.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/cgroup-resource-controls.json @@ -25,8 +25,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/host/cgroup-resource-controls/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/host/cgroup-resource-controls/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/host/cgroup-resource-controls/SKILL.md", "src/inferrsi/data/skill_library/hardware/host/cgroup-resource-controls/metadata.json", @@ -43,7 +43,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/host/cgroup-resource-controls/SKILL.md": "fd1f5a7e6a55714730044155f5b38c1d700383d4e16c3e91963bbfbe1c773af5", "src/inferrsi/data/skill_library/hardware/host/cgroup-resource-controls/metadata.json": "e0cf9b8e1b92899da1564df2f2fd5c842fa2113925b188358c4984f5e31712ce", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "cgroup-resource-controls", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/checkpoint-recovery-consistency.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/checkpoint-recovery-consistency.json index 05aada082..9bc97d4ad 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/checkpoint-recovery-consistency.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/checkpoint-recovery-consistency.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/checkpoint-recovery-consistency/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/checkpoint-recovery-consistency/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/checkpoint-recovery-consistency/SKILL.md", "src/inferrsi/data/skill_library/methods/checkpoint-recovery-consistency/metadata.json", @@ -42,7 +42,7 @@ "src/inferrsi/data/skill_library/methods/checkpoint-recovery-consistency/SKILL.md": "30d55d7d0475d95f369f7c9aa3e594406a9407ded2a4e623df0c39cd487ba724", "src/inferrsi/data/skill_library/methods/checkpoint-recovery-consistency/metadata.json": "5c18a225cd3b88051c57cb29d3d9cca3dcf18826aff86cef1cec39f315a1fc1d", "src/inferrsi/data/skill_library/methods/checkpoint-recovery-consistency/references/sources.md": "7c95f3058bbfeda1dd7e83d05917a94be0548350c95d7066dd5bf49cb232775a", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "checkpoint-recovery-consistency", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/choose-parallel-training-plan.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/choose-parallel-training-plan.json index 9f9fad1ec..e9ede79cc 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/choose-parallel-training-plan.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/choose-parallel-training-plan.json @@ -16,8 +16,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/choose-parallel-training-plan/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/choose-parallel-training-plan/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/choose-parallel-training-plan/SKILL.md", "src/inferrsi/data/skill_library/methods/choose-parallel-training-plan/metadata.json", @@ -36,7 +36,7 @@ "src/inferrsi/data/skill_library/methods/choose-parallel-training-plan/SKILL.md": "25d04aed9782b2a8ff1186828becdc7fce93c4418039ceaef9b8dcc5ec9bcd24", "src/inferrsi/data/skill_library/methods/choose-parallel-training-plan/metadata.json": "858fdd88c6bd2f118bd1d527bb37d47bcc2722cc6140d52ee4657e646cdce080", "src/inferrsi/data/skill_library/methods/choose-parallel-training-plan/references/source-notes.md": "9f9184d46b19312138a6e805e439e43dc0594a0cc6b46675255a582dc946ca5b", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "choose-parallel-training-plan", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/cite-check-downstream-pr.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/cite-check-downstream-pr.json index b2cb48f97..15d627a58 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/cite-check-downstream-pr.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/cite-check-downstream-pr.json @@ -16,8 +16,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/workflows/cite-check-downstream-pr/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/workflows/cite-check-downstream-pr/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/skill-index.json", "src/inferrsi/data/skill_library/workflows/cite-check-downstream-pr/SKILL.md" @@ -31,7 +31,7 @@ }, "checkout_ref": "checkout:cite-check-downstream-pr", "source_files": { - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8", + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909", "src/inferrsi/data/skill_library/workflows/cite-check-downstream-pr/SKILL.md": "485932cdf3b1fba5dc69be73e75d38f86a7b79442d26e82a296c963a7598d226" }, "skill_metadata": { diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/claim-contract.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/claim-contract.json index eeb01194c..332c781c7 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/claim-contract.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/claim-contract.json @@ -16,8 +16,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/workflows/claim-contract/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/workflows/claim-contract/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/skill-index.json", "src/inferrsi/data/skill_library/workflows/claim-contract/SKILL.md" @@ -31,7 +31,7 @@ }, "checkout_ref": "checkout:claim-contract", "source_files": { - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8", + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909", "src/inferrsi/data/skill_library/workflows/claim-contract/SKILL.md": "3af1dbbe81d91cb05fc2ef15a2ae25d7592a08558e1a5535f49c52f33a488c55" }, "skill_metadata": { diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/comfy-ui.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/comfy-ui.json index 9919b10ba..688de9d52 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/comfy-ui.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/comfy-ui.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/repositories/comfy-ui/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/repositories/comfy-ui/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/repositories/comfy-ui/LICENSE", "src/inferrsi/data/skill_library/repositories/comfy-ui/SKILL.md", @@ -50,7 +50,7 @@ "src/inferrsi/data/skill_library/repositories/comfy-ui/references/model-residency.md": "014c68386e7cdfdb7b4f566836e8b2ff61bd5a5a506e7b4723f83fe80c5d4de7", "src/inferrsi/data/skill_library/repositories/comfy-ui/references/repo-provenance.md": "d7ca20c50652ef70da198de9216315b98cfbaa28b22bfe036f293a85a8f20261", "src/inferrsi/data/skill_library/repositories/comfy-ui/references/validation-boundaries.md": "1913dedbeb558d872d6a0e015cc11d822910c3cb259a236db29fb60ad8a64263", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "comfy-ui", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/competitor-scan.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/competitor-scan.json index 4a487a9d8..ba1b18f13 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/competitor-scan.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/competitor-scan.json @@ -16,8 +16,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/workflows/competitor-scan/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/workflows/competitor-scan/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/skill-index.json", "src/inferrsi/data/skill_library/workflows/competitor-scan/SKILL.md" @@ -31,7 +31,7 @@ }, "checkout_ref": "checkout:competitor-scan", "source_files": { - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8", + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909", "src/inferrsi/data/skill_library/workflows/competitor-scan/SKILL.md": "0ede58ace65a9324a538b4816add1684f230ba536d4364fe5a5dd2c0bbed3776" }, "skill_metadata": { diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/dcp-async-save-performance.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/dcp-async-save-performance.json index 6e70789e7..f8863f3b9 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/dcp-async-save-performance.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/dcp-async-save-performance.json @@ -23,8 +23,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md", "src/inferrsi/data/skill_library/frameworks/pytorch/dcp-async-save-performance/metadata.json", @@ -43,7 +43,7 @@ "src/inferrsi/data/skill_library/frameworks/pytorch/dcp-async-save-performance/SKILL.md": "645139ee0c88ec865cad952e24cda35f4648d4b777b732c9b7d274b6c7830845", "src/inferrsi/data/skill_library/frameworks/pytorch/dcp-async-save-performance/metadata.json": "2452e400592bc6e084981b27153f6936b5a12e7f3584951cb8e3373e9c27db4e", "src/inferrsi/data/skill_library/frameworks/pytorch/dcp-async-save-performance/references/sources.md": "22afe4f4d77f441096cc657d55d763473bfcbea8fb531c8e493fe49ab84955f7", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "dcp-async-save-performance", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/dense-performance-evidence.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/dense-performance-evidence.json index f06523e4e..636edb9c1 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/dense-performance-evidence.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/dense-performance-evidence.json @@ -21,8 +21,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/dense-performance-evidence/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/dense-performance-evidence/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/dense-performance-evidence/SKILL.md", "src/inferrsi/data/skill_library/methods/dense-performance-evidence/metadata.json", @@ -39,7 +39,7 @@ "source_files": { "src/inferrsi/data/skill_library/methods/dense-performance-evidence/SKILL.md": "53cc1867a13f54d1006d50517dea8848869ca50df3f0cd610b2424cc8d18c933", "src/inferrsi/data/skill_library/methods/dense-performance-evidence/metadata.json": "d8cfc7c93d0a96d5240f085732f3f6a45159c125238a693220687489d6fd1979", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "dense-performance-evidence", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/diagnose-vllm-admission-pressure.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/diagnose-vllm-admission-pressure.json index 7239a0402..1f0ffa97f 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/diagnose-vllm-admission-pressure.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/diagnose-vllm-admission-pressure.json @@ -21,8 +21,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/SKILL.md", "src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/metadata.json", @@ -67,7 +67,7 @@ "src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/loongserve-readset.md": "f1a383e7bdbbd5a5a472d7f6e1292c8a17d4910ff736ca5a6aa422e1bee74f8c", "src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/orca-readset.md": "a497ef23b7e60c0fbc15d95b7e07cd20547cfe56f37c24dfe6ed70d79ef7a3cf", "src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/source-notes.md": "84033fd514773f10c0bddd9914fabb980497ba2b308f226a922c3e1888413827", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "diagnose-vllm-admission-pressure", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/diffusers-invocation-state.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/diffusers-invocation-state.json index d8a5398c9..609c509d8 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/diffusers-invocation-state.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/diffusers-invocation-state.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/frameworks/diffusers/diffusers-invocation-state/SKILL.md", "src/inferrsi/data/skill_library/frameworks/diffusers/diffusers-invocation-state/metadata.json", @@ -46,7 +46,7 @@ "src/inferrsi/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/ownership.md": "40688cc44183027efbf2fca8d3548c299cdf2e33fef54f76ab3549353eb9bd1b", "src/inferrsi/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/source-map.md": "10562a75683bd9bd07ec2e37a1cbd0f93f2534d6f85b55a566adb2d97485215f", "src/inferrsi/data/skill_library/frameworks/diffusers/diffusers-invocation-state/references/validation.md": "5a0b854b87d091d7d505b0bdd5c03330052507a851ab6edf35b93a6675f2d4f7", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "diffusers-invocation-state", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/draft-acceptance-issue.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/draft-acceptance-issue.json index a0ade2cf9..b4679cc71 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/draft-acceptance-issue.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/draft-acceptance-issue.json @@ -16,8 +16,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/workflows/draft-acceptance-issue/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/workflows/draft-acceptance-issue/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/skill-index.json", "src/inferrsi/data/skill_library/workflows/draft-acceptance-issue/SKILL.md" @@ -31,7 +31,7 @@ }, "checkout_ref": "checkout:draft-acceptance-issue", "source_files": { - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8", + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909", "src/inferrsi/data/skill_library/workflows/draft-acceptance-issue/SKILL.md": "ce983acfb7d7d3272775e7cf00c92e4ddadd97511b602575a76e5ab7ade01b56" }, "skill_metadata": { diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/evaluate-elastic-serving-placement.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/evaluate-elastic-serving-placement.json index bdbfeb9d2..5e3dea43d 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/evaluate-elastic-serving-placement.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/evaluate-elastic-serving-placement.json @@ -16,8 +16,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md", "src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/metadata.json", @@ -36,7 +36,7 @@ "src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md": "82d0099d3c33706f46c657d4fa7f76d3d486e019b36760d1de001921328a60cc", "src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/metadata.json": "9efdc98c82784f818994d9edc3ce5411ea38c7904f81ceefa15357dc1c94bd2e", "src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/references/source-notes.md": "ea416831fd7d0874e06d0a76fb0c7196df53072b717673a0cdf635db6bfd917b", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "evaluate-elastic-serving-placement", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/generation-cache-lifecycle.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/generation-cache-lifecycle.json index b5caf63ef..f511dd2be 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/generation-cache-lifecycle.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/generation-cache-lifecycle.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/SKILL.md", "src/inferrsi/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/metadata.json", @@ -48,7 +48,7 @@ "src/inferrsi/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/generation-batch.md": "f16d3c57d45e63bcaffd3871aa41c76972ee11f7dbf1e12b45d9b170303cad9b", "src/inferrsi/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/source-map.md": "e3d815d4307b5eb3a61545ecee6e56208349755fb9a0ee530bb112b90af60e7f", "src/inferrsi/data/skill_library/repositories/mlx-lm/generation-cache-lifecycle/references/validation.md": "73b5dd347017baba519ecc85415aead9993c1751d116b9ba4f07689f58c5f783", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "generation-cache-lifecycle", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/host-memory-management.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/host-memory-management.json index 70c0e1a84..9e58cb43d 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/host-memory-management.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/host-memory-management.json @@ -25,8 +25,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/host/host-memory-management/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/host/host-memory-management/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/host/host-memory-management/SKILL.md", "src/inferrsi/data/skill_library/hardware/host/host-memory-management/metadata.json", @@ -43,7 +43,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/host/host-memory-management/SKILL.md": "b42e565be94d60e0a1557d5ed76a0548e89b8e054dc249c46215e7b90fe4277f", "src/inferrsi/data/skill_library/hardware/host/host-memory-management/metadata.json": "f98060ed3c2ef815832eae549bfb21ca02811ff980a629bf995a5b8e7f48c7cd", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "host-memory-management", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/host-overhead-optimization.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/host-overhead-optimization.json index d2ec56b7a..aed907f1c 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/host-overhead-optimization.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/host-overhead-optimization.json @@ -22,14 +22,15 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md", "src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/metadata.json", + "src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md", "src/inferrsi/data/skill_library/skill-index.json" ], - "content_sha256": "db8357dab79cfa474561732849b5bb153222f63e6688b6f873f45800595f70b8" + "content_sha256": "36ef88826a32ac3147b869001004ded17a8d5a491e25bfe105005ba39ea9d1eb" } ], "index_type": "packaged-skill", @@ -38,9 +39,10 @@ }, "checkout_ref": "checkout:host-overhead-optimization", "source_files": { - "src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md": "db8357dab79cfa474561732849b5bb153222f63e6688b6f873f45800595f70b8", + "src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md": "36ef88826a32ac3147b869001004ded17a8d5a491e25bfe105005ba39ea9d1eb", "src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/metadata.json": "90941490debf1ec9f02310fa033658ec294a8e968783d4bb9d004bcded0f73fe", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md": "f452bf40132c9195d739f1c86b3aa18857138c7ed0f1861af5853d93b9955924", + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "host-overhead-optimization", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/inference-benchmark-design.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/inference-benchmark-design.json index 9b0993654..114bc2c90 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/inference-benchmark-design.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/inference-benchmark-design.json @@ -21,8 +21,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md", "src/inferrsi/data/skill_library/methods/inference-benchmark-design/metadata.json", @@ -39,7 +39,7 @@ "source_files": { "src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md": "dcaa6d7bdcbe4c29116b9679d95ec076faaada3e66c647e95acdd46c2343a2eb", "src/inferrsi/data/skill_library/methods/inference-benchmark-design/metadata.json": "1e04bcca099b0cdbcbdfd8e9c88b47c1accb656cb5317ccb7a929fcacb0a0d87", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "inference-benchmark-design", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/inference-optimization-casebook.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/inference-optimization-casebook.json index 748d51fd2..673468512 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/inference-optimization-casebook.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/inference-optimization-casebook.json @@ -21,8 +21,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/inference-optimization-casebook/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/inference-optimization-casebook/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/inference-optimization-casebook/SKILL.md", "src/inferrsi/data/skill_library/methods/inference-optimization-casebook/cases/admission-drain-occupancy.json", @@ -55,7 +55,7 @@ "src/inferrsi/data/skill_library/methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.json": "a2df4056af372fbbb2fbe90db01acc926fad78775844fa91cff8bc2f3286ff58", "src/inferrsi/data/skill_library/methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.md": "fdcadb4488e29ca45edf21a350ed28ed4f6c6a9fb4772a61d90ca8f62ce5bb04", "src/inferrsi/data/skill_library/methods/inference-optimization-casebook/metadata.json": "6aa0dd8ae9a8b45d770593f236ce39a6c2e06a8e67ad7176b47d484686028f63", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "inference-optimization-casebook", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/kernel-writing-guide.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/kernel-writing-guide.json index 92a313ded..878e97ca1 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/kernel-writing-guide.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/kernel-writing-guide.json @@ -24,8 +24,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/kernel-writing-guide/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/kernel-writing-guide/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/kernel-writing-guide/SKILL.md", "src/inferrsi/data/skill_library/methods/kernel-writing-guide/metadata.json", @@ -42,7 +42,7 @@ "source_files": { "src/inferrsi/data/skill_library/methods/kernel-writing-guide/SKILL.md": "e238a3102b06af1086bb805f1c3f0f24aacaa03032760e0d95231bcf25878dd1", "src/inferrsi/data/skill_library/methods/kernel-writing-guide/metadata.json": "7e82c7b5bcbd695981eaf0ce3ea45fe37dd78c43a73204e1465af5379a205bfb", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "kernel-writing-guide", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/kv-tier-management.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/kv-tier-management.json index dcf51461c..79762907f 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/kv-tier-management.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/kv-tier-management.json @@ -24,8 +24,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/repositories/lmcache/kv-tier-management/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/repositories/lmcache/kv-tier-management/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/repositories/lmcache/kv-tier-management/SKILL.md", "src/inferrsi/data/skill_library/repositories/lmcache/kv-tier-management/metadata.json", @@ -42,7 +42,7 @@ "source_files": { "src/inferrsi/data/skill_library/repositories/lmcache/kv-tier-management/SKILL.md": "e4b78f368d70c8abb74a935acb910414015f1aeec1b50f7a295e836822ec4b27", "src/inferrsi/data/skill_library/repositories/lmcache/kv-tier-management/metadata.json": "20f7989a27a438170e4f59eb6bf0691dec25bb3ea7bcb8cc3d364e48b9a698bd", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "kv-tier-management", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/lazy-evaluation-stream-lifetimes.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/lazy-evaluation-stream-lifetimes.json index dcca4b77b..eb54990a3 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/lazy-evaluation-stream-lifetimes.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/lazy-evaluation-stream-lifetimes.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/SKILL.md", "src/inferrsi/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/metadata.json", @@ -46,7 +46,7 @@ "src/inferrsi/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/execution-lifetime.md": "c4bc41e08aca68f661325f61e6f0db1760a079dc362f295c847bdc374fee3bf6", "src/inferrsi/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/source-map.md": "ccbc70ece4e910b5f3734617cd189fb61b6bb75bd652d2add7a59663d478f83e", "src/inferrsi/data/skill_library/frameworks/mlx/lazy-evaluation-stream-lifetimes/references/validation.md": "ebfc00bf7d6ab0a1dc415953ae84d56ccdc713d5614aff85e433c4a843793bff", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "lazy-evaluation-stream-lifetimes", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/npu-skill-factory.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/npu-skill-factory.json index 14b630ca5..8e9ccdd7f 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/npu-skill-factory.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/npu-skill-factory.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/workflows/npu-skill-factory/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/workflows/npu-skill-factory/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/skill-index.json", "src/inferrsi/data/skill_library/workflows/npu-skill-factory/SKILL.md", @@ -54,7 +54,7 @@ }, "checkout_ref": "checkout:npu-skill-factory", "source_files": { - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8", + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909", "src/inferrsi/data/skill_library/workflows/npu-skill-factory/SKILL.md": "74cd863df2e80dc2bda1713d24207809173a3116f30325e3487e1e3517cfec1e", "src/inferrsi/data/skill_library/workflows/npu-skill-factory/metadata.json": "8d82e5e7bf2eece16056076e1ab888669330c2533fee26fabe747fbb551e1388", "src/inferrsi/data/skill_library/workflows/npu-skill-factory/references/papers-20261015/COMPARISON.md": "38587678a96581f5c2dc697d982127343565504bffa191352811dffcc165aa23", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json index c40c878e3..53b81a324 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json @@ -21,8 +21,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md", "src/inferrsi/data/skill_library/methods/optimization-search-policy/metadata.json", @@ -51,7 +51,7 @@ "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/papers-20261015/knowledge/adaptation-interfaces.md": "5e761957cb5ecd93926e6ffc13deee7c830c6e8f0ee3c3846e9dd3b50dc74446", "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/papers-20261015/references/source-notes.md": "54d83c3713045d59a48821bcf93598374ba4a630e3891ad39498f919317c4fb8", "src/inferrsi/data/skill_library/methods/optimization-search-policy/references/papers-20261015/updates/optimization-search-policy.md": "d3db44fa9a6831b86a02c638b323ed943e69fe256866213f10d68c72d762879e", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "optimization-search-policy", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/os-process-placement.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/os-process-placement.json index d1e298f30..4d9360bdc 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/os-process-placement.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/os-process-placement.json @@ -24,8 +24,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/host/os-process-placement/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/host/os-process-placement/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/host/os-process-placement/SKILL.md", "src/inferrsi/data/skill_library/hardware/host/os-process-placement/metadata.json", @@ -44,7 +44,7 @@ "src/inferrsi/data/skill_library/hardware/host/os-process-placement/SKILL.md": "8d38af7ee1cd08a99bc6981a4ed1891be6331a6a56ea2537364fc0a1b79702b9", "src/inferrsi/data/skill_library/hardware/host/os-process-placement/metadata.json": "88cea21bc171c8ef89b73013672cca7c3af1e15eab00e388dd05b114368c1411", "src/inferrsi/data/skill_library/hardware/host/os-process-placement/references/llama-numa-thread-readset.md": "748b254b3b043a5ddb0f480eb4a566854d27eedd7b111499863e9bf0a5a6951e", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "os-process-placement", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/performance-ceiling-analysis.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/performance-ceiling-analysis.json index 8a7e6157a..edb686c65 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/performance-ceiling-analysis.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/performance-ceiling-analysis.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/SKILL.md", "src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/metadata.json", @@ -72,7 +72,7 @@ "src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/inferencebench.md": "21f47111197a0b7f8d6384ea918d139dc8318a62ca3313ce060048271f50b639", "src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/prior-coverage.md": "35d47ac078591221fcb666219bcd824f9400fc17aafecdc0499d8336f7178238", "src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/references/vidur.md": "d38602af5219f5feeb49e6cd6bf53d66098c77fad141dae676f1d8a6743cf25d", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "performance-ceiling-analysis", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/plan-activation-residency.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/plan-activation-residency.json index f985ba2fa..442289177 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/plan-activation-residency.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/plan-activation-residency.json @@ -16,8 +16,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/plan-activation-residency/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/plan-activation-residency/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/plan-activation-residency/SKILL.md", "src/inferrsi/data/skill_library/methods/plan-activation-residency/metadata.json", @@ -36,7 +36,7 @@ "src/inferrsi/data/skill_library/methods/plan-activation-residency/SKILL.md": "542d8a1e6963f5019986e1fbd79d3a247a045a51487c09313026087ac457c15e", "src/inferrsi/data/skill_library/methods/plan-activation-residency/metadata.json": "858fdd88c6bd2f118bd1d527bb37d47bcc2722cc6140d52ee4657e646cdce080", "src/inferrsi/data/skill_library/methods/plan-activation-residency/references/source-notes.md": "5ff52655cfd06fcf970ed2a7d4d4fe1c31115b102af2d41497de0c74848ed2b3", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "plan-activation-residency", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/repository-evolution-analysis.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/repository-evolution-analysis.json index 92944ecc2..798d19836 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/repository-evolution-analysis.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/repository-evolution-analysis.json @@ -21,14 +21,15 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/repository-evolution-analysis/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/repository-evolution-analysis/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/repository-evolution-analysis/SKILL.md", "src/inferrsi/data/skill_library/methods/repository-evolution-analysis/metadata.json", + "src/inferrsi/data/skill_library/methods/repository-evolution-analysis/references/capability-presence-audit.md", "src/inferrsi/data/skill_library/skill-index.json" ], - "content_sha256": "8cac8e0d52fa5bffdf3076425c45a6041a3fe4402ffd5219abdf23ccaf338f38" + "content_sha256": "7e20d3390f2501263e5e298e026c47cac16c234f2397833131e15ae5d34c0b97" } ], "index_type": "packaged-skill", @@ -37,9 +38,10 @@ }, "checkout_ref": "checkout:repository-evolution-analysis", "source_files": { - "src/inferrsi/data/skill_library/methods/repository-evolution-analysis/SKILL.md": "8cac8e0d52fa5bffdf3076425c45a6041a3fe4402ffd5219abdf23ccaf338f38", + "src/inferrsi/data/skill_library/methods/repository-evolution-analysis/SKILL.md": "7e20d3390f2501263e5e298e026c47cac16c234f2397833131e15ae5d34c0b97", "src/inferrsi/data/skill_library/methods/repository-evolution-analysis/metadata.json": "bbde3cec2545a2613653ec521053f972235742fb53b630badcffd9a61fa5f19f", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/methods/repository-evolution-analysis/references/capability-presence-audit.md": "c21eb4a6aabe811a49a6540a34c54dbcfdcf7efae3052a79b54034c7838026a5", + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "repository-evolution-analysis", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/rsi-evidence.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/rsi-evidence.json index ba8aad6a0..b5c154f39 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/rsi-evidence.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/rsi-evidence.json @@ -16,8 +16,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/workflows/rsi-evidence/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/workflows/rsi-evidence/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/skill-index.json", "src/inferrsi/data/skill_library/workflows/rsi-evidence/SKILL.md" @@ -31,7 +31,7 @@ }, "checkout_ref": "checkout:rsi-evidence", "source_files": { - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8", + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909", "src/inferrsi/data/skill_library/workflows/rsi-evidence/SKILL.md": "a55610874b658b86a39a1a049efb4e4eab0140f6579ed74dd7718a4ca8cedaca" }, "skill_metadata": { diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/skill-evolution-playbook.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/skill-evolution-playbook.json index 79a26a6d9..3fac5e673 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/skill-evolution-playbook.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/skill-evolution-playbook.json @@ -21,8 +21,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/skill-evolution-playbook/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/skill-evolution-playbook/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/skill-evolution-playbook/SKILL.md", "src/inferrsi/data/skill_library/methods/skill-evolution-playbook/metadata.json", @@ -39,7 +39,7 @@ "source_files": { "src/inferrsi/data/skill_library/methods/skill-evolution-playbook/SKILL.md": "9cd37f21fe0d862069e4798edca8a04d59f8b840e55d521901594a63dddd9ba3", "src/inferrsi/data/skill_library/methods/skill-evolution-playbook/metadata.json": "3132b700334856ab8e1eec4396bcb57627e987adf28cb86d5dc7974820ee9968", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "skill-evolution-playbook", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/storage-io-measurement.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/storage-io-measurement.json index 516f1076c..57cb35419 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/storage-io-measurement.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/storage-io-measurement.json @@ -24,8 +24,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/host/storage-io-measurement/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/host/storage-io-measurement/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/host/storage-io-measurement/SKILL.md", "src/inferrsi/data/skill_library/hardware/host/storage-io-measurement/metadata.json", @@ -42,7 +42,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/host/storage-io-measurement/SKILL.md": "63b883c10a8d3e7d918abfc7175b72e2b401ff1e02697f94b8382ec1c8f369b7", "src/inferrsi/data/skill_library/hardware/host/storage-io-measurement/metadata.json": "1d3195ca8ac8e1b0e94acda01863b83046cff7d773e8131e9f3f925150bad66e", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "storage-io-measurement", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/training-input-pipeline.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/training-input-pipeline.json index 8f058b5d0..b2e901530 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/training-input-pipeline.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/training-input-pipeline.json @@ -23,8 +23,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/training-input-pipeline/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/training-input-pipeline/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/training-input-pipeline/SKILL.md", "src/inferrsi/data/skill_library/methods/training-input-pipeline/metadata.json", @@ -43,7 +43,7 @@ "src/inferrsi/data/skill_library/methods/training-input-pipeline/SKILL.md": "64bd2c6b795d966045d1d3cca55976d3d49b08b06df7c7e06a70c2824cf55b7e", "src/inferrsi/data/skill_library/methods/training-input-pipeline/metadata.json": "f971cbd0b9ee12afa896227fbc2625c97d307a250a31634c76611da1977855e8", "src/inferrsi/data/skill_library/methods/training-input-pipeline/references/sources.md": "60e0f3e88655d8e1d5eb4797f39ca4370deda01614f265b8d2ea4559d8ac020f", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "training-input-pipeline", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/training-resource-budget.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/training-resource-budget.json index 545be092d..6ee773595 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/training-resource-budget.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/training-resource-budget.json @@ -23,8 +23,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/training-resource-budget/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/training-resource-budget/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/training-resource-budget/SKILL.md", "src/inferrsi/data/skill_library/methods/training-resource-budget/metadata.json", @@ -57,7 +57,7 @@ "src/inferrsi/data/skill_library/methods/training-resource-budget/references/papers-20261015/references/source-readset.md": "2a5c166767f56fa47e7eba7386f1f36da8cb73702e5b5bbe14d6a456ca9ebfae", "src/inferrsi/data/skill_library/methods/training-resource-budget/references/sources.md": "6d248f58af8b350fc66c22898988d6f451ac60d03bcd805cd6e84df5e55174ac", "src/inferrsi/data/skill_library/methods/training-resource-budget/references/stas-decision-notes.md": "f1a993e6c4ca9a035a2effcaf5500081f227b0cc56cc885a6b22fda0041fc744", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "training-resource-budget", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/weight-loading-optimization.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/weight-loading-optimization.json index 8ef6e455d..6ecb75b8c 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/weight-loading-optimization.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/common/skills/weight-loading-optimization.json @@ -24,8 +24,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/methods/weight-loading-optimization/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/methods/weight-loading-optimization/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/methods/weight-loading-optimization/SKILL.md", "src/inferrsi/data/skill_library/methods/weight-loading-optimization/metadata.json", @@ -44,7 +44,7 @@ "src/inferrsi/data/skill_library/methods/weight-loading-optimization/SKILL.md": "df168c23a6f0fadddc147925a11e9eefc61f340be8e7c01a3fbe3aab002aa490", "src/inferrsi/data/skill_library/methods/weight-loading-optimization/metadata.json": "857481988451065967f9edb8a0c60b16d51508d79dbd37ecc2326fbf95df9348", "src/inferrsi/data/skill_library/methods/weight-loading-optimization/references/llama-loading-readset.md": "6c640f28ee3f0253a17b81461c865cb9f947367002cb75bf72550c4572d02e97", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "weight-loading-optimization", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/cuda/skills/diagnose-attention-bottlenecks.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/cuda/skills/diagnose-attention-bottlenecks.json index 8b0865679..f7fd1ec52 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/cuda/skills/diagnose-attention-bottlenecks.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/cuda/skills/diagnose-attention-bottlenecks.json @@ -16,8 +16,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/SKILL.md", "src/inferrsi/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/metadata.json", @@ -52,7 +52,7 @@ "src/inferrsi/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/cases/partitioning-after-fusion.md": "4d4cabe0991eab554dcfbd4fa9c8199e61fa4767709b0c4cd646e2588bb398a2", "src/inferrsi/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/papers-20261015/knowledge/attention-cost-model-boundaries.md": "5ed8f888443e38ccb72545b6025447aab7b92936f78c151fed680e582126239b", "src/inferrsi/data/skill_library/hardware/cuda/diagnose-attention-bottlenecks/references/sources.md": "53436e0de4e6dda2d29426591bf61f8526378cdc6f9ec7967b88b1bcd4c61e6f", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "diagnose-attention-bottlenecks", diff --git a/src/inferrsi/data/repo_packs/inferrsi/knowledge/cuda/skills/host-overhead-profiling.json b/src/inferrsi/data/repo_packs/inferrsi/knowledge/cuda/skills/host-overhead-profiling.json index a22c94180..3b76b3046 100644 --- a/src/inferrsi/data/repo_packs/inferrsi/knowledge/cuda/skills/host-overhead-profiling.json +++ b/src/inferrsi/data/repo_packs/inferrsi/knowledge/cuda/skills/host-overhead-profiling.json @@ -22,8 +22,8 @@ "sources": [ { "type": "git", - "url": "https://github.com/InferMatrixLab/InferRSI/blob/b5178de4f8e279b301614dd8a613ebd1c1985984/src/inferrsi/data/skill_library/hardware/cuda/host-overhead-profiling/SKILL.md", - "commit": "b5178de4f8e279b301614dd8a613ebd1c1985984", + "url": "https://github.com/InferMatrixLab/InferRSI/blob/a704774293ea320b0a2f1f5229d61d7b307fc089/src/inferrsi/data/skill_library/hardware/cuda/host-overhead-profiling/SKILL.md", + "commit": "a704774293ea320b0a2f1f5229d61d7b307fc089", "paths": [ "src/inferrsi/data/skill_library/hardware/cuda/host-overhead-profiling/SKILL.md", "src/inferrsi/data/skill_library/hardware/cuda/host-overhead-profiling/metadata.json", @@ -40,7 +40,7 @@ "source_files": { "src/inferrsi/data/skill_library/hardware/cuda/host-overhead-profiling/SKILL.md": "326e566fbe30d3ba04e4b50c3e87aad9df037f0ac4ad6fe129165ae97dc5f91d", "src/inferrsi/data/skill_library/hardware/cuda/host-overhead-profiling/metadata.json": "e89f1fbbfdf63b666d42adec7826eb4b4a80428b288e476ddb9c977d9aeb3ab2", - "src/inferrsi/data/skill_library/skill-index.json": "08010d947578cf7d16b5e3cd1288762c05a3a36e0816b794105dd2036a3c53c8" + "src/inferrsi/data/skill_library/skill-index.json": "326cafc7a19ae61582f696c169336ba93166bba47a952048afab3a5f9039a909" }, "skill_metadata": { "name": "host-overhead-profiling", diff --git a/src/inferrsi/data/skill_library/README.md b/src/inferrsi/data/skill_library/README.md index cfb9f22f2..fb3626568 100644 --- a/src/inferrsi/data/skill_library/README.md +++ b/src/inferrsi/data/skill_library/README.md @@ -1,6 +1,6 @@ # InferRSI skill library -The English-only library contains **76 skill documents**: 55 registered skills +The English-only library contains **78 skill documents**: 57 registered skills and 21 engine and research-router skills. Complete procedures, metadata, references, cases and source notices ship with the package. @@ -20,7 +20,7 @@ existing resource directory. ## Read registered skills -[`skill-index.json`](skill-index.json) maps 55 stable skill IDs to their canonical +[`skill-index.json`](skill-index.json) maps 57 stable skill IDs to their canonical paths and backend scopes. Install the optional `skills` extra for these commands: ```bash @@ -53,7 +53,7 @@ loading and host placement remain with their existing owners. Five repository entry skills, 15 focused sub-skills and one research router provide source-grounded procedures. These 21 documents are available through -the links and catalog records below, separately from the 55-ID registry. +the links and catalog records below, separately from the 57-ID registry. The registered ComfyUI and stable-diffusion.cpp audits are listed alongside these imports and use generated abstracts rather than runtime repository packs. diff --git a/src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md b/src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md index 59c43e421..6c7ab365d 100644 --- a/src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md +++ b/src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/SKILL.md @@ -49,6 +49,13 @@ needs matched full-stack measurement before a service claim. Re-profile and reconsider batching/instance configuration after a change, under the same budget and all SLOs. Reading this skill starts no autonomous loop or deployment. +For a worked instance of moving host prep rather than shrinking it, see +[zero-overhead scheduling in DeepInfer](references/deepinfer-zero-overhead-scheduling.md): +the previous step's post-process is deferred to overlap the next forward, at the +cost of enumerating and repairing the three quantities that become one step +stale. It is the alternative to this skill's line-profiling loop when the prep +is Python-bound and the device has slack. + ## Profiling Setup ### line_profiler (Primary Method) diff --git a/src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md b/src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md new file mode 100644 index 000000000..470fddeb7 --- /dev/null +++ b/src/inferrsi/data/skill_library/hardware/host/host-overhead-optimization/references/deepinfer-zero-overhead-scheduling.md @@ -0,0 +1,82 @@ +# Case: zero-overhead scheduling in DeepInfer + +A worked, source-level instance of hiding exposed host prep behind device execution. It is the same trade-off this skill's loop optimizes, applied one layer down: instead of making the Python cheaper, it moves the Python so it runs while the GPU is busy. + +Source: [deepseek-ai/DeepInfer @ 8b16149741a4935e1dec8cea27853f273b374c5b](https://github.com/deepseek-ai/DeepInfer/tree/8b16149741a4935e1dec8cea27853f273b374c5b) (Apache-2.0), read statically. Nothing was built or run. **F** is observed source behaviour at this pin; **P** is proposed. The bounded readset is the cited worker, engine, scheduler, remap, constructor and synchronization paths; it does not establish native preemption safety or an end-to-end speedup. + +## What the bubble is + +A decode step alternates device work with CPU work, and the CPU work must consume a device-to-host result from the step it post-processes. The naive order is: + +```text +forward(N) -> sample(N) -> D2H -> post-process(N) -> schedule(N+1) -> forward(N+1) +``` + +The D2H wait plus post-process plus schedule sit between two device launches, so the GPU idles for their duration. That is exposed host prep: the same phenomenon this skill measures before deciding whether to optimize or redirect. + +## The reorder + +```text +forward(N) -> post-process(N-1) -> sample(N) -> [defer post-process(N)] +``` + +`F`: [deepinfer/worker/worker.py:182-187](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/worker.py#L182-L187). Prompt steps never defer; only decode steps, and only when `DEEPINFER_ZERO_OVERHEAD_SCHEDULING` is on: + +```python +if is_prompt or not envs.DEEPINFER_ZERO_OVERHEAD_SCHEDULING: + request_outputs = np.concatenate([request_outputs, self._post_process_step(curr_step_state)]) + self.prev_step_state = None +else: + self.prev_step_state = curr_step_state +``` + +After launching the next forward, the worker post-processes the retained previous state ([worker.py:113-144](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/worker.py#L113-L144)). This is the source's intended overlap; its duration was not measured here. + +## What becomes stale, and the repair + +This is the part worth copying. Three quantities the scheduler reads are now one step behind: + +| Quantity | Why it is stale | Repair | +|---|---|---| +| Per-request query length | Under adaptive verification the per-request split uses previous-step confidence estimates | Redistributed on device from those estimates while preserving the CPU-selected total row budget | +| Per-request key length | Rejected drafts mean the previous step accepted fewer tokens than it requested | Recomputed on device from previous-step accepted counts | +| Input token ids | Rows continuing from the previous batch reuse the previous step's sampled ids | Scatter from the previous batch via a remap | + +`F`: [worker.py:191-245](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/worker.py#L191-L245). The remap matches current batch slots to previous batch slots by stable sequence id; `-1` marks a fresh row. Equal batch order produces an identity remap; otherwise an ID lookup builds it. The correction path is skipped when there is no overlap, not merely when order matches ([csrc/engine/zos.cpp:142-175](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/zos.cpp#L142-L175)). + +```python +if envs.DEEPINFER_ENABLE_MTP_VARLEN: + compute_seqlens_q_by_confidence_rates(total_q, remap[:n_seq], prev.sampler_output.confidence_rates_gpu, seqlens_q[:n_seq]) +if self.nextn > 0: + compute_seqlens_k_zos(...) +``` + +Input-token repair consumes that remap and the previous sampled/draft IDs in [prepare_tensor_input_zos.py:235-244](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/input_metadata/prepare_tensor_input_zos.py#L235-L244), with fresh rows prepared in [zos.cpp:104-113](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/zos.cpp#L104-L113). Corrected query lengths feed the main forward, then the worker synchronizes their D2H metadata before current-step sampling ([worker.py:118-152](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/worker.py#L118-L152)). + +Note the dependency this creates: the deferred path is only correct because the *next* step repairs what the deferred post-process would have produced. If your engine cannot enumerate the stale quantities, deferring the post-process silently produces wrong metadata rather than a crash. + +## Conditions and their source support + +1. **Keep host scheduling independent of the device work being hidden.** The inspected [Scheduler::_schedule releases the GIL](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/scheduler.hpp#L275-L277). This permits other Python work; it is not by itself a proof that every scheduler branch is free of device synchronization. +2. **The standard ZOS constructor selects a host DP group.** [LLMEngine._create_scheduler selects cpu_data_parallel_group when ZOS is enabled](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/llm_engine.py#L307-L314); [that group is created with Gloo](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/parallel_utils/parallel_state.py#L89-L94). The scheduler [recognizes Gloo](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/scheduler.hpp#L109-L111) and [uses a CPU tensor for its synchronous vote](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/scheduler.hpp#L169-L181). The alternate group path can use the default device; do not generalize the host property to arbitrary scheduler construction. +3. **The launch scope has a synchronization debug guard.** Forward and sample launches use `forbid_cuda_sync()` ([model_runner.py:321-324](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/model_runner.py#L321-L324), [356-359](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/model_runner.py#L356-L359)). It targets torch-visible CPU-GPU synchronization, but Event.synchronize is outside its coverage, another thread's allow scope lifts enforcement, and DEEPINFER_DISABLE_SYNC_GUARD disables it ([utils.py:119-132](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/utils.py#L119-L132)). Guard presence is not a measured absence of synchronization. + +## The cost + +- One extra step of retained metadata, request outputs, state slots and KV blocks. +- A device-side repair path that must be maintained alongside the scheduler. +- Deferred output delivery needs a final drain; the worker supplies [flush_inflight_reqs](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/worker.py#L274-L283). End-to-end output latency was not measured. +- The standard ZOS path's CPU DP group must remain available; a custom construction must preserve the intended host/device ordering. + +## Relevance to this skill + +Use this case to interpret the exposed-prep metrics rather than to replace the loop. A large exposed-prep ratio with a large inter-step gap is the signal; the options are to make the prep cheaper (this skill's line-profiling loop) or to move it (this case). The second is cheaper to implement when the prep is Python-bound and the device has slack, which is exactly the condition a profile should show first. + +Do not copy the repair list. It is specific to engines where speculation width and accepted length feed back into the next step's row layout. + +## Unresolved + +- **F — preemption bookkeeping has guards.** [The scheduler rejects COMPLETION recompute configuration](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/scheduler.hpp#L98-L104). Its [_preempt path either stops the request or resets it to WAITING for recompute](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/scheduler.hpp#L766-L789). Deferred decode output [skips WAITING requests](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/scheduler.hpp#L917-L925), as does [MTP post-processing](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/scheduler.hpp#L1212-L1224). Therefore "no guard found" is not an accurate description of this readset. +- **F — some resource release is explicitly deferred.** [Worker post-processing synchronizes the step's sample event before its callback](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/worker.py#L259-L271); the engine [collects sequence metadata after post-processing](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/llm_engine.py#L1010-L1015). [DeferredRelease releases collected metadata/output objects and waits for a clone event before unpinning clone resources](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/llm_engine.py#L89-L116). These are specific lifetimes, not a proof about every state-slot or KV-block reuse path. +- **HYPOTHESIS / not established — device resource reuse under preemption.** An unsafe reuse claim would need a concrete in-flight write, release/reallocation path and ordering that allows them to conflict. No native experiment was run; the bounded source readset establishes neither such a reachable race nor complete preemption safety. Preserve it as an unvalidated lifetime question, not a known bug or a verdict that ZOS is unsafe. +- **The default is constructor-dependent.** [EnvVar.bool defaults false](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/environ.py#L69-L71), while [EngineArgs sets the unset variable and rearms it](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/arg_utils.py#L48-L55) from [NORMAL/COMPLETION true, PROMPT false](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/engine_type.py#L15-L17). This supports the README's default description for the standard NORMAL/COMPLETION path without treating the standalone environment default as a contradiction. Explicit environment overrides remain effective. diff --git a/src/inferrsi/data/skill_library/methods/repository-evolution-analysis/SKILL.md b/src/inferrsi/data/skill_library/methods/repository-evolution-analysis/SKILL.md index 9ddd92c69..426fb7324 100644 --- a/src/inferrsi/data/skill_library/methods/repository-evolution-analysis/SKILL.md +++ b/src/inferrsi/data/skill_library/methods/repository-evolution-analysis/SKILL.md @@ -155,6 +155,15 @@ When adding a new provider, adapter, or capability class, follow the existing registration pattern — do not introduce a new abstraction unless 2+ concrete implementations already exist. +## Auditing a claimed capability + +When a release note or README claims a capability and you need to state whether +it runs at a given revision, use +[capability presence audit](references/capability-presence-audit.md). It classifies +each claim into implemented, hook, documented only or contradicted, and requires +the enabling gate for anything not implemented. This is narrower than the +capability model above: it answers presence and wiring, not speed. + ## Temporal-split benchmark construction (LearningToCommit method) For skill evaluation, construct historical tasks with a hard temporal cutoff. diff --git a/src/inferrsi/data/skill_library/methods/repository-evolution-analysis/references/capability-presence-audit.md b/src/inferrsi/data/skill_library/methods/repository-evolution-analysis/references/capability-presence-audit.md new file mode 100644 index 000000000..ebd0c9c62 --- /dev/null +++ b/src/inferrsi/data/skill_library/methods/repository-evolution-analysis/references/capability-presence-audit.md @@ -0,0 +1,72 @@ +# Capability presence audit + +A procedure for answering one narrow question: **does the claimed capability actually run at this revision, and through what gate?** It is narrower than a capability-family analysis and wider than reading a README. + +Use this reference when a release note, README or vendor document lists features and you must state what the code does. Do not use it for performance claims — this classifies presence and wiring, not speed. + +## The four buckets + +Every claim lands in exactly one: + +| Bucket | Test | Typical evidence | +|---|---|---| +| **Implemented** | A reachable call path from the public entrypoint changes behaviour | A caller that is not gated off, plus a body that is not empty | +| **Hook** | A symbol exists and is called, but its body is empty, or its caller is gated off by a value that never occurs | Empty virtual method with no subclass; a body behind a constant-false gate | +| **Documented only** | No symbol, or symbols only in tests and docs | A tree-wide search that returns nothing in the called graph | +| **Contradicted** | The code does something different from the claim | A default that inverts the claim; a backend that asserts instead of degrading | + +A claim that fits two buckets is under-specified. Split it before classifying: "asynchronous scheduling is enabled by default" is two claims, one about existence and one about the default. + +## Procedure + +1. **Record the claim verbatim.** Do not paraphrase into something easier to confirm. Note whether it is a feature list, a hardware list, or an exclusion list. Exclusion lists are the easiest to audit and the most informative: a README that says "does not ship X" is making a falsifiable claim. + +2. **Find the entrypoint that would exercise the claim.** Follow from the public entry, not from the directory that shares the claim's name. Name-adjacency is the most common false positive. + +3. **Classify.** Use the table above. + +4. **For every hook, find the gate.** Ask what would have to be true for the body to run, then check whether that condition can be true in this build. An empty virtual method plus no subclass is a hook. A real body behind a flag whose value is a constant is also a hook. A flag that defaults false is *not* a hook if the documentation says it is optional. + +5. **Check for subclasses and registries.** A base class with virtual methods proves nothing. Search for derived types and for registration. Record the command shape, because "no subclass found" is a claim about a search. + +6. **Check defaults, not just existence.** A feature can be present and inert. Read the environment-variable or configuration default and quote it. Then check whether some other construction path overrides it — a `setdefault` in a dataclass `__post_init__` can make a variable default-false in one sense and default-true in another. + +7. **Separate the four claim strengths.** Exists, is correct, is fast, is end-to-end usable. Never let one imply another. + +8. **State the extraction boundary.** Say explicitly whether the tree is complete. Only a complete tree lets "absent" mean absent; a sparse checkout licenses only "not located in the inspected paths". + +## Worked instance + +Applied to [deepseek-ai/DeepInfer @ 8b16149741a4935e1dec8cea27853f273b374c5b](https://github.com/deepseek-ai/DeepInfer/tree/8b16149741a4935e1dec8cea27853f273b374c5b), by static source reading only. The supplied source-reading archive includes the parent and six direct vendor snapshots, but omits nine listed media files and leaves ten nested vendor gitlinks unexpanded. Its untruncated upstream tree inventory describes the snapshot before those omissions; it does not make the delivered archive a complete checkout or build environment. The observations below are bounded to the named parent-source paths, not an absence proof over omitted or external code. + +| Claim (verbatim from README) | Bucket | Deciding evidence | +|---|---|---| +| "Asynchronous scheduling (zero-overhead batch scheduler)" / "Enabled by default." | **Implemented**; constructor-dependent default | [The worker defers decode post-processing](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/worker.py#L182-L187). [EngineArgs uses setdefault and rearms the variable](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/arg_utils.py#L48-L55), with [NORMAL/COMPLETION true and PROMPT false](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/engine_type.py#L15-L17). The standalone [EnvVar.bool default is false](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/environ.py#L69-L71). These construction contexts are not contradictory; an explicit environment value is preserved. | +| "Parallelism: Attention DP / balanced CP, MoE EP + EPLB" — the EPLB rearrangement part | **Implemented**, opt-in; not a Hook | [Model sampling calls the EPLB manager](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/models/deepseek.py#L402-L409), which [calls each balancer](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/eplb/__init__.py#L378-L384). Its [real update/rearrange path](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/eplb/__init__.py#L131-L165) returns when the interval is zero and skips EP size one. The [interval defaults to zero](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/environ.py#L111-L117), using the [integer default](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/environ.py#L62-L63). This disables dynamic rearrangement by default, not the implementation or all load statistics. | +| "A KV cache system" — under the README's complete-service exclusion heading | Service save-integration **Hook**; in-engine KV **Implemented** | [README:38-42](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/README.md#L38-L42) excludes components needed for a complete service. The inspected [save-integration methods have no implementation](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/llm_engine.py#L815-L822); this does not mean device KV is missing: [CacheEngine allocates KV/state storage and constructs GPUCacheInterface](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/cache_engine.py#L918-L966). | +| "VE/Prefill/Decode disaggregation (most required engine functionality is already implemented)" — the same service exclusion list | **Hook** at the inspected integration seam | [init_pd_split](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/llm_engine.py#L800-L809), [send_processed_prompts and post_process_model_outputs](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/llm_engine.py#L848-L858) have no implementation in the base engine. The bounded parent-source search described below located no subclass; this does not negate the README's statement that supporting engine functionality exists. | +| "Load balancing across data-parallel replicas for both prefill and decode" — the same service exclusion list | **Documented only** within this readset | [README:38-42](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/README.md#L38-L42) states the service exclusion. This bounded hook/default audit did not establish an application request router or audit omitted/external implementations; it makes no stronger absence claim. | + +Two subtleties the exercise surfaced: + +- **`finalize_seq` is not empty.** It is inside the same virtual block but [calls scheduler.free_seq](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/llm_engine.py#L804-L806). So the hook block is heterogeneous: an implementation inheriting it would get partial behaviour rather than nothing. A blanket "all these are empty" statement would be wrong. +- **A body can exist behind an inactive base-class gate.** [do_copy_state](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/llm_engine.py#L787-L798) performs a copy only when its buffer exists, while [should_copy_state returns False](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/llm_engine.py#L811-L813) in the inspected base engine. Body-present alone is not evidence that this entrypoint executes it. + +## Pitfalls + +- **Name-adjacency.** A directory or class named after the feature is not evidence that the feature runs. In the audited source, three claimed-absent components were present as named empty methods — the name was *more* misleading than absence would have been. +- **Body-present-is-not-reachable.** See the state-copy case above. +- **Flag defaults.** EPLB dynamic rearrangement is opt-in, whereas ZOS is enabled by the standard NORMAL/COMPLETION EngineArgs constructor when no environment override exists. Reading the variable's standalone default alone gives the wrong answer for that entrypoint. +- **Hardware lists.** A supported-device list is not a compatibility guarantee. Check whether unsupported cases assert or degrade silently; in the audited source one property hard-asserted on the non-NVIDIA backend. +- **Documentation that describes the deployment, not the code.** A production Q&A can truthfully describe a shape the open release does not implement. Label it DOC and keep it separate from CODE findings — both statements can be true. +- **Comments as evidence, carefully.** A source comment saying "virtual method, handles finished requests" is the author's description, not the behaviour. Read the body. The same applies in reverse: a comment stating a correctness requirement (as `mtp_varlen.hpp:160` does) is evidence of intent that must then be checked against the code that follows it. +- **Sparse checkouts.** Absence in a curated skill-doc checkout proves nothing. Say so rather than reporting a feature as missing. + +## Output + +A table with one row per claim: claim verbatim, bucket, deciding evidence, the gate that would enable it, residual uncertainty. Then a short list of findings that change how the release may be described. + +## Evidence this instance produced + +- The full-SHA path/line links above distinguish feature code, constructor defaults and the complete-service exclusion list; none is native execution evidence. +- Bounded search: `rg --no-ignore -n -F LLMEngine deepinfer tests`, plus class-declaration inspection, at the stated pin. The located declaration is [llm_engine.py:130](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/engine/llm_engine.py#L130); the public entry [constructs that engine through EngineArgs](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/entrypoints/llm.py#L77-L84). No subclass was located in those two inspected directories. Vendors, unexpanded gitlinks and external deployment code are outside this search. diff --git a/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md b/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md new file mode 100644 index 000000000..c33c55f98 --- /dev/null +++ b/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/SKILL.md @@ -0,0 +1,109 @@ +--- +name: bound-windowed-attention-replay +description: Bound replay or prefill query rows across a producer/consumer boundary whose retained inputs and separately preserved historical state satisfy the full dependency closure. Use when a model has shared hidden state or preserved-compressed KV and the retained region must be derived rather than assumed. Derives the retain rule from window plus rollback depth, separates query-row compaction from dropping historical keys, and names what becomes unrecoverable. +--- + +# Bound a windowed-attention replay + +## Trigger and boundary + +Use when all of the following hold: + +- The architecture has an explicit boundary: layers before it run over the full prefix, while layers after it compute a bounded suffix of query rows. Historical keys or other full-layout state may remain preserved separately. A YoCo-style split, a shared-hidden design, or a preserved-compressed-KV design can qualify after deriving its dependencies. +- That boundary is where replay cost concentrates, so shortening the post-boundary query is what saves the work. +- You control metadata construction, not only the attention kernel. + +Do **not** use when: + +- A retained query needs a value unavailable from retained producer outputs or explicitly preserved historical state, and that value cannot be reconstructed. Reading a historical key outside the retained query suffix is allowed when that key is preserved. Derive the dependency first (step 1) and stop if you cannot establish it. +- The request needs per-token outputs for the clipped region, such as prompt logprobs. +- The system relies on bit-identical results across preemption, because recompute moves the retained rectangle's anchor. + +**Model dependency.** The retain arithmetic `keep = window + rollback` is a small, transferable identity. What does not transfer is everything that makes it *safe*: which layers are producers, which are consumers, where the boundary sits, whether the consumer's KV is compressed or shared across layers, and the per-model window and ring sizes. Treat this procedure as a template for deriving those answers, not as a formula to apply. + +## Status and source + +Status: **candidate**. Empirical validation: **not_run**. + +Source: `deepseek-ai/DeepInfer` @ `8b16149741a4935e1dec8cea27853f273b374c5b` (Apache-2.0), read statically; no equivalence run was performed here. **F** is a fact observed in the pinned source; **P** is proposed procedure. + +Read [source-notes.md](references/source-notes.md) for the evidence ledger and [validation.md](references/validation.md) for the unrun checks. + +## Required inputs + +1. **The producer/consumer split.** Which layers run over the full prefix and write the persistent cache; which layers run compact query rows and which historical state they still read. Read the layer construction, not the config comment. +2. **The dependency closure.** For each consumer, what it reads and where that was produced. Follow it transitively: a consumer reading a shared hidden that was itself computed from a compressed KV needs both accounted for. +3. The window and rollback depth that apply at the boundary. +4. The boundary layer index, or a rule that identifies it. +5. Whether consumer KV is compressed, shared across layers, or per-layer, and whether the historical keys are preserved in full. + +## Procedure + +1. **Derive the dependency closure before touching any window arithmetic.** Enumerate, per consumer layer, every producer it reads: previous-layer hidden state, a shared hidden captured at the boundary, a compressed KV written by an earlier layer, a top-k index pool. For each retained query, establish whether its inputs are available from retained producer outputs or explicitly preserved historical state. Stop if a required value was discarded and cannot be reconstructed. One layer's finite window does not establish the whole stack's dependency horizon; dependencies can extend across layers. + +2. **Confirm the boundary hosts a full-length write.** The architecture has to preserve, at the boundary, the state the consumer needs at full length. In the reference this is the captured full hidden plus the main/index KV stored at full length during the compact query's forward (`F`: `deepinfer/model_executor/models/v4_modules/attn.py:672-719`, `decoder_layer.py:134-141`). If no such full-length write exists, shortening the query changes the cache and the procedure does not apply. + +3. **Compute the retain length.** `keep = window + max_rollback`. Enforce a floor on `max_rollback` equal to the speculation width plus one, so a full-width rollback after a full-width draft still leaves `window` tokens (`F`: `deepinfer/environ.py:132`, `max(value, USE_NEXTN_DECODE + 1)`). + +4. **Enforce `keep` at every entry.** The value can arrive through an environment variable, so a single check is not enough. In the reference it is checked at model-metadata construction (`F`: `deepinfer/model_executor/model_metadata_utils.py:91-92`), at worker setup (`F`: `deepinfer/worker/model_runner.py:107-114`) and in the C++ metadata builder (`F`: `csrc/engine/input_metadata/prepare_tensor_input_v4.cpp:350-357`). + +5. **Define the visible region explicitly.** For a query at absolute position `p`, visible keys are `[max(p − window + 1, rect_lo, valid_start), p]`, where `rect_lo = data_len − keep_tokens` is the retained rectangle's lower bound and `valid_start` guards against attending to zeroed state after a recompute. Clamp and assert bounds; an out-of-range start slot is a silent correctness bug (`F`: `prepare_tensor_input_v4.cpp:155-166`). + +6. **Separate query-row compaction from historical-key retention.** These are two independent decisions and the reference makes only the first: + - **Query-row compaction** removes query rows whose visible region is empty. That is where the compute saving comes from. + - **Historical-key retention** is a separate question. A consumer may legitimately keep full-layout keys in a compressed or shared index while computing over fewer query rows. Compact queries are not evidence that historical keys were dropped. + + Do not state or assume that whole-context compressed attention is incompatible with this procedure. A retained query may read keys outside the retained query region when the required historical state is preserved. Preserving keys alone does not establish that every other hidden-state dependency is available; step 1 must still account for those inputs. + +7. **Decide the mode, and keep both.** Provide a diagnostic mode that keeps all rows and only narrows visibility, and a saving mode that drops the clipped rows. In the reference these are `mask` and `compact` (`F`: `deepinfer/environ.py:188`). Make the diagnostic one the target of the equivalence test, because it is compute-neutral and isolates the visibility math from the gather. + +8. **Preserve the full layout while building the compact copy.** At the boundary, compute the full-length representation the consumer needs before or alongside producing the compact copy. In the reference the boundary helper returns both the compact inputs and the original full hidden/residual, and the caller stores the full hidden into the cross-layer runtime before continuing with the compact values (`F`: `deepinfer/model_executor/models/v4_modules/yoco.py:93-122`, `deepinfer/model_executor/models/decoder_layer.py:134-141`). Note the shape of this requirement: it is "the full-layout value must exist and be stored", not a global ordering that KV writes precede a gather. + +9. **Handle a fully clipped chunk.** If a chunk lies entirely before the rectangle, keep one row so a sampling position always exists. The reference does this in the clip computation itself, by clamping the clip prefix to `q_len − 1` rather than `q_len` (`F`: `prepare_tensor_input_v4.cpp:225`). + +10. **Disable what becomes invalid.** Prompt logprobs cannot be reconstructed from a clipped region. The reference forces them off with a warning rather than rejecting the request (`F`: `deepinfer/engine/llm_engine.py:529-537`). Note the actual behaviour: the request proceeds with `prompt_logprobs = False`. If your engine must reject instead, that is a policy change, not a description of this source. + +11. **State the anchor.** Define the rectangle relative to `prompt + generated` length, and document that preemption-and-recompute moves it. + +## Expected outputs + +- A dependency closure listing, per consumer, what it reads and where that was produced. +- A retain length, with every point that enforces it. +- Per-request compact metadata: query count, position ids, store slots and visible-range indices. +- A statement of whether historical keys are preserved, compacted, or compressed, kept separate from the query-compaction decision. +- A diagnostic mode usable as the equivalence reference. + +## Evidence to collect + +- **Metadata equivalence**, using a matching built extension without model weights: visible-key sets, store slots and compressed-key ranges between diagnostic and saving modes at chunk boundaries. Runtime is not measured here. +- **End-to-end equivalence** on a fixed teacher-forcing trajectory: logprobs and top-k compared element-wise. +- **Prompt-logprob handling**: confirm the actual behaviour — forced off with a warning, or rejected — rather than assuming either. +- **Preemption count during the comparison run**: must be zero, or the results are not comparable. + +## Preconditions and caveats + +- **The dependency closure is the real gate.** A stack of windowed layers is not automatically safe to shorten; only a demonstrated boundary dependency is. +- **The retain rule is not a suggestion.** Below `window + rollback`, the decoder sees fewer than `window` tokens after a rollback. The failure can be loud (an assert) or numerical (diagnostic mode), depending on the mode. +- **Diagnostic mode is compute-neutral.** Do not report it as an optimization. +- **Recomputed SWA state before the valid rectangle is unavailable.** Cache reuse and prefill-to-decode transfer must respect that validity boundary. This does not invalidate the separately preserved full-layout main/index KV. +- **Ring sizing has nested gates.** The inference ring is `window + rollback` per sequence when `window > 0`, otherwise zero (`F`: `deepinfer/worker/cache_engine.py:88-95`). The *prefill/kvstore* ring adds margins only when `DEEPINFER_PREFILL_STORE_EXTRA_STATE` is set: that branch adds the store-granularity margin, plus the truncation keep margin when truncation mode is not `off` (`F`: `cache_engine.py:97-110`). The keep margin therefore requires both gates. Do not describe the larger ring as unconditional. +- **Chunk boundaries do not align with store boundaries.** Prefill chunks do not land on store boundaries, so a longer ring is required to reassemble a complete window at a store point. +- **Intermediate chunk samples are discarded.** Only the last prompt chunk's last row produces the token. +- **Config may not be authoritative.** If the window comes from a config the parser did not read, the check compares against a wrong value. + +## Stop conditions + +- Stop if the dependency closure shows any consumer reading state produced outside the retained region and no full-length copy of that state is preserved. +- Stop if you cannot capture the full-layout representation at the boundary. +- Stop if the comparison run preempts; the anchor moved and the result is not evidence. + +## Configuration in the reference source + +| Variable | Default | Effect | +|---|---|---| +| `DEEPINFER_YOCO_PREFILL_TRUNCATION_MODE` | `off` | `off` / `mask` (diagnostic, compute-neutral) / `compact` (removes clipped query rows) | +| `DEEPINFER_YOCO_PREFILL_KEEP_TOKENS` | `0` (must be set when truncation is on) | Retained trailing query count. The `> 0` and `>= window + rollback` checks apply only when the mode is not `off`; with `off` the value is unused. | +| `DEEPINFER_MAX_BACKTRACKING_TOKENS` | `max(value, nextn + 1)` | Rollback depth folded into the retain rule | +| `DEEPINFER_PREFILL_STORE_EXTRA_STATE` | unset → false | Gates both kvstore-ring margins; the keep margin also requires truncation mode not `off` | + +Truncation is **off by default** in the reference. Enabling it without setting the keep length fails an assertion rather than silently clipping. diff --git a/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/metadata.json b/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/metadata.json new file mode 100644 index 000000000..c6e17c30d --- /dev/null +++ b/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/metadata.json @@ -0,0 +1,58 @@ +{ + "tags": [ + "deepinfer", + "sliding-window-attention", + "prefix-replay", + "kv-cache", + "metadata", + "correctness" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/deepseek-ai/DeepInfer", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "paths": [ + "csrc/engine/input_metadata/prepare_tensor_input_v4.cpp", + "deepinfer/engine/llm_engine.py", + "deepinfer/model_executor/input_metadata/yoco.py", + "deepinfer/model_executor/model_metadata_utils.py", + "deepinfer/model_executor/models/decoder_layer.py", + "deepinfer/model_executor/models/v4_modules/attn.py", + "deepinfer/model_executor/models/v4_modules/yoco.py", + "deepinfer/worker/cache_engine.py", + "deepinfer/worker/model_runner.py", + "deepinfer/environ.py", + "LICENSE" + ], + "readset": "references/source-notes.md" + } + ], + "applicability": { + "scope": "Decoders with an explicit producer/consumer boundary and a demonstrated dependency closure for compact query rows, including separately preserved historical state", + "prerequisites": "A derived dependency closure; a full-length KV write at the boundary; known rollback depth; control over metadata construction", + "hardware": "Backend-independent retain arithmetic; window sizes, boundary index and ring lengths come from the model config and do not transfer" + }, + "limitations": [ + "Source-derived candidate; no equivalence run was performed.", + "Applies to architectures with a demonstrated boundary dependency, not to arbitrary sliding-window stacks.", + "Window sizes, boundary-layer index and ring lengths are model-config specific and are not transferable.", + "Query-row compaction does not imply dropping historical keys; the two decisions are separate.", + "Prompt logprobs are unrecoverable for the clipped region; the reference forces them off with a warning.", + "The truncation path is off by default in the reference source." + ], + "provenance_readsets": [ + "references/source-notes.md", + "references/validation.md" + ], + "references": [], + "consumption_mode": "packaged" +} diff --git a/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/source-notes.md b/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/source-notes.md new file mode 100644 index 000000000..1a4e456dd --- /dev/null +++ b/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/source-notes.md @@ -0,0 +1,132 @@ +# Source notes: bound windowed-attention replay + +## Provenance and limits + +- Upstream: [deepseek-ai/DeepInfer](https://github.com/deepseek-ai/DeepInfer), revision [`8b16149741a4935e1dec8cea27853f273b374c5b`](https://github.com/deepseek-ai/DeepInfer/commit/8b16149741a4935e1dec8cea27853f273b374c5b). +- Snapshot: `source.tar.xz`, SHA-256 `7de61b01434549e2871779b12dec307fb4e283e5da3255bb19036e7be52c65ea`, 1,086 regular files. That is archive size, not reading coverage; the selected paths below were read statically. Nothing was built, installed or executed; no weights were fetched. +- The parent's [Apache-2.0 license](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/LICENSE#L1-L3) was inspected. The six vendor submodules retain independent licenses. +- **F** means code behaviour at this pin; **P** means proposed, unvalidated procedure. Empirical validation: **not_run**. +- The model this implements is DeepSeek-V4.1-Flash. Window sizes, layer counts and ring lengths below are that model's; the *rule* is what transfers, and only the rule. + +## What the architecture actually is + +This is the part that decides whether the procedure transfers, so it comes first. + +DeepSeek-V4.1-Flash splits the decoder stack at `n_prefill_layer`. Layers before the split are producer layers and run over the full chunk. Layers from `n_prefill_layer` onward are **consumer** layers and run over a compact query built from a retained suffix (`F`: `deepinfer/model_executor/models/v4_modules/yoco.py:15-25`). + +Three properties make the shortening valid, and all three are architectural rather than incidental: + +1. **A shared hidden is captured at the boundary.** The first consumer layer's attention-norm output is the shared hidden; later consumer layers consume it, and some SWA layers project their KV from it (`F`: `yoco.py:24-42`, `decoder_layer.py:145-152`). +2. **Full-length KV is written at the boundary.** The consumer KV-source layer projects KV from the full hidden and stores main/index KV at full length, so the persistent cache is complete even though the query is compact (`F`: `attn.py:672-719`). +3. **The SWA path has an explicit visible-region rule.** Its visible region intersects the sliding window, retained rectangle and valid-state start (F3 below). This does not restrict every consumer attention path to SWA: the main/index KV path above preserves full historical keys while query rows are compacted. + +None of these follows from "the layers have a finite window". A stack of individually windowed layers could still have a consumer whose dependence reaches further back, through a shared state or a compressed index. That is why `SKILL.md` step 1 asks for the dependency closure rather than a per-layer window check. + +## F evidence ledger + +### F1 — The retain rule and the rollback floor + +[`deepinfer/environ.py:132`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/environ.py#L132): + +```python +DEEPINFER_MAX_BACKTRACKING_TOKENS = EnvVar(lambda v: max(_int(v), envs.DEEPINFER_USE_NEXTN_DECODE + 1)) +``` + +The rollback depth cannot be configured below the speculation width plus one. That floor is what makes `window + rollback` sufficient: a full-width rollback after a full-width draft still leaves a full window. + +### F2 — The retain rule is enforced at three layers + +The C++ metadata builder checks it at [`csrc/engine/input_metadata/prepare_tensor_input_v4.cpp:350-357`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/input_metadata/prepare_tensor_input_v4.cpp#L350-L357): + +```cpp +if (args.yoco_prefill_truncation_mode == YocoPrefillTruncationMode::OFF) + return; +const int32_t keep_tokens = args.yoco_prefill_keep_tokens; +TORCH_CHECK_GT(keep_tokens, 0); +TORCH_CHECK_GE(keep_tokens, args.window_size + args.max_backtracking_tokens) + << "DEEPINFER_YOCO_PREFILL_KEEP_TOKENS must be >= window_size (" << args.window_size + << ") + DEEPINFER_MAX_BACKTRACKING_TOKENS (" << args.max_backtracking_tokens << ")"; +``` + +Note the guard: the whole entry point returns early when the mode is `off`, so these checks apply only when truncation is enabled. There is no assertion about `keep_tokens` in the disabled path. `KEEP_TOKENS` is `EnvVar.int()` with no default (`F`: `environ.py:194-197`), which resolves to 0, so enabling truncation without setting it fails this check rather than silently clipping. + +The same condition appears twice more before that: + +- [`deepinfer/model_executor/model_metadata_utils.py:90-92`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/model_metadata_utils.py#L90-L92) asserts `yoco_enabled and n_prefill_layer > 0` and `KEEP_TOKENS > 0`, under the same `!= "off"` guard. +- [`deepinfer/worker/model_runner.py:107-114`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/model_runner.py#L107-L114) asserts `window_size > 0` and the `window + rollback <= keep` relation, again under the same guard. + +### F3 — The visible-region formula + +[`prepare_tensor_input_v4.cpp:155`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/input_metadata/prepare_tensor_input_v4.cpp#L150-L167): + +```cpp +int32_t abs_lo = std::clamp(std::max({pos - window_size + 1, rect_lo, valid_start}), 0, pos + 1); +``` + +Three lower bounds intersect: the sliding window start, the retained rectangle's lower bound (`rect_lo = data_len - keep_tokens`, documented at `:145`), and `valid_start`. The third exists because zero-state recompute clears SWA state in `[0, valid_start)`, so the window must be clamped past it to avoid attending to cleared slots (`:153-155`). Skipping that clamp is a silent numerical bug, not a crash. An empty visible region is represented by `abs_lo == pos + 1` (`:150-152`). + +### F4 — Two modes with different costs + +[`deepinfer/environ.py:188-194`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/environ.py#L188-L194): + +```python +DEEPINFER_YOCO_PREFILL_TRUNCATION_MODE = EnvVar.choice(default="off", choices=("off", "mask", "compact")) +``` + +- `mask` keeps every query row and only narrows the visible region. Compute-neutral; useful as a reference. +- `compact` removes the clipped query rows, which is where the compute saving is. + +The metadata mirror is [`deepinfer/model_executor/input_metadata/yoco.py:32-72`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/input_metadata/yoco.py#L32-L72): `mask` mode produces `yoco_mask_swa_index_pivots` on the full layout, `compact` mode produces the `yoco_consumer_*` family on the compact layout, and `get_mask_view` / `get_consumer_view` are the two consumers. + +The C++ side states the same split in the clip computation: `mask` mode sets `clip_prefix = 0` for all requests, so no rows are dropped (`F`: `prepare_tensor_input_v4.cpp:224-225`). + +### F5 — Compact is gated on mode, not on CP + +[`yoco.py:67-72`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/input_metadata/yoco.py#L67-L72) makes `yoco_compaction_enabled` a check of the truncation mode alone. Context parallelism is handled separately by compacting first and then applying the CP mirror (`F`: `yoco.py:55-77` computes the contributed-token count per rank and the compact-plus-CP length), rather than by disabling compaction. + +### F6 — Full layout and compact copy coexist + +[`deepinfer/model_executor/models/v4_modules/yoco.py:93-122`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/models/v4_modules/yoco.py#L93-L122) defines `compact_yoco_boundary`, whose docstring states it "returns the compact copies and the original full hidden/residual". The caller at [`decoder_layer.py:134-141`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/models/decoder_layer.py#L134-L141) stores `consumer_full_hidden = self.attn_norm_forward(*full_inputs)[1]` and only then replaces `hidden_states, residual` with the compact versions. + +The correct statement is therefore: the full-layout representation is computed and retained for the boundary's full-length KV write, while the compact copy continues downstream. It is **not** a global ordering rule that KV writes precede gathers; in the consumer's forward the full KV write is a separate pass over the full metadata (`F`: `attn.py:672-719`, `_store_yoco_full_kv`). + +`gather_hc_lazy` exists because the hyperconnection state is a structured bundle rather than a plain tensor: each per-token field has to be gathered over the kept intervals separately (`F`: `yoco.py:79-90`). The reason is field structure, not host residency — the bundle holds device tensors, and nothing in the module places them in host memory. + +### F7 — Ring sizing has two separate additions + +[`deepinfer/worker/cache_engine.py:88-110`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/cache_engine.py#L88-L110): + +```python +def get_inference_swa_block_size(window_size: int) -> int: + return (window_size + envs.DEEPINFER_MAX_BACKTRACKING_TOKENS) if window_size > 0 else 0 + +def get_yoco_store_extra_tokens() -> int: + return max(0, envs.DEEPINFER_YOCO_PREFILL_KEEP_TOKENS) if envs.DEEPINFER_YOCO_PREFILL_TRUNCATION_MODE != "off" else 0 + +def get_kvstore_swa_block_size(window_size: int) -> int: + return get_inference_swa_block_size(window_size) + ( + (envs.DEEPINFER_PREFILL_STORE_EXTRA_SIZE + get_yoco_store_extra_tokens()) if envs.DEEPINFER_PREFILL_STORE_EXTRA_STATE else 0 + ) +``` + +A nonpositive window produces an inference-ring size of zero. For a positive window, the inference ring is `window + rollback` per sequence. The kvstore function adds margins under nested conditions: `DEEPINFER_PREFILL_STORE_EXTRA_STATE` gates both additions; within that branch the store-granularity margin is added, and the truncation keep margin is added only when the mode is not `off`. + +The comment states the store-granularity reason: kvstore writes only at multiples of 128, so prefill may not trigger a write and an extra span is needed to guarantee multi-round hits. + +### F8 — The equivalence harness ships + +`examples/yoco/yoco_equiv.py` and `run_yoco_equiv_l2.sh` are present at the pin, with `examples/yoco/README.md` describing the two-level check. This is TEST-SOURCE only: the harness was read, not run, and it requires a built extension plus the target device. + +## P — Proposed items in the parent skill + +- The dependency-closure step is P. The source implements one specific architecture; the requirement to derive the closure before shortening is a generalization proposed here, not something the source states. +- Separating query-row compaction from historical-key retention as two independently-decided questions is P. The source makes the first decision and preserves full-length KV at the boundary; the framing is proposed to prevent over-reading it. +- Recommending that a rejecting engine reject prompt-logprob requests rather than silently disabling them is P. The source disables with a warning. +- The instruction to state the anchor and document preemption movement is P. + +## Not established + +- No equivalence run. Metadata equivalence between `mask` and `compact` is argued from the shared `abs_lo` formula, not measured. +- Transfer to another architecture, including one with a different compressed index or historical top-k dependency, was not analysed. Its required historical state and query dependency closure must be derived independently. +- Preemption behaviour during a comparison run was not observed; whether recompute moves the anchor in practice was not tested. +- The kvstore ring's extra span was read from code and comments, not from an executed boundary case. diff --git a/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/validation.md b/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/validation.md new file mode 100644 index 000000000..c4cb629ce --- /dev/null +++ b/src/inferrsi/data/skill_library/repositories/deepinfer/bound-windowed-attention-replay/references/validation.md @@ -0,0 +1,48 @@ +# Validation: bound windowed-attention replay + +Nothing below has been run. This file records what would establish or refute each claim in `SKILL.md`. + +## Static and CPU checks + +| # | Check | Refutes if | +|---|---|---| +| V1 | Derive the dependency closure: for every consumer layer, list what it reads and where that was produced | Any consumer reads state whose value depends on tokens outside the retained region and no full-length copy of it is preserved | +| V2 | Confirm a full-length KV write exists at the boundary, independent of the compact query | No full-length write exists, so shortening the query changes the persistent cache | +| V3 | Confirm the retain rule is enforced wherever the value can enter, including model metadata, worker setup and metadata construction | The rule is enforced in fewer places than the value can arrive through, so a misconfiguration fails numerically rather than loudly | +| V4 | Confirm the checks are guarded by the truncation mode, so they do not fire when truncation is disabled | The checks fire in the disabled path, or `keep_tokens = 0` is accepted while truncation is on | +| V5 | Confirm the rollback floor is `max(configured, speculation_width + 1)` | The floor can be configured below the speculation width, allowing a short window after rollback | +| V6 | Recompute the visible-region formula and compare against the source expression, including the `valid_start` clamp and the empty-region representation | The clamp is absent or different, so zero-state recompute can attend cleared slots | +| V7 | Confirm `mask` mode produces the same visible-key set as `compact` mode on the same input, at a synthetic chunk boundary | The two modes disagree, so `mask` is not a valid reference | +| V8 | Confirm the boundary returns both a compact copy and the original full representation, and that the full one is stored | Only the compact copy survives the boundary, which corrupts the full-length KV write | +| V9 | Confirm the ring length is `window + rollback` per sequence and zero when the model has no SWA layers | The ring is short, so a rollback loses history | +| V10 | Confirm `PREFILL_STORE_EXTRA_STATE` gates both kvstore-ring additions; within that branch the keep margin also requires truncation mode not `off` | A keep margin is added without both gates, or a store-granularity margin is added while extra-state is disabled | +| V11 | Read the hyperconnection state bundle and confirm where its tensors live | The bundle is host-resident, which would make the gather reason different from field structure | +| V12 | Confirm the prompt-logprob behaviour is forced-off-with-warning rather than rejection | The engine rejects, or returns wrong values instead of doing either | + +## Single-device checks + +| # | Check | Refutes if | +|---|---|---| +| G1 | Metadata equivalence at chunk boundaries: visible-key sets, store slots and compressed-key ranges identical between `mask` and `compact` | Any of the three differ, which is a silent correctness bug | +| G2 | End-to-end equivalence on a fixed teacher-forcing trajectory: logprobs and top-k element-wise | Divergence outside tolerance | +| G3 | Chunk-boundary sweep: chunk sizes that do and do not align with store granularity | A boundary case diverges only when chunks do not align | +| G4 | Fully clipped chunk: a chunk entirely before the retained rectangle | The run crashes or produces no sampling position | +| G5 | Preemption count during the comparison run | Non-zero, which makes the run incomparable and any divergence uninterpretable | +| G6 | Prompt-logprob request under truncation: record whether it is forced off, warned about, or rejected | The observed behaviour differs from what the engine claims, or per-token values are returned for the clipped region | +| G7 | Measure the `compact` saving against the `mask` baseline, reporting both | `compact` shows no compute difference from `mask`, so the saving claim is empty | +| G8 | Peak memory at fixed context length, truncation on versus off, with the same declared resource budget | Required state is missing or the declared memory budget is exceeded; query compaction alone does not promise lower persistent-KV memory | +| G9 | A dependency-probe model: keep all historical keys, shorten only the query, and confirm output equality | Shortening the query alone changes the output, which would mean the procedure is not separable from key retention | + +## What a PASS would require + +V1 and G9 establish the architectural precondition, then G1 and G2 together establish equivalence, with G5 confirming no preemption. G1 alone is a metadata argument; G2 alone hides where a divergence came from. + +G9 is the check that decides transfer. It separates "the query was shortened" from "the keys were dropped", which is exactly the distinction this procedure depends on and the one most likely to be collapsed when the same idea is applied to another stack. + +## What this skill must not claim + +- No general applicability to arbitrary sliding-window stacks. The dependency closure has to be derived for the target model. +- No optimization from `mask` mode; it is compute-neutral by construction. +- No prompt-logprob support for clipped regions. +- No claim that compaction implies dropped historical keys. Query-row compaction and historical-key retention are separate decisions. +- No bit-identical results across preemption; recompute moves the anchor. diff --git a/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md b/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md new file mode 100644 index 000000000..8b88b6a2d --- /dev/null +++ b/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/SKILL.md @@ -0,0 +1,101 @@ +--- +name: budget-speculative-verification +description: Decide how many draft tokens to verify per decode step when acceptance varies by request and position. Use when an engine already runs a draft head with multi-token speculation and verification cost grows with query rows. Separates draft cost from verify cost, builds a cumulative-acceptance rate vector, fits a cost model, and allocates a per-step row budget. +--- + +# Budget speculative verification + +## Trigger and boundary + +Use when all of the following hold: + +- The engine performs multi-token speculative decoding with a draft head. +- Verification is a batched forward whose cost grows with the number of query rows. +- Acceptance probability varies materially by request and by draft position. +- A per-position acceptance signal exists, or can be trained and exported. + +Do **not** use when: + +- The draft head is a separate model whose forward cost is proportional to the number of drafts requested. Then the budget must also throttle drafting, and many implementations cannot. The procedure below budgets verification only. +- There is no per-step cost model at all, because the budget decision needs a cost-versus-rows estimate. + +## Status and source + +Status: **candidate**. Empirical validation: **not_run**. + +Source: `deepseek-ai/DeepInfer` @ `8b16149741a4935e1dec8cea27853f273b374c5b` (Apache-2.0), read statically. Nothing was built or executed, and no gain was reproduced. Evidence labels follow the readset convention: **F** is a fact observed in the pinned source; **P** is a proposed procedure. + +Read [source-notes.md](references/source-notes.md) for the evidence ledger and [validation.md](references/validation.md) for checks that have not been run. + +## Required inputs + +1. Draft width and how the engine represents a request's rows — typically one fresh row plus `k` verification rows. +2. A per-position acceptance estimate with known semantics: individual/conditional confidence estimates, or already cumulative. This distinction decides step 2 and cannot be inferred from a field name or treated as proof of statistical independence. +3. A table of `(batch_rows, step_time)` for the verify forward, or a way to collect it at warmup. +4. A sampler cost estimate per batch, because the total step includes sampling. +5. Which step's acceptance values are available at schedule time. If the scheduler runs ahead of sampling, the current step's values do not exist yet. + +## Procedure + +1. **Separate the two costs.** Measure the draft forward and the verify forward independently. The inspected DFlash decode path produces drafts at its configured width: the sampler calls `decode_forward`, which builds fixed-width DFlash metadata and samples all `max_MTP_value` positions (`F`: `deepinfer/model_executor/layers/sampler.py:225-251`; `models/mtp/dflash.py:103,141,308-336,367-387`). The variable-length scheduler budgets verification rows; its non-budgeted `1 + draft_tokens.size()` branch alone is not evidence about the draft producer. If a different draft model's cost scales with requested width, include that cost in the decision. + +2. **Normalize the acceptance signal to a cumulative rate.** Convert individual/conditional per-position confidence estimates according to their defined contract into a non-increasing cumulative vector. The reference multiplies its estimates (`F`: `csrc/engine/scheduler.hpp:1243-1250`); that implementation does not prove the underlying random variables are statistically independent. A cumulative vector makes grants of additional verification rows follow a prefix over draft positions. + +3. **Fit a cost model.** `time(rows) ≈ baseline(rows) + slope × rows + sampler(requests)`. Collect the curve at warmup under the graph sizes you will actually replay. Clamp the slope at zero so extrapolation cannot produce a negative or non-positive time (`F`: `mtp_varlen.hpp:226`, `std::max(0.0, _forward_time_slope.get())`). + +4. **Choose the global row budget.** Maximize `expected_accepted_tokens / time(rows)` over the sorted candidate rows. Seed the expectation with one token per request, because every request always samples at least one (`F`: `mtp_varlen.hpp:231`, `cur_expected = static_cast(num_seqs)`). + +5. **Allocate per request.** Walk the cumulative rates in descending order and grant one extra row at a time until the global budget is spent. Break ties by request index so the result is deterministic (`F`: `mtp_varlen.hpp:204-214`). + +6. **Enforce the floor.** Every scheduled request needs at least one row. Assert that the total is no smaller than the number of requests rather than trusting the search (`F`: `mtp_varlen.hpp:260`, `TORCH_CHECK(best_bsz >= num_seqs)`). This is an allocation contract, not a measured comparison with plain decode. + +7. **Preserve the proposal's per-position decision independence when choosing which round's confidence to read.** At a fixed historical global budget and otherwise fixed allocation state, confidence at later positions `i+1 … n` of the same proposal must not decide whether draft position `i` is verified. Later confidence may depend on an already sampled earlier draft token; letting it select that token for verification may bias the complete emitted distribution and requires a causal and sampling-correctness check. The reference states the intended requirement directly (`F`: `mtp_varlen.hpp:160`, translated from the source comment: "confidence for positions `i+1, …, nextn` must not affect whether `draft_token_i` is verified"). This is not an invariance requirement for arbitrary historical rates: changing the N−2 cost-estimation queue may legitimately change the global row budget. + + The two-round protocol exists to satisfy that requirement under a pipelined scheduler. CPU scheduling takes the global row budget from N−2; its per-request queue always reads `accept_rates`, which is N−1 when schedule/process are paired and N−2 when one forward is still in flight (`F`: `mtp_varlen.hpp:160-207`). In the overlapping ZOS path, the worker uses the previous step's GPU confidence to redistribute the fixed total row budget before the current main forward, then synchronizes corrected metadata back before sampling (`F`: `deepinfer/worker/worker.py:118-152,191-219`). Do not describe the CPU queue alone as the final N−1 allocation. A different timing is a new design and must establish the same decision-independence contract; source comments alone are not a sampling-correctness proof. + +8. **Propagate the allocation to the metadata.** The verify kernels read per-request query length and the per-position row map. Both must be built from the budget, not from the draft width. + +## Expected outputs + +- A per-step row budget and a per-request query length in `[1, draft_width + 1]`. +- A cumulative acceptance vector per request. +- Log fields: predicted acceptance, realized acceptance, and their error. + +## Evidence to collect + +Verification rows and accepted tokens measure different things, so report them separately rather than as one before/after pair: + +- **Allocation invariants**: every grant is a prefix over draft positions, and no request receives fewer than one row. +- **Sampling correctness**, against an independently specified oracle: does the accepted-prefix decision match the rejection rule you intend, under explicitly stated assumptions about the draft and target distributions. +- **Cost and throughput**: verification rows per step, accepted tokens per step, total step time, and end-to-end throughput, all with the draft forward included. +- **Realized-versus-predicted acceptance error**, aggregated per step. + +A lower accepted length is a legitimate outcome of budgeting, not a failure: the objective is expected accepted tokens per unit time, and verifying fewer drafts can lose accepted tokens while gaining more. Do not use unchanged accepted length as the acceptance criterion. Deterministic token comparison is a valid check only where the decoding path is deterministic and the sampling semantics justify it; under stochastic sampling, two runs that consume random draws in different orders do not have to agree token for token. + +## Preconditions and caveats + +- **The draft forward is unchanged.** Quoting a throughput gain without saying so overstates the effect: the reference saves verification rows only. +- **Two different temperatures, and neither implies the other.** The confidence-head temperature (`DEEPINFER_DFLASH_CONFIDENCE_HEAD_TEMPERATURE`) scales the confidence head's output; it calibrates the acceptance estimate (`F`: `deepinfer/model_executor/models/mtp/dflash.py:62-64,76`). A request's sampling temperature is a separate per-request parameter that is clamped at the low end with `top_p`/`top_k` reset (`F`: `deepinfer/sampling_params.py:76-79`). Do not infer sampling behaviour from confidence calibration, or the reverse. +- **Rejection is gated on draft logprobs, not on temperature.** The reject path runs when draft logprobs are supplied (`F`: `deepinfer/model_executor/layers/tilelang_ops/fused_sample.py:200`), and request temperature is clamped rather than allowed to reach zero. Together these support the reading that a request configured as greedy still goes through verification; this is a reading of the gating logic, not a measured end-to-end result. +- **Low acceptance still reserves state.** KV blocks are reserved at maximum width and trimmed later, so a request that accepts nothing still pins extra slots until the next scheduling pass. +- **Wide top-k pools scale badly.** Verification logprob pools scale as `rows × topk × positions`; a full-vocabulary top-k can dominate memory before it dominates time. +- **Cross-platform parity is unproven.** The reference has different rejection-comparison arithmetic on its two backends and no cross-platform equality assertion. + +## Stop conditions + +- Stop if the draft forward cost dominates and cannot be throttled; then the verification saving is a small share of the step. +- Stop if sampler cost dominates verify cost at your occupancy. +- Stop if the acceptance signal cannot be made cumulative; a non-prefix grant breaks the row map. +- Stop if the scheduler can see no acceptance estimate at all, from any round. + +Uniform acceptance is **not** by itself a stop condition. A population with near-zero acceptance can still make one row per request the best choice, because it avoids paying for verification that would be discarded. Whether the budget helps depends on the actual cost model and the fixed overhead, not on how uniform the rates are. Measure it rather than excluding the case in advance. + +## Configuration in the reference source + +| Variable | Default | Effect | +|---|---|---| +| `DEEPINFER_USE_NEXTN_DECODE` | `5` | Maximum speculative width; `0` disables speculation entirely | +| `DEEPINFER_ENABLE_MTP_VARLEN` | unset → false | Enables the budgeted path; without it the draft width is fixed | +| `DEEPINFER_DFLASH_CONFIDENCE_HEAD_TEMPERATURE` | `1.0` | Per-position scale applied inside the confidence head; clamped above zero. It does not set the request sampling temperature. | + +The budget is **off by default** in the reference. Do not describe it as the engine's default behaviour. diff --git a/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/metadata.json b/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/metadata.json new file mode 100644 index 000000000..e93b8b5bc --- /dev/null +++ b/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/metadata.json @@ -0,0 +1,57 @@ +{ + "tags": [ + "deepinfer", + "speculative-decoding", + "mtp", + "verification", + "cost-model", + "scheduling" + ], + "status": "candidate", + "empirical_validation": "not_run", + "profile": "local-skill-v1", + "schema_version": 1, + "kind": "skill", + "origin": [ + "external" + ], + "sources": [ + { + "type": "git", + "url": "https://github.com/deepseek-ai/DeepInfer", + "commit": "8b16149741a4935e1dec8cea27853f273b374c5b", + "paths": [ + "csrc/engine/mtp_varlen.hpp", + "csrc/engine/scheduler.hpp", + "deepinfer/worker/worker.py", + "deepinfer/environ.py", + "deepinfer/sampling_params.py", + "deepinfer/model_executor/models/mtp/dflash.py", + "deepinfer/model_executor/layers/sampler.py", + "deepinfer/model_executor/layers/tilelang_ops/fused_sample.py", + "LICENSE" + ], + "readset": "references/source-notes.md" + } + ], + "applicability": { + "scope": "Engines with multi-token speculative decoding whose verification forward cost grows with query rows", + "prerequisites": "A calibrated per-position acceptance signal; a measured (rows, step_time) curve; a handle on metadata construction", + "hardware": "Backend-independent procedure; the reference implementation targets NVIDIA SM100 and Ascend 950 only" + }, + "limitations": [ + "Source-derived candidate; no local efficacy validation and no reproduced gain.", + "The reference budgets verification rows only; draft compute is unchanged.", + "The budgeted path is off by default in the reference source.", + "Budgeting can lower accepted length; equal accepted length is not an acceptance criterion.", + "Per-position independence of the confidence used for verification is a correctness requirement, not only a scheduling convenience.", + "Cross-backend rejection-comparison parity is unproven; the reference has no equality assertion.", + "Numeric acceptance rates and warmup curves are deployment-specific and are not carried here." + ], + "provenance_readsets": [ + "references/source-notes.md", + "references/validation.md" + ], + "references": [], + "consumption_mode": "packaged" +} diff --git a/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/source-notes.md b/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/source-notes.md new file mode 100644 index 000000000..bbe531801 --- /dev/null +++ b/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/source-notes.md @@ -0,0 +1,117 @@ +# Source notes: budget speculative verification + +## Provenance and limits + +- Upstream: [deepseek-ai/DeepInfer](https://github.com/deepseek-ai/DeepInfer), revision [`8b16149741a4935e1dec8cea27853f273b374c5b`](https://github.com/deepseek-ai/DeepInfer/commit/8b16149741a4935e1dec8cea27853f273b374c5b). +- Snapshot: `source.tar.xz`, SHA-256 `7de61b01434549e2871779b12dec307fb4e283e5da3255bb19036e7be52c65ea`, 1,086 regular files. That is archive size, not reading coverage: selected parent paths were read statically; nothing was built, installed or executed, and no weights were fetched. +- The parent's [Apache-2.0 license](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/LICENSE#L1-L3) was inspected; source headers attribute the vLLM origin. The six vendor submodules retain independent licenses; this source map does not extend the parent's attribution over them. +- Bounded excerpts below are attributed to DeepSeek-AI. The operational procedure in `SKILL.md` is proposed here, not an upstream endorsement. +- **F** means code behaviour at this pin. **P** means a proposed, unvalidated procedure. Empirical validation: **not_run**. + +Line references use four-digit line numbers and are stable at this pin. Upstream comments in the cited files are Chinese; the glosses below are translations and are marked as such. + +## F evidence ledger + +### F1 — The budgeted path is optional + +[`deepinfer/environ.py:129-132`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/environ.py#L129-L132): + +```python +DEEPINFER_USE_NEXTN_DECODE = EnvVar.int(default=5) +DEEPINFER_ENABLE_MTP_VARLEN = EnvVar.bool() +DEEPINFER_MAX_BACKTRACKING_TOKENS = EnvVar(lambda v: max(_int(v), envs.DEEPINFER_USE_NEXTN_DECODE + 1)) +``` + +`EnvVar.bool()` defaults to false (`environ.py:70-71`). So the fixed-width path is the default and the budgeted path must be enabled. Speculation itself is on by default at width 5; setting `USE_NEXTN_DECODE=0` disables it. + +### F2 — Draft cost is full width; only verification is budgeted + +[`csrc/engine/mtp_varlen.hpp:150-158`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/mtp_varlen.hpp#L150-L158) controls non-budgeted verification lengths, not draft generation. The actual producer chain is [sampler.py:225-251](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/layers/sampler.py#L225-L251) → [DFlash.decode_forward/run_drafts:347-387](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/models/mtp/dflash.py#L347-L387). DFlash's width comes from `USE_NEXTN_DECODE` (`dflash.py:103,141`), builds request-count × fixed-width draft inputs (`:230-239`) and samples every configured position (`:308-336`). The inspected DFlash path remains full width while variable-length scheduling saves verification rows; this is not a claim about arbitrary draft-model implementations. + +### F3 — Cumulative, sorted, prefix grants + +[`mtp_varlen.hpp:204-214`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/mtp_varlen.hpp#L204-L214) builds two queues of `(accept_rate, seq_idx)` pairs and sorts both descending by rate, breaking ties by ascending sequence index: + +```cpp +// descending by rate, ascending by sequence index on ties +static bool by_rate_desc(const Pair& a, const Pair& b) { + return a.first != b.first ? a.first > b.first : a.second < b.second; +} +``` + +Grants walk this sorted order one row at a time, so a request can only receive verification for position `k` if it already received positions `< k`. + +### F4 — Cost model and the non-negative slope clamp + +[`mtp_varlen.hpp:216-247`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/mtp_varlen.hpp#L216-L247): + +```cpp +const double sampler_ms = estimate.value_or(0.0); +const double slope_ms = std::max(0.0, _forward_time_slope.get()); +... +const double time_ms = baseline_ms + slope_ms * graph_size + sampler_ms; +const double cur_throughput = cur_expected / time_ms; +``` + +The sampler term is a per-request-count median over a 9-sample window with a 5-sample minimum before it is trusted (`mtp_varlen.hpp:22-46`). The forward slope is fitted from `(forward_ms - baseline_ms) / graph_size` and aggregated per graph size before a second median (`mtp_varlen.hpp:43-60`). Clamping prevents a negative slope from extrapolating to a non-positive time at larger graphs. + +The optimized quantity is `cur_expected / time_ms`, where `cur_expected` starts at `num_seqs` and accumulates the acceptance rate of each additional row (`:231`, `:252`). Fewer verified rows can therefore lower the accepted-token count while still raising this ratio. + +### F5 — The floor is asserted, not trusted + +[`mtp_varlen.hpp:260`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/mtp_varlen.hpp#L260): `TORCH_CHECK(best_bsz >= num_seqs);` + +One row per request is the minimum meaningful verification, and the search is seeded with that (`:231-232`). + +### F6 — Per-position independence is a correctness constraint + +[`mtp_varlen.hpp:160`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/mtp_varlen.hpp#L160) states, translated from the Chinese source comment: "confidence for positions `i+1, …, nextn` must not affect whether `draft_token_i` is verified." + +The comment continues to describe the two-round protocol as the way to satisfy that requirement while the scheduler runs ahead of sampling: use round N−2's rates for the global batch size and round N−1's rates for the per-request split. This makes the lag a correctness requirement on *which* rates may be combined, not only a workaround for an unavailable value. + +[`mtp_varlen.hpp:161-183`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/mtp_varlen.hpp#L161-L183) implements it. Per-request `schedule_cnt` and `process_cnt` counters decide which round's vector is valid: + +- `schedule_cnt == process_cnt` — the request's forward and post-process are paired, so `accept_rates` is from round N−1 and `prev_accept_rates` from N−2. +- `schedule_cnt == process_cnt + 1` — the latest forward has not produced rates yet, so both shift back one round. + +The asserted invariant is `process_cnt <= schedule_cnt <= process_cnt + 1`. The pairing flag selects the batch-size queue's N−2 vector (`:204`). The per-request CPU queue always reads `accept_rates` (`:207`), hence N−1 when paired and N−2 when one forward is in flight. In the latter overlapping ZOS path, [worker.py:191-219](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/worker.py#L191-L219) uses the previous sampler's GPU confidence to redistribute fixed `total_q`. This correction feeds the current main forward, then its D2H metadata is synchronized before sampling ([worker.py:118-152](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/worker/worker.py#L118-L152)). The CPU queue alone is not the final N−1 per-request allocation. + +The source computes cumulative confidence by multiplication ([scheduler.hpp:1243-1250](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/csrc/engine/scheduler.hpp#L1243-L1250)). Its description of individual token estimates must not be read as a demonstrated statistical-independence theorem; DFlash's optional Markov branch uses the preceding sampled token (`dflash.py:308-320`). + +### F7 — Confidence is a model property, not an engine option + +[`deepinfer/model_executor/models/mtp/dflash.py:159-160`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/models/mtp/dflash.py#L159-L160): `self.enable_confidence = True` with the comment that the HF DFlash checkpoint always contains a confidence head. The head is exported as `confidence_head_fused` (`:15`, `:73`). + +Consequence for transfer: a model without a trained confidence head cannot use this procedure as written. A substitute per-position acceptance estimate would need its own validation, because the procedure assumes the vector is already calibrated. + +### F8 — Two distinct temperatures + +The confidence-head temperature is a per-position scale applied **inside the confidence head**, defaulting to `1.0` and clamped to `SAMPLING_EPS` = `1e-5` (`F`: `environ.py:134-147` defines the variable; `dflash.py:62-64` applies `max(value, SAMPLING_EPS)`; `dflash.py:76` passes `self.temperature[step_i]` into `confidence_head_fused`). + +The request sampling temperature is separate. It is a field of `SamplingParams`, and values below `SAMPLING_EPS` are raised to `SAMPLING_EPS` with `top_p` set to `1.0` and `top_k` to `-1` (`F`: `deepinfer/sampling_params.py:76-79`). + +Neither value implies the other. Confidence calibration does not tell you how the request samples, and the request's sampling temperature does not change the acceptance estimate. + +### F9 — The reject path is gated on draft logprobs + +[`deepinfer/model_executor/layers/tilelang_ops/fused_sample.py:200`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/layers/tilelang_ops/fused_sample.py#L200) sets `do_reject_sampling = last_nextn_token_logprobs is not None`; the flag then gates the residual computation, the acceptance comparison and the residual resample. Request temperature is not part of that condition. The sampler contract describes the decode path as accepting or rejecting drafts by the speculative-decoding rule (`F`: `deepinfer/model_executor/layers/sampler.py:76-77`, source comment). + +Combined with the clamp in F8, this supports the statement that a request configured as greedy still goes through rejection. That reading is from the gating logic; no run confirmed it end to end. + +### F10 — Structured output interacts with per-draft verification + +[`deepinfer/model_executor/layers/sampler.py:363-376`](https://github.com/deepseek-ai/DeepInfer/blob/8b16149741a4935e1dec8cea27853f273b374c5b/deepinfer/model_executor/layers/sampler.py#L363-L376) advances the grammar matcher once per draft, fills a mask per draft position, and rolls back by the number of accepted drafts. A grammar-constrained request therefore gets a per-position mask alongside the per-position acceptance rate; the two are independent signals and must not be conflated. + +## P — Proposed items in the parent skill + +- The framing of "expected accepted tokens per unit time" as the objective to report is F; the specific reporting template in `SKILL.md` is P. +- The recommendation to log realized-versus-predicted acceptance error is P. +- The statement that near-zero acceptance can still favour a one-row budget is derived from the cost objective at `:243-252` plus the one-row floor, not from a measurement. + +## Not established + +- No gain was reproduced. The source's own reported figure is a DOC-class claim in a deployment Q&A, not a local measurement. +- Whether the fitted slope generalizes past the graph sizes seen at warmup is unknown; the code only clamps it. +- Cross-backend equality of the rejection comparison has no assertion and was not tested. +- Memory behaviour of wide top-k verification pools was not measured. +- The greedy-still-verifies reading (F9) was not confirmed by running a temperature-0 request through the verify path. diff --git a/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/validation.md b/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/validation.md new file mode 100644 index 000000000..81e7742a0 --- /dev/null +++ b/src/inferrsi/data/skill_library/repositories/deepinfer/budget-speculative-verification/references/validation.md @@ -0,0 +1,70 @@ +# Validation: budget speculative verification + +Nothing below has been run. This file records what would establish or refute each claim in `SKILL.md`, so a later authorized test has a target. Numeric thresholds are deliberately absent; they belong to an evaluation plan with a named device, concurrency and workload, because the cost model depends on all three. + +## Static and CPU checks + +| # | Check | Refutes if | +|---|---|---| +| V1 | Read `EnvVar.bool()` default and confirm the budgeted path is off unless `DEEPINFER_ENABLE_MTP_VARLEN` is set | The budgeted path proves to be default-on, which changes the "off by default" statement | +| V2 | Confirm the draft forward is not throttled by the budget: trace that draft token count is produced before budget allocation | Draft cost scales with the allocation, which invalidates the "verification only" framing | +| V3 | Check exact cumulative values: for conditional per-position rates `p_i`, require `c_i = product(p_j, j <= i)`; preserve inputs already cumulative, including ties and zero rates | Values differ from the required products or already-cumulative inputs are multiplied again; monotonicity alone also accepts an incorrect running minimum | +| V4 | Unit-test the allocator: sorted descending with index tie-break produces a prefix grant for every request, and `best_bsz >= num_seqs` | The allocator can grant a non-prefix set, or the floor can fail | +| V5 | Unit-test the slope clamp with a synthetic negative slope | Extrapolation can produce a non-positive time | +| V6 | Confirm which round's rates the allocator reads for batch size versus per-request length, and that the pairing flag selects between the two stored vectors | The two-round description is wrong, or the pairing flag does not control which vector is used | +| V7 | Confirm the confidence head is presented as a model property, not an engine option | A model without the head could still use the path as written | +| V8 | Confirm the confidence-head temperature is clamped above zero and is applied inside the confidence head | A zero value is reachable, or the value reaches the request's sampling path | +| V9 | Confirm the request sampling temperature is a separate field and is clamped independently | The two temperatures are the same value, which would make F8 wrong | +| V10 | Confirm the reject path is gated on draft logprobs rather than on request temperature | A temperature-0 request bypasses rejection, changing the "greedy still verifies" reading | + +## Single-device checks + +Correctness and benefit are separate questions and need separate runs. Do not combine them into one before/after token comparison. + +### A. Allocation correctness (no sampling semantics needed) + +| # | Check | Refutes if | +|---|---|---| +| G1 | Every request receives a contiguous prefix of its draft positions, and at least one row | A request receives position `k` without positions `< k`, or receives no row | +| G2 | The realized row budget equals the value the allocator selected, measured from the built metadata | Metadata disagrees with the allocation, so the budget is not being applied | +| G3 | Hold the historical global budget, the same proposal's legal cumulative prefix through `i`, and other allocation state fixed; perturb only that proposal's later confidence at `j > i` within its signal contract and confirm the verify decision at `i` is unchanged. Check the actual round and queue used; changing historical rates to select a different total budget is outside this test | The same proposal's later confidence changes inclusion of `i` under those fixed conditions; investigate the causal dependency and complete emitted distribution with G5 | + +### B. Sampling correctness against an independent oracle + +| # | Check | Refutes if | +|---|---|---| +| G4 | Compare the accepted-prefix decision against a separately implemented rejection rule, on synthetic draft and target distributions with known acceptance probabilities | The two disagree beyond the stated tolerance, which means the engine's accept/reject differs from the documented rule | +| G5 | At a fixed prefix compare the complete emitted next-token distribution against the target over many samples, with proposal `q` different from target `p`; include acceptance, rejection/residual resampling and target fallback | The complete emitted distribution differs from the target beyond the predeclared statistical criterion; accepted drafts alone are a selected subset and need not follow the target distribution | +| G6 | Grammar-constrained request under budgeting: confirm the per-position mask and the acceptance rate stay independent signals | One is derived from the other, or masks are applied at the wrong position | + +Token-for-token equality between budget-on and budget-off runs is **not** a valid check here. Budgeting changes how many drafts are verified, so it can legitimately change the accepted length and the token stream even with a fixed seed, because the sampling path consumes random draws in a different order. Use G4–G6 instead. + +### C. Cost and benefit + +| # | Check | Refutes if | +|---|---|---| +| G7 | Report verification rows, accepted tokens and total step time, budget on versus off; assess expected accepted tokens per total time alongside G9 | The declared cost objective or matched service acceptance is not met; rows and accepted tokens falling together is allowed if time falls sufficiently, and their ratio alone establishes no benefit | +| G8 | Report total step time decomposed into draft forward, verify forward and sampler | A component moves in a direction the cost model does not predict | +| G9 | End-to-end throughput and latency at a stated concurrency, with the draft forward included | The gain disappears once draft cost is counted; this confirms the caveat rather than refuting the mechanism | +| G10 | Acceptance-uniformity sweep: all-high, all-low, mixed | Any of the three underperforms plain decode by a margin larger than the allocator's overhead, which bounds where the procedure should be applied | +| G11 | Sampler cost share at production occupancy | Sampler dominates, so the budget cannot help | +| G12 | Peak memory at fixed concurrency, budget on versus off | Memory grows unexpectedly from wide top-k pools | +| G13 | Graph sizes outside the warmup sweep | The fitted slope extrapolates badly outside the collected range | +| G14 | Cross-backend comparison of rejection arithmetic | The two backends disagree, which makes the parity claim false | + +## What a PASS would require + +Two independent results, not one combined result: + +1. **Correctness**: G1–G3 plus G4–G5, at a stated set of sampling configurations. +2. **Benefit**: G7–G9 at a stated concurrency and device, reported as throughput and latency rather than as rows removed. + +G1 alone shows only that the allocator does not violate its own invariants. G9 alone shows only that a number moved upward. + +## What this skill must not claim + +- No efficiency gain. None was reproduced. +- No default behaviour. The budgeted path is opt-in. +- No cross-platform equivalence. +- No transfer to a model without a trained confidence head. +- No claim that accepted length stays constant when budgeting is enabled. diff --git a/src/inferrsi/data/skill_library/skill-index.json b/src/inferrsi/data/skill_library/skill-index.json index b9dec6f6d..d9b401b47 100644 --- a/src/inferrsi/data/skill_library/skill-index.json +++ b/src/inferrsi/data/skill_library/skill-index.json @@ -220,6 +220,14 @@ "diffusers-invocation-state": { "path": "frameworks/diffusers/diffusers-invocation-state", "backend": "common" + }, + "bound-windowed-attention-replay": { + "path": "repositories/deepinfer/bound-windowed-attention-replay", + "backend": "common" + }, + "budget-speculative-verification": { + "path": "repositories/deepinfer/budget-speculative-verification", + "backend": "common" } } } diff --git a/tests/check_installed.py b/tests/check_installed.py index 9e566b21c..e2d568740 100644 --- a/tests/check_installed.py +++ b/tests/check_installed.py @@ -17,7 +17,7 @@ "This check requires an installed package" ) records = knowledge_records() -assert len(records) == 97, "Knowledge is missing from the installation" +assert len(records) == 102, "Knowledge is missing from the installation" assert ( files("inferrsi") .joinpath("data", "repo_packs", "vllm_omni", "knowledge", "README.md") @@ -27,7 +27,7 @@ assert search_knowledge(repo="vllm_omni", backend="ascend") assert all(not item["verification"]["locally_verified"] for item in records) skill_index = search_knowledge(repo="inferrsi", limit=100) -assert len(skill_index) == 55, "Missing installed skill abstracts" +assert len(skill_index) == 57, "Missing installed skill abstracts" assert all(record["index_type"] == "packaged-skill" for record in skill_index) indexed = KnowledgeReader().read("inferrsi.ascend.skills.ascend_operator_profiling") assert "data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md" in indexed.content @@ -59,7 +59,7 @@ data = resource.read_bytes().replace(b"\r\n", b"\n") assert hashlib.sha256(data).hexdigest() == expected_hash, relative registry = json.loads(library.joinpath("skill-index.json").read_text(encoding="utf-8"))["skills"] -assert len(registry) == 55 +assert len(registry) == 57 for entry in registry.values(): assert library.joinpath(*entry["path"].split("/"), "SKILL.md").is_file() router = library.joinpath("workflows", "inference-engine-research") diff --git a/tests/test_knowledge_contract.py b/tests/test_knowledge_contract.py index dc5ccd6bb..39f6d7d9a 100644 --- a/tests/test_knowledge_contract.py +++ b/tests/test_knowledge_contract.py @@ -24,7 +24,7 @@ def test_local_profiles_preserve_core_and_structured_ownership(): assert metadata["kind"] == "skill" for readset in metadata["provenance_readsets"]: assert (sidecar.parent / readset).is_file() - assert len(normalized) == 50 + assert len(normalized) == 52 def test_contradictory_legacy_metadata_fails(tmp_path): diff --git a/tests/test_skill_index.py b/tests/test_skill_index.py index 6b14e82ab..a8cf73bd8 100644 --- a/tests/test_skill_index.py +++ b/tests/test_skill_index.py @@ -23,7 +23,7 @@ def test_complete_index_retains_metadata_states_and_unknown_license(): records = search_knowledge(repo="inferrsi", limit=100) names = {record["name"] for record in skill_records(ROOT)} registry = json.loads((ROOT / "skill-index.json").read_text(encoding="utf-8"))["skills"] - assert len(records) == len(names) == 55 + assert len(records) == len(names) == 57 assert {record["skill_metadata"]["name"] for record in records} == names states = [] for record in records: @@ -41,7 +41,7 @@ def test_complete_index_retains_metadata_states_and_unknown_license(): } assert set(record["source_files"]) == set(record["sources"][0]["paths"]) states.append(metadata.get("status")) - assert states.count("candidate") == 50 + assert states.count("candidate") == 52 assert states.count(None) == 5 profiling = next(r for r in records if r["title"] == "ascend-operator-profiling") assert profiling["skill_metadata"]["license"] == "UNKNOWN" @@ -161,7 +161,7 @@ def test_package_index_and_checkout_skill_have_distinct_read_boundaries(): assert reader.resolve("claim-contract") == "checkout:claim-contract" assert reader.read("claim-contract").metadata["kind"] == "skill" collected = reader.collect(repo="inferrsi", kind="knowledge", limit=100, material=True) - assert len(collected["items"]) == 55 + assert len(collected["items"]) == 57 assert not collected["collisions"] and not collected["omissions"] assert all(item["authority"] == "package" for item in collected["items"]) assert all("record.json" in item["resources"] for item in collected["items"]) @@ -191,9 +191,9 @@ def test_index_cli_search_show_context_and_execution_boundary(capsys): def test_hardware_filters_include_only_matching_and_shared_skills(): - for backend, count in (("ascend", 53), ("cuda", 42), ("cpu", 40)): + for backend, count in (("ascend", 55), ("cuda", 44), ("cpu", 42)): records = search_knowledge(repo="inferrsi", backend=backend, limit=100) assert len(records) == count assert all(record["backend"] in (backend, "common") for record in records) assert not (PROJECT / "skills").exists() - assert len(list(ROOT.rglob("SKILL.md"))) == 76 + assert len(list(ROOT.rglob("SKILL.md"))) == 78 diff --git a/tests/test_skill_library.py b/tests/test_skill_library.py index c169be062..d7ea3636c 100644 --- a/tests/test_skill_library.py +++ b/tests/test_skill_library.py @@ -154,7 +154,7 @@ def test_library_root_exposes_only_owner_categories_and_required_entrypoints(): assert {path.name for path in LIBRARY.iterdir() if path.is_dir()} == { "hardware", "frameworks", "repositories", "methods", "workflows", } - assert len(list(LIBRARY.rglob("SKILL.md"))) == 76 + assert len(list(LIBRARY.rglob("SKILL.md"))) == 78 assert not any(path.name == "zh-CN" for path in LIBRARY.rglob("*") if path.is_dir()) references = LIBRARY / "workflows/inference-engine-research/references" assert (references / "import.md").is_file()