diff --git a/docs/reference-matrix.md b/docs/reference-matrix.md index 1429e1f14..40ccea3a9 100644 --- a/docs/reference-matrix.md +++ b/docs/reference-matrix.md @@ -105,6 +105,16 @@ It preserves document versus implementation evidence, source/recipe dates and attribution, and different mutation, completion and measurement boundaries. The two sources do not establish a tested combined runtime stack. +### TensorRT-LLM retained source responsibilities + +The [source and shared-assumption map](../src/inferrsi/data/skill_library/README.md#tensorrt-llm-source-and-responsibility-routing) +connects the historical `f33e0e5fd` skill-document row to the separately pinned +`c9b18eefa731dc59b0091bdfff6d890aa9d1e98e` PD/KV-transfer peer card. +It distinguishes startup and request-time denominators, local H2D and network +transport, and the workload/layout/resource controls needed for a PD comparison. +Original prefix precision, missing methodology inspection dates, mixed-source +attribution, owner readsets and candidate/imported limits remain unchanged. + ### Reviewed systems-paper mechanisms `mined` below means pack-mediated extraction into the linked candidate artifacts. diff --git a/docs/searchspace-matrix.md b/docs/searchspace-matrix.md index ddbb9f69a..5539d4f77 100644 --- a/docs/searchspace-matrix.md +++ b/docs/searchspace-matrix.md @@ -323,6 +323,16 @@ boundaries; graph replay is not staging/upload completion, and DCP completion does not establish replay eligibility. Existing source-derived maturity and resource coverage remain unchanged; no combined protocol is introduced. +## TensorRT-LLM shared assumptions + +Use the [methodology and disaggregation map](../src/inferrsi/data/skill_library/README.md#tensorrt-llm-source-and-responsibility-routing) +for the TensorRT-LLM Accelerator, RAM/Host, SSD/Storage, Communication/Network +and OS/System entries. Its evidence row fixes the comparison contract; the five +resource rows assess host exposure, startup/request units, transport/layout and +queue costs, kernel correctness and placement controls against the existing +peer card. The retained methodology and PD pins are separate source scopes; +this adds no measured/green cell, new runtime mechanism or compatible-stack claim. + ## Issue #27: Drama3 NPU readiness (guidance only, no new coverage) The [Drama3 NPU readiness supplement](drama3-npu-readiness.md) is a scoped task diff --git a/src/inferrsi/data/knowledge_packs/README.md b/src/inferrsi/data/knowledge_packs/README.md index 03f35cdba..947b40009 100644 --- a/src/inferrsi/data/knowledge_packs/README.md +++ b/src/inferrsi/data/knowledge_packs/README.md @@ -21,6 +21,12 @@ to the existing [DCP async-save owner](../skill_library/frameworks/pytorch/dcp-a keeping its fixed implementation and live recipe separate from compiler documentation, with each owner's completion and version boundaries intact. +For TensorRT-LLM, the [methodology and disaggregation map](../skill_library/README.md#tensorrt-llm-source-and-responsibility-routing) +routes the existing mixed-source performance methods alongside the +[PD/KV-transfer card](peer/tensorrt_llm/cuda/llm/prefill-decode-disaggregation.json). +It assesses shared measurement controls and separate startup, host and network +costs while retaining the card's source revision and transfer-hypothesis limits. + ## Storage and compatibility ```text diff --git a/src/inferrsi/data/skill_library/README.md b/src/inferrsi/data/skill_library/README.md index 80c901702..0fb3b5978 100644 --- a/src/inferrsi/data/skill_library/README.md +++ b/src/inferrsi/data/skill_library/README.md @@ -89,6 +89,31 @@ recompilation, capture and steady-state replay; DCP measures including first-use versus reused buffers. These are owner-specific checks, not a combined compiler/checkpoint calling protocol or compatibility result. +## TensorRT-LLM source and responsibility routing + +TensorRT-LLM's retained methodology row covers **skill documents**, not an +engine-runtime audit. It records prefix `f33e0e5fd`; procedures keep +`f33e0e5f`, while the [ceiling sidecar](methods/performance-ceiling-analysis/metadata.json) +retains the longer **prefix** `f33e0e5fd55b`. An original inspection date for +that methodology extraction is not individually recorded. These are partial +source identities, not a verified common runtime version. + +The separate [PD/KV-transfer peer card](../knowledge_packs/peer/tensorrt_llm/cuda/llm/prefill-decode-disaggregation.json) +records `c9b18eefa731dc59b0091bdfff6d890aa9d1e98e`, accessed 2026-10-15. +Its upstream-reported technique remains imported/unverified. The methods below +remain candidate / empirical validation `not_run`, with their other sources +and source notices retained. This cross-reading assesses when to combine the +guidance; it establishes no cross-pin implementation compatibility. + +| Decision / resource | Existing entry | Shared prerequisite or accounting judgment | Keep separate | +| --- | --- | --- | --- | +| Evidence and measurement | [Evidence levels](methods/dense-performance-evidence/SKILL.md#claim-evidence-matching), [headroom inputs](methods/performance-ceiling-analysis/SKILL.md#evidence-to-inspect-first) and [case reuse](methods/inference-optimization-casebook/SKILL.md#adapt-do-not-copy) | Apply the peer card's fixed input/output-length and arrival distributions, same total device budget and SLO workload; report TTFT, TPOT, tail latency, goodput and total GPU/network resources | Analytical/component evidence and a precedent do not establish service gain; re-derive model/hardware-specific values | +| Accelerator | [Kernel verification](methods/kernel-writing-guide/SKILL.md#workflow-verify-before-benchmark) | A kernel change still needs reference parity over target shapes/dtypes and critical-path profiling | Kernel correctness does not establish reduced prefill/decode interference or end-to-end PD benefit | +| RAM / Host | [Serving-trace prerequisites](hardware/cuda/host-overhead-profiling/SKILL.md#prerequisites) and [confirmed host bottleneck](hardware/host/host-overhead-optimization/SKILL.md#confirming-the-bottleneck) | Establish exposed host work from serving traces before attributing a PD regression to CPU preparation | A line profiler locates CPU time; it does not establish that CPU is limiting or measure the KV interconnect | +| SSD / Storage | [Startup accounting](methods/weight-loading-optimization/SKILL.md#2-account-startup-fixed-overhead-separately) | Loading/warmup is once per process; request KV delivery needs its own per-request bytes and timing | Do not combine those denominators; disk-loading mechanics retain their separately attributed sources | +| Communication / Network | [Disaggregation peer card](../knowledge_packs/peer/tensorrt_llm/cuda/llm/prefill-decode-disaggregation.json) | Consider PD with measured prefill/decode interference, compatible KV layouts and qualified NVIDIA GPU transport; include transfer, layout-conversion and queue costs, and verify backpressure, cancellation and failed-worker recovery | Host/device loading overlap and local H2D do not establish this interconnect path; diffusion-stage splitting does not inherit the autoregressive PD cost model | +| OS / System | [Placement discipline](hardware/host/os-process-placement/SKILL.md#1-cpu-affinity-check-then-pin-then-keep-constant) | Record and control affinity, environment and device contention across matched pool comparisons | Placement methodology controls an experiment; it does not implement PD routing or OS-priority scheduling | + ## Engine knowledge Five repository entry skills, 15 focused sub-skills and one research router