Add: TMR kernel-mode resource contracts and init admission（2a） - #2177
Open
Leaf-Salix wants to merge 4 commits into
Open
Add: TMR kernel-mode resource contracts and init admission（2a）#2177Leaf-Salix wants to merge 4 commits into
Leaf-Salix wants to merge 4 commits into
Conversation
Leaf-Salix
marked this pull request as draft
September 9, 2026 09:32
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (44)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughThe change adds kernel-mode runtime contracts, lifecycle state, borrowed-device handling, pipeline admission, ABI exports, callable scalar-count metadata, and validation tests across onboard and simulated host runtimes. ChangesKernel mode and callable metadata
Priority: ➖ Normal Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant ChipWorker
participant host_runtime
participant KernelExecutionState
participant PipelineContract
Client->>ChipWorker: initialize runtime
ChipWorker->>host_runtime: resolve kernel lifecycle symbols
Client->>host_runtime: simpler_kernel_mode_init
host_runtime->>PipelineContract: validate kernel contract
host_runtime->>KernelExecutionState: initialize borrowed context
KernelExecutionState-->>host_runtime: lifecycle state
host_runtime-->>Client: initialization result
Merge Risk: ⚪ Minimal · up to Kernel-mode initialization remains unsupported and does not establish kernel-mode state, so the reviewed guard paths do not affect current runtime behavior. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 13.81% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 210 functions across 37 files. (7 skipped: 7 unsupported.) Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. A rabbit checks the kernels bright, Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2153406a04
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Kernel mode borrows the caller's device and stream for a bounded asynchronous operator. Define its C ABI, invocation envelope, and lifecycle contracts without enabling kernel execution in any backend. - Add the four simpler_kernel_mode entries, validating unsupported stubs, and distinct invalid-argument and invalid-state errors. - Latch the execution identity once. Program init latches PROGRAM before side effects; errors and finalization cannot change the mode. Separate thread binding, program attachment, and borrowed-device adoption, and guard the onboard device lifecycle by that identity. - Latch PROGRAM in ensure_acl_ready before its first ACL call. The entry is reachable without simpler_init, so taking ACL ownership is itself what makes a context a program context; leaving it unlatched would let a later kernel latch coexist with acl_ready_, whose finalize resets a device the context does not own. - Decide a kernel-mode arena capacity refusal across every region before touching any of them. The commit sequence rolls all regions back on failure, so a refusal raised from inside it would release the committed bases a captured graph still names. - Pin the 40-byte invocation header and every field offset. Make its trailing reserved bytes explicit and require both reserved fields to be zero. Payload validation belongs to the AICPU consumer. - Derive scalar counts from the callable signature. Preserve the existing factory signatures and serialized layout, with fixed callable offsets and tests that ignore historical padding bytes. - Require the kernel capability probe when loading a runtime. Resolve its lifecycle entries only for a supported context and publish the pointers after initialization succeeds. - Provide the kernel phase/cleanup state machine and restricted, non-throwing operation tables, compiled in its unit test until production integration. Keep stream/event handles encapsulated. - Cover wire bytes, scalar counts, error contracts, cleanup retries, arena refusal over real committed regions, and shared-library capability negotiation including failed-init recovery. Keep uniform-export checks for all eight components. Kernel init remains unsupported, so no production call latches KERNEL. Production integration must supply generation checks, safe enqueue/close serialization, and graph-resource lifetime enforcement.
Compute TMR kernel arena requirements from the existing per-architecture sizing and reserve-only layout without acquiring device resources. Keep candidates call-local and publish output only on success. Share mode-aware contract and topology validation while keeping the TMR resource set explicit. Preserve valid program behavior and reject unserviceable stream declarations before context creation. Define execution modes in a neutral header shared with the unchanged invocation envelope. Connect the internal builder to validating kernel init stubs; HBG remains unsupported and kernel execution stays disabled. Cover sizing bounds, packed input, independent concurrent calls, real loader admission, sim entry behavior, and C/C++ header compatibility. Relevant tests and sim partitions pass. Full CTest retains the existing profiler quiesce failure (139/140); onboard validation remains pending.
Use the upstream execution-mode header as the single definition. Keep resource contracts independent of kernel lifecycle and invocation headers, and verify that boundary through compiler dependencies. Leave invocation layout ownership and its original wire tests with K1; remove the resource-contract test that additionally pins wire offsets. TMR sizing, resource validation, and init preflight remain unchanged.
Classify invalid TMR sizing input as INVALID_ARGUMENT while retaining INTERNAL for producer invariants and an invalid output pointer. Update entry tests to the shared kernel ABI error contract. Give capability-loader fixtures valid execution-stream declarations without weakening admission or retry assertions.
Leaf-Salix
force-pushed
the
dev/kernel-pr2a-tmr
branch
from
September 11, 2026 09:08
2153406 to
8f79a42
Compare
Leaf-Salix
marked this pull request as ready for review
September 11, 2026 09:29
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Supporting L2 kernel mode in TMR (
tensormap_and_ringbuffer) requires separating resource requirements from resource allocation, mutable execution state, and task execution. Requirements must be computable and validatable before execution, without invoking the existing program bind path just to discover sizes: no tensor staging, device allocation, or runtime image upload.This PR connects real resource sizing → shared validation → admission in the production kernel init entry:
This is not a kernel executor implementation.
simpler_kernel_mode_supported()still returns0, and valid init still returnsPTO_RUNTIME_ERR_UNSUPPORTED. No kernel resources are established, and no context becomes capable of kernel execution.Resource contract and admission
Runtime producers calculate the requirements
An internal hook is declared in
src/common/platform/include/host/kernel_pipeline_contract.h:a2a3anda5TMRhost/runtime_maker.cppimplementations reuse existing configuration parsing, size formulas, and their architecture-specificruntime_reserve_layout().HostApicalls. It does not enter program bind, tensor staging, or cache paths.UNSUPPORTEDstub with the same signature so the shared C API links in every variant. HBG kernel sizing is not implemented by this PR.get_pipeline_contract(void)signature and its read-only program declaration are unchanged.The TMR kernel declaration is below. The existing program declaration and depth of
2are unchanged.1GM_HEAPDEVICE_SCRATCH, nonzero required usable bytes derived from sizingGM_SMDEVICE_SCRATCH, nonzero required usable bytes derived from sizingRUNTIME_IMAGEDEVICE_SCRATCH, nonzero required usable bytes derived from layoutTASK_ARGSHOST_PER_RUN,bytes_per_copy = 0AICPU_STREAM/AICORE_STREAMEXEC_HANDLE,bytes_per_copy = 0Zero bytes for
TASK_ARGSmeans this field does not express an argument-size limit, not that arguments consume no memory. The three arena sizes describe required usable bytes per copy, not total committed HBM, capacity budgets, or callable residency budgets. Stream entries declare execution roles; copy counts derived by the helper do not prescribe the number of physical streams to create.Shared admission without coupling runtime-specific policies
src/common/worker/pipeline_contract.hretains the single-argument program validator and adds explicit-mode validation, stream serviceability checks, and a separately named TMR completeness helper.is_valid_tmr_kernel_pipeline_contract()checks TMR's six resource kinds, their classes, and depth1. It is called by the TMR builder. Shared init neither imposes this complete resource set on HBG nor branches on runtime names.simpler_kernel_mode_init()entries invoke the builder and shared admission after K1's basic argument checks. Invalid TMR configurations/contracts returnINTERNAL; successful admission still ends atUNSUPPORTED. C++ exceptions are translated toINTERNALat this C API boundary.Of K1's kernel-mode definitions, the shared contract depends only on the execution-mode definition, not the K9 invocation layout, kernel lifecycle state objects, or execution operation tables. New tests enforce this include boundary. This PR does not duplicate or redefine the K1 state machine.
Lifecycle and program-path impact
Lifetime and concurrency: The builder borrows config only for the duration of the call. Sizing, the temporary arena, and the candidate contract are call-local; no config pointer is retained and no mutable global/TLS cache is introduced. The caller-exclusive
outis assigned once, at the end of a successful call; failure leaves it unchanged. Independent calls may run concurrently. Sharing an output or concurrently modifying inputs or process environment variables is unsupported. Existing cold-path logging is retained; zero host allocation is not promised.Boundary safety: Packed configuration fields retain their existing
memcpyreads. Ring ranges, theINT32_MAXbound on the total window, heap sums, and necessary size/alignment bounds are checked before layout calculation, preventing invalid configurations from reaching assertion or truncation paths.Program-path impact:
ChipWorker::init()only gains a stream declaration serviceability check, rejecting invalid declarations before context creation. Existing valid program contracts, depth, resource setup, and execution paths are preserved. Declarations that violate the new stream admission rules fail earlier. Public C ABI function signatures and struct layouts are unchanged.Left to follow-up PRs: Stream/event creation and capture integration; arena capacity establishment, commit, and rollback; argument snapshots; device dispatch; ACLGraph capture/replay; and callable residency/generation management. Neither depth
1nor the builder concurrency tests establish safety for device execution, replay, or close races. Validating requirements does not guarantee that device capacity is available.Tests
Validation corresponds to the final version,
2153406a. The results below distinguish committed tests, inherited baseline regression tests, and supplemental checks. They are not a claim that the full CI matrix has passed.Coverage added or extended in this PR
tests/ut/cpp/hierarchical/test_pipeline_contract.cpp: Mode/byte rules, TMR's six resource kinds and depth, missing/duplicate/misclassified streams, structural validity versus serviceability, and shared admission that does not impose TMR's resource set on HBG.tests/ut/cpp/common/test_trb_runtime_temp_buffer.cpp(both architecture-specific maker targets): Default and explicit configurations, genuinely unaligned packed input, invalid rings, window/heap/alignment overflow, and unchanged output on failure. Small-configuration requirements are compared with actual bind requests. Large configurations reserve layout without device allocation. Four threads use independent config/output pairs, each making 32 calls and comparing results.tests/ut/cpp/hierarchical/test_pipeline_contract_loader.cpp: A fixture runtime SO with official function signatures exercises the realChipWorkerloader. Invalid stream declarations cause zero context factory calls; a valid declaration reaches the factory, where a sentinel failure proves that admission was passed. Four real sim runtimes are also loaded to check invalid TMR init returningINTERNAL, valid init/HBG stubs returningUNSUPPORTED, supported/committed remaining zero, and launch remaining rejected.tests/ut/py/test_host_runtime_abi.py: Standalone and cross-order inclusion of execution-mode and invocation headers under C11/C++17, plus dependency isolation between the shared contract and K9/lifecycle headers. Existing export checks for all eight components are retained.Executed results
a2a3/a5 × sim/onboard × TMR/HBG: 8/8 passed. This validates exports, not device execution.a2a3simanda5sim: 4 passed.0. Coverage includes TMR/HBG, pipeline slots, prepared callables, stream reuse, concurrent prepare, and the DFX cases selected by default.The A3 hardware environment used
Ascend910_9392, CANN 9.0.0, and the repository-pinned PTO-ISA version. Hardware tests ran under an exclusive single-device lock. The source tree used for UT/sim validation matched the final commit's contents; before onboard validation, the checkout was advanced to that commit and verified clean.The only L2 regression skip was
TestPreparedCallableHbg::test_failed_double_prepare_closes_unpublished_host_handle: the test explicitly validates the unpublished hostdlopenguard only on sim. The 208 deselected cases are not counted as passes. Standalone vector smoke and the L2 regression may overlap, so their counts are not summed as unique test cases.Main reproduction commands, from the repository root after installation and UT target builds following the repository testing guide:
Not validated or not implemented: A5 hardware, the full UT/ST suites, manual/SDMA-specific suites, multi-device execution, L3/L4 full-network integration, actual kernel launch/capture/replay, and subsequent capacity establishment/rollback. Passing program regression tests does not imply that kernel mode supports those execution capabilities.
Commits
Two separate commits are retained on top of #2064:
48e37b3b—Add: declare and validate TMR kernel resource requirements: Resource sizing, shared validation, production entry admission, and tests.2153406a—Update: align resource contract dependencies with kernel ABI: Use K1's single execution-mode definition, remove 2a's additional coupling to a specific K9 layout, and strengthen the shared contract's dependency-isolation test.Merge order: #2064 → this PR (2a). This PR neither replaces K1 nor implements HBG sizing (2b) or the subsequent kernel execution modules.
中文总结
dc1268cd，新增48e37b3b、2153406a两个 commit；必须等 Add: kernel-mode C ABI skeleton and wire headers (K1) #2064 合并后再合并本 PR。PipelineContract声明与校验，以及 sim/onboard 真实 kernel init 准入；不是 kernel 执行实现，supported 仍为0，合法 init 仍返回UNSUPPORTED。2保持不变，TMR kernel 声明 depth1。TASK_ARGS = 0不表示参数不占内存。实际定容、分配、回滚及 callable 管理由后续 PR 负责。