Skip to content

feat(arty_io_core): add injectable I/O driver contracts - #737

Draft
martintmk wants to merge 24 commits into
mainfrom
user/martintomka/20260907-add-arty-io-driver
Draft

martintmk wants to merge 24 commits into
mainfrom
user/martintomka/20260907-add-arty-io-driver

Conversation

@martintmk

@martintmk martintmk commented Sep 7, 2026

Copy link
Copy Markdown
Member

Summary

Introduces the initial stable arty_io_core contract for injecting I/O drivers after runtime startup.

  • DriverContext selects its associated provider, so get_context::<MyContext>() needs no registration value.
  • The first lookup initializes one driver instance on every active worker before returning the caller-local context.
  • Later lookups reuse the cached context without creating more drivers.
  • Multiple unrelated and semver-incompatible driver types can coexist in one runtime.
  • Driver remains thread-local, boxable as Box<dyn Driver<Context = C>>, and exposes only shared-reference callbacks.
  • Parker defines the latched wait/wake contract.
  • SystemTasks is a cloneable crate-owned handle that hides runtime shared-ownership plumbing.
  • Shutdown is initiated exactly once by the runtime and returns an object-safe boxed completion future.
  • Contexts and in-flight operations retain state through safe ownership handles rather than an unsafe is_inert contract.

The crate includes REQUIREMENTS.md, DESIGN.md, public-surface tests, and a modular two-worker example. The example registers two independent driver types after worker startup, performs in-memory I/O through both contexts, caches repeated lookups, drains all four driver instances, and rejects operations through retained contexts after shutdown.

Review

  • Multi-model reviews with Opus, Sonnet, and GPT found no remaining issues.
  • Pragmatic Rust public API review found no remaining issues after removing speculative Clone from DriverInit and replacing exposed Arc<dyn ...> plumbing with SystemTasks.
  • Package coverage is 100%.

Validation

  • just package=arty_io_core clippy
  • just package=arty_io_core format
  • just package=arty_io_core readme
  • just package=arty_io_core spellcheck
  • cargo test -p arty_io_core
  • cargo run -p arty_io_core --example two_thread_runtime
  • cargo coverage-gate --package arty_io_core
  • cargo public-api --all-features --manifest-path crates/arty_io_core/Cargo.toml

CI infrastructure blocker

The Windows coverage job fails after all 6,867 tests pass. cargo llvm-cov generates a 33,956-character llvm-cov.exe export command for 445 objects, exceeding the Windows 32,767-character process limit (os error 206). The failed job was retried once and reproduced identically. Linux and Linux ARM coverage, Codecov, mutation tests, static analysis, MSRV, semver, external-type exposure, spelling, and other completed checks pass.

martintmk and others added 5 commits September 7, 2026 13:59
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
@martintmk martintmk added the agency-rocket Touched by a rocket skill label Sep 7, 2026
martintmk and others added 9 commits September 7, 2026 14:44
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0

@martintmk martintmk left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI AGENT]: I reviewed all eight added public contracts (Driver, DriverContext, DriverInit, DriverProvider, Parker, Shutdown, SystemTask, and SystemTaskSpawner), their rustdoc, the example runtime, dependencies, naming, object safety, shutdown behavior, and the new public-surface tests. Two targeted probes reproduced lifecycle contradictions below. The remaining public surface is documented and intentionally narrow; I found no additional dependency, naming, performance, resilience, or test-weakening issue. CI was still running with no reported failures at the final check.

Comment thread crates/arty_io_core/src/driver.rs Outdated
Comment thread crates/arty_io_core/examples/two_thread_runtime/sample_driver.rs
@codecov

codecov Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.23077% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.5%. Comparing base (efd8178) to head (5626a06).
⚠️ Report is 18 commits behind head on main.

Files with missing lines Patch % Lines
crates/arty_io_core/src/system_tasks.rs 66.6% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##             main    #737      +/-   ##
=========================================
- Coverage   100.0%   98.5%    -1.5%     
=========================================
  Files         583     659      +76     
  Lines       62930   89164   +26234     
=========================================
+ Hits        62930   87857   +24927     
- Misses          0    1307    +1307     
Flag Coverage Δ
linux 98.5% <94.2%> (-1.5%) ⬇️
linux-arm 98.5% <94.2%> (-1.5%) ⬇️
scheduled ?
windows 98.6% <94.2%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

martintmk and others added 5 commits September 7, 2026 15:53
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 58677290-64b2-44fc-9e40-6c80b7e9e5c0

@martintmk martintmk left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI AGENT]: Pragmatic Rust public API review completed with no findings. Verified the all-features cargo public-api surface, generated rustdoc JSON, package tests, clippy, spelling, the two-driver example, and the 100% coverage gate.

Coverage: reviewed Driver, DriverContext, DriverProvider, DriverInit, Parker, SystemTasks, and SystemTask from the downstream consumer and evolution perspective. The audit cannot prove runtime behavior or soundness from API output alone; those were covered separately by tests and multi-model review.

The resulting surface removes speculative Clone from DriverInit, hides Arc<dyn ...> behind the cloneable SystemTasks handle, and documents the boxed shutdown future as the deliberate object-safe shape for Box<dyn Driver<Context = C>>.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 206cc953-3282-4edf-b114-ee2b7f518bdf
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 206cc953-3282-4edf-b114-ee2b7f518bdf
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 206cc953-3282-4edf-b114-ee2b7f518bdf
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 6a475d84-6296-497e-975b-7442019e433b

@martintmk martintmk left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted by an AI agent

Reviewed the exact PR head across API design, correctness, tests, performance, naming, telemetry, resilience, consistency, public API, and public documentation. I found three blocking issues and two non-blocking issues. Four are anchored inline; the remaining non-blocking issue is that the PR description still documents the obsolete DriverContext/Parker/boxed-future API and should be refreshed to match the current IoContext, interruptor, and blocking shutdown(self) contract.

Focused validation: cargo check -p arty_io_core passed; specialist runs also passed the 13 public-surface tests and the two-thread example. The focused correctness probe reproduced shutdown returning while admitted I/O was still active.

Posted by an AI agent · Non-blocking

The PR summary describes an obsolete arty_io_core API

Problem
The PR body still says DriverContext selects the provider, Driver exposes only shared-reference callbacks, Parker defines waiting, and shutdown returns an object-safe boxed future. At the pinned head, IoContext::provider selects the provider, process_completions takes &mut self, interruptor returns Waker, and shutdown(self) blocks and returns Result<(), ShutdownError>.

Why this matters
Reviewers and future readers are given a materially different registration, completion, and shutdown contract from the code and the new requirements/design documents.

Suggested fix
Refresh the PR summary to name IoContext, process_completions/interruptor, and the consuming blocking shutdown result, removing the obsolete Parker, shared-reference-callback, and boxed-future claims.

/// Panics when this worker's driver instance cannot be initialized. Driver registration is
/// runtime-fundamental: after one worker fails to initialize, the runtime cannot continue in a
/// coherent partially registered state.
fn create(self, context: DriverContext) -> Self::Driver;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted by an AI agent

DriverProvider::create makes environmental initialization failures fatal

Problem
DriverProvider::create returns Self::Driver and its contract requires implementations to panic whenever a worker instance cannot be initialized. Driver initialization can fail for runtime conditions such as exhausted OS resources or a permission change after the separately suggested capability check, so the public contract provides no typed way to report a normal initialization failure.

Why this matters
A runtime cannot roll back registration or report the affected driver reliably; with panic = abort the mandated path terminates the process, and even unwinding forces every runtime to treat ordinary setup failure as a panic across a foundational extension boundary.

Suggested fix
Make create return Result<Self::Driver, DriverInitializationError> (or the crate's canonical extensible error type), and let the runtime abort and roll back the partial registration explicitly after receiving the error.


fn shutdown(self) -> Result<(), ShutdownError> {
// Release publishes admission closure before graceful cleanup starts.
self.state.shutdown_started.store(true, Ordering::Release);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted by an AI agent

Example shutdown returns before admitted I/O completes

Problem
EchoContext::perform_io checks shutdown_started and then performs the operation without acquiring an active-operation lease, while EchoDriver::shutdown only stores the flag and immediately returns; SampleContext uses the same check-then-act pattern at sample_driver.rs:38. A focused probe using the exact head modules started a 256 MiB uppercase operation, waited for it to enter perform_io, and then shut down the runtime; shutdown completed and printed shutdown returned while operation was still active, contradicting R6's requirement that shutdown close admission and wait for active operations to drain.

Why this matters
The canonical runtime example demonstrates that a successful graceful shutdown may return while previously admitted I/O is still running, so driver authors following it can release resources or report shutdown completion before their operations and callbacks are finished.

Suggested fix
Give each example state a synchronized closed/active-operation lifecycle: atomically acquire a lease while admission is open, release it when the operation finishes, and have shutdown close admission before waiting with a bounded deadline for the active count to reach zero. Add a focused concurrent example test that proves shutdown cannot return before an admitted operation completes.


context.system_tasks().spawn(|| {});

assert_eq!(context.thread(), &worker);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted by an AI agent

The public-surface test never exercises SystemTasks::Debug

Problem
The compile-time assertion proves that SystemTasks implements Debug, but this test formats only DriverContext and ProviderContext. The matching package coverage run reports system_tasks.rs:39-41—the complete Debug::fmt implementation—as uncovered, leaving arty_io_core at 94.23% line coverage and matching Codecov's three missing lines.

Why this matters
The new test suite does not preserve the observable debug representation it advertises, and the uncovered implementation currently contributes to the failing codecov/patch and codecov/project checks.

Suggested fix
Add an assertion that formats the reachable SystemTasks handle and checks its stable type name, for example assert!(format!("{:?}", context.system_tasks()).contains("SystemTasks"));.

/// coalesced, but an interrupt is never dropped. The returned interruptor remains safe to
/// invoke after the driver is dropped.
#[must_use]
fn interruptor(&self) -> Waker;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted by an AI agent · Non-blocking

Driver::interruptor obscures that the handle is a standard Waker

Problem
This method returns std::task::Waker, while the workspace names the same returned concept waker (for example, arty_executor::wake::Wake::waker at crates/arty_executor/src/wake.rs:152). Renaming it to the newly coined interruptor makes this trait the only sibling to hide the foundational type behind a different noun.

Why this matters
Driver implementers and runtime authors must learn a crate-specific synonym, and the divergence propagates through the public trait, tests, examples, requirements, design document, generated README, and spelling dictionary.

Suggested fix
Rename the method back to waker and update those references; the current PR title does not mention either term and needs no change.

Suggested change
fn interruptor(&self) -> Waker;
fn waker(&self) -> Waker;

Add a separate proposal for coordinating independently registered I/O drivers across shared IOCP and multiple io_uring sources. Document participation modes, notification routing, service and parking, safe draining, trade-offs, and public references without changing the current contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9fcc1db7-eda6-4c33-bbbe-77ff8b5e5ae6

@martintmk martintmk left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted by an AI agent

Reviewed the current PR head across API design, correctness, tests, performance, naming, telemetry, resilience, consistency, public API, and public documentation. Three blocking issues and two non-blocking issues remain applicable.

Four are already represented by unresolved current threads: provider creation mandates panic, graceful shutdown can return before admitted I/O drains, SystemTasks::Debug remains untested, and Driver::interruptor obscures its Waker contract. One new non-blocking naming issue is posted inline: SystemTasks names submitted work rather than its spawning capability.

Focused validation passed all 13 arty_io_core public-surface tests and the two-thread runtime example. Paired cargo public-api and rustdoc JSON extraction completed for the exact base/head snapshot.

/// The facility remains available until every driver that received it has completed shutdown, so
/// cleanup work submitted during shutdown can still run.
#[derive(Clone)]
pub struct SystemTasks {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted by an AI agent · Non-blocking

SystemTasks names the submitted work instead of the spawning capability

Problem
The type is documented as a cloneable handle and its only operation is spawn, but SystemTasks reads as a collection of tasks. The workspace names this role anyspawn::Spawner (crates/anyspawn/src/spawner.rs:99) and the boxed work BoxedBlockingTask (crates/anyspawn/src/custom.rs:51).

Why this matters
This is a new stable public contract, so consumers will otherwise have to learn an unexplained synonym and a later correction would require a breaking rename.

Suggested fix
Rename the handle to SystemTaskSpawner (and update DriverContext::system_tasks, docs, tests, examples, and generated README references); retain the narrower wrapper rather than exposing the broader anyspawn::Spawner capability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agency-rocket Touched by a rocket skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant