refactor: collapse duplicated harness and engine implementations - #57
Merged
Merged
Conversation
Benchmarks: consolidate the per-binary copies of env parsing, RSS sampling, file cleanup guards, write retry, latency sampling, and splitmix64 into shared benchmarks::config and the new benchmarks::harness. The copies had drifted; the unified versions keep knob names and report formats, and align a few error messages with the existing config style. Library: give the little-endian field codecs and the zeroed-field CRC32C pattern one home in the new codec module instead of four reimplementations (the shared reader also guards offset overflow, which two copies lacked). Collapse the identical POSIX and io_uring IoEngine impl blocks into trait default methods behind inner() and runtime_io_stats() accessors, share retry_interrupted with the uring engine, keep one is_read_pressure classifier, delegate the three global_slot computations to IndexPartitionRange, and move the three TestFile copies into fixtures so file-based tests clean up on failure.
Use the shared setting reader for I/O mode and L1 eviction policy so non-UTF-8 values fail instead of selecting defaults. Reuse boolean parsing for IOPOLL and keep the generic setting helper private. Add a subprocess regression test that supplies invalid environment bytes without mutating the parallel test process.
Return a plain RuntimeInner reference from IoEngine accessors. Default operations only borrow the runtime, while each implementation retains its own Arc ownership.
Delegate fixed-width index readers to the shared codec and retain local expects for their internal bounds contract. Existing golden fixtures cover the persisted representation.
Inject a competing read at the shutdown snapshot boundary through a per-runtime test callback. Keep issued requests observable and verify that late submissions are rejected by the admission fence. Both shutdown tests pass; temporarily removing the fence makes the late-read regression fail.
Replace the engine trait and RuntimeInner wrappers with a concrete IoEngine. Keep driver selection in constructors and share request admission, completion, cancellation, statistics, and shutdown through Arc<IoEngine>. Use shared file statistics directly and retain workers as they start so partial startup failures use the same shutdown path. Preserve request-owned buffers and slots through physical completion.
Adapt background I/O timeout recovery to the concrete IoEngine and migrate its tests. Preserve recovery interruption before mutation and worker shutdown. Validation: cargo x check, cargo x test, cargo x lint, Linux io-uring all-targets check, and Linux all-targets Clippy with warnings denied.
Expose one checksum function over borrowed slices and remove the incremental wrapper and checksum helpers from the field codec. Keep checksum layouts and comparisons within each persistent format, with shared encode/decode calculations and existing length validation. Migrate all callers and retain reference, golden-format, and property coverage. Validated with cargo x check, cargo x test, cargo x lint, and Linux io-uring all-targets check and Clippy.
leiysky
added a commit
that referenced
this pull request
Sep 20, 2026
#57 collapsed BackendIoEngine; the Adaptive pause test now uses IoEngine::for_test.
leiysky
added a commit
that referenced
this pull request
Sep 20, 2026
* feat: add adaptive fill admission before I/O timeouts * refactor: fail-open fill observations and rename FillLimits A full observation table must not fail cache I/O, and the 100 ms monitor must not be reset by each submit. Observe would_reject now tracks pause and budget policy only. Shared rate ceilings are FillLimits rather than AdaptiveFillOptions. * refactor: pace fill control on I/O workers Keep put competing only for staging. Workers checkpoint old I/O and pace non-essential flush so Adaptive can pause without a monitor thread. * fix: keep fill pacing usable at configured ceilings Raise the record ceiling to the packed-credit limit, preserve refill time across sub-tick wakes, and refund only consumed burst. Document that FillLimits are instance-wide. * fix: construct fill-pressure tests with the unified IoEngine #57 collapsed BackendIoEngine; the Adaptive pause test now uses IoEngine::for_test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Consolidate duplicated benchmark helpers, persistent-format codecs, and I/O engine behavior. The library's public API and persisted formats remain unchanged.
Design Notes
Callers share Arc; the final owner shuts down its workers. Driver-specific resources stay with the driver, and both paths report through shared file counters. Issued requests retain their buffers and capacity through actual completion. Partial worker startup uses the same shutdown path.
Shutdown race tests now coordinate a competing read at the runtime's snapshot boundary using a per-instance test callback. This preserves deterministic coverage without a mock engine trait. Removing the admission fence makes the late-read regression fail.
Validation:
cargo x check,cargo x test(including golden fixtures and extended tests),cargo x lint, Linux io_uring cross-compilation of all targets, and Linux Clippy with warnings denied. The benchmark subprocess regression test also fails against the previous enum parsers and passes with the fix. Linux io_uring runtime execution remains covered by CI.