Skip to content

feat: use opaque memory in HTTP APIs - #684

Draft
martintmk wants to merge 13 commits into
mainfrom
user/martintmk/20260821-use-opaque-pool
Draft

martintmk wants to merge 13 commits into
mainfrom
user/martintmk/20260821-use-opaque-pool

Conversation

@martintmk

@martintmk martintmk commented Aug 21, 2026

Copy link
Copy Markdown
Member

Summary

  • change HttpBodyBuilder to store OpaqueMemory while accepting any MemoryShared provider
  • make OpaqueMemory::new preserve an existing OpaqueMemory instead of adding another wrapper
  • change fetch dependencies to expose OpaqueMemory, allowing customers to choose their memory provider
  • keep GlobalPool as the default provider and retain the existing bytesbuf type names

Validation

  • full workspace clippy with all targets and features
  • full workspace tests with all features
  • affected-package formatting, no-default-features build, and external-type checks
  • repository spelling and generated README checks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
@martintmk martintmk added the agency-rocket Touched by a rocket skill label Aug 21, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (a6ca774) to head (ceac638).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff            @@
##             main     #684    +/-   ##
========================================
  Coverage   100.0%   100.0%            
========================================
  Files         554      555     +1     
  Lines       61509    60642   -867     
========================================
- Hits        61509    60642   -867     
Flag Coverage Δ
linux 89.3% <100.0%> (-10.7%) ⬇️
linux-arm 89.1% <100.0%> (-10.9%) ⬇️
scheduled ?
windows 90.0% <100.0%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread crates/bytesbuf/src/mem/opaque_memory.rs Outdated
Comment thread crates/http_extensions/src/body/builder.rs Outdated
Comment thread crates/fetch/src/custom.rs Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
@martintmk martintmk changed the title feat(bytesbuf): add opaque memory pool APIs feat: use opaque memory in HTTP APIs Aug 21, 2026
Comment thread crates/http_extensions/src/body/builder.rs Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
Comment thread crates/http_extensions/src/body/builder.rs Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c98d886c-c9b6-42aa-ac1e-990eb589d5c0
@github-actions

Copy link
Copy Markdown

⚠️ Potential breaking changes detected

cargo semver-checks flagged the following on this PR. This is informational -- breaking changes between commits are expected; the major-version bump happens at release time, not on every PR.

bytesbuf

     Cloning origin/main
    Building bytesbuf v0.8.0 (current)
       Built [   3.236s] (current)
     Parsing bytesbuf v0.8.0 (current)
      Parsed [   0.004s] (current)
    Building bytesbuf v0.8.0 (baseline)
       Built [   3.095s] (baseline)
     Parsing bytesbuf v0.8.0 (baseline)
      Parsed [   0.005s] (baseline)
    Checking bytesbuf v0.8.0 -> v0.8.0 (no change; assume minor)
     Checked [   0.017s] 196 checks: 195 pass, 1 fail, 0 warn, 49 skip

--- failure method_requires_different_generic_type_params: method now requires a different number of generic type parameters ---

Description:
A method now requires a different number of generic type parameters than it used to. Uses of this method that supplied the previous number of generic types will be broken.
        ref: https://doc.rust-lang.org/reference/items/generics.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/method_requires_different_generic_type_params.ron

Failed in:
  bytesbuf::mem::OpaqueMemory::new takes 1 generic types instead of 0, in /home/runner/work/oxidizer/oxidizer/crates/bytesbuf/src/mem/opaque_memory.rs:34

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [   7.379s] bytesbuf

fetch

     Cloning origin/main
    Building fetch v0.15.0 (current)
       Built [  27.386s] (current)
     Parsing fetch v0.15.0 (current)
      Parsed [   0.006s] (current)
    Building fetch v0.15.0 (baseline)
       Built [  26.054s] (baseline)
     Parsing fetch v0.15.0 (baseline)
      Parsed [   0.006s] (baseline)
    Checking fetch v0.15.0 -> v0.15.0 (no change; assume minor)
     Checked [   0.012s] 196 checks: 194 pass, 2 fail, 0 warn, 49 skip

--- failure constructible_struct_adds_field: externally-constructible struct adds field ---

Description:
A pub struct constructible with a struct literal has a new pub field. Existing struct literals must be updated to include the new field.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/constructible_struct_adds_field.ron

Failed in:

  field CustomDeps.memory in /home/runner/work/oxidizer/oxidizer/crates/fetch/src/custom.rs:56

     Summary semver requires new major version: 2 major and 0 minor checks failed
--- failure struct_pub_field_missing: pub struct's pub field removed or renamed ---
    Finished [  55.016s] fetch

Description:
A publicly-visible struct has at least one public field that is no longer available under its prior name. It may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/struct_pub_field_missing.ron

Failed in:
  field global_pool of struct CustomDeps, previously in file /home/runner/work/oxidizer/oxidizer/target/semver-checks/git-origin_main/2015f49773bae6feddac285d64bfecd2c655a4b2/crates/fetch/src/custom.rs:56

http_extensions

     Cloning origin/main
    Building http_extensions v0.9.0 (current)
       Built [   7.772s] (current)
     Parsing http_extensions v0.9.0 (current)
      Parsed [   0.004s] (current)
    Building http_extensions v0.9.0 (baseline)
       Built [   7.832s] (baseline)
     Parsing http_extensions v0.9.0 (baseline)
      Parsed [   0.004s] (baseline)
    Checking http_extensions v0.9.0 -> v0.9.0 (no change; assume minor)
     Checked [   0.016s] 196 checks: 195 pass, 1 fail, 0 warn, 49 skip

--- failure inherent_method_missing: pub method removed or renamed ---

Description:
A publicly-visible method or associated fn is no longer available under its prior name. It may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/inherent_method_missing.ron

Failed in:
  HttpBodyBuilder::with_custom_memory, previously in file /home/runner/work/oxidizer/oxidizer/target/semver-checks/git-origin_main/2015f49773bae6feddac285d64bfecd2c655a4b2/crates/http_extensions/src/body/builder.rs:96

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [  16.513s] http_extensions

Comment thread crates/bytesbuf/src/mem/opaque_memory.rs

@martintmk martintmk left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted by an AI agent

I found one test gap that leaves the new no-double-wrap invariant unprotected, plus two non-blocking coverage/naming improvements. The intentional public API breaks were not raised: the existing PR discussion explicitly requested the constructor removal and CustomDeps rename, and release tooling handles versioning. Correctness, resilience, telemetry, consistency, and measured performance checks found no runtime regression. cargo check -p bytesbuf -p fetch -p http_extensions --all-targets --all-features passed, and current required CI is green.

}

#[test]
fn accepts_existing_opaque_memory() {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted by an AI agent

The no-double-wrap regression test cannot distinguish nested wrappers

Problem
accepts_existing_opaque_memory only checks that reserve still works after wrapping twice. The previous implementation that always nested an OpaqueMemory produces the same capacity, so this test stays green if the new TypeId/downcast branch is removed.

Why this matters
The no-double-wrap behavior requested in this PR can regress without this test detecting it.

Suggested fix
Assert an internal invariant that distinguishes one erased layer from two, such as a test-only depth or type probe.


#[cfg_attr(miri, ignore)]
#[tokio::test]
async fn custom_deps_accept_custom_opaque_memory() {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted by an AI agent · Non-blocking

The custom-provider test does not prove provider wiring

Problem
This custom provider behaves exactly like GlobalPool, and the test only checks the HTTP status. It would still pass if CustomDeps::memory were ignored and the default pool were substituted; the corresponding HttpBodyBuilder test has the same gap.

Why this matters
The new provider-plumbing contract can break without either test detecting that body allocation bypasses the supplied provider.

Suggested fix
Use a counting or callback provider and assert that body allocation reaches it.

Comment thread crates/fetch/src/tokio.rs
pub clock: Clock,
/// Memory pool for usage-neutral memory allocations.
pub global_pool: bytesbuf::mem::GlobalPool,
pub global_pool: bytesbuf::mem::OpaqueMemory,

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted by an AI agent · Non-blocking

The Tokio dependency field understates the provider contract

Problem
global_pool now accepts an OpaqueMemory, so it can hold any MemoryShared provider rather than specifically a GlobalPool. CustomDeps::memory already uses the broader name.

Why this matters
The two public dependency structs describe the same contract inconsistently, which can mislead callers about what providers the Tokio path accepts.

Suggested fix
Rename this field and its generated builder and example call sites to memory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agency-rocket Touched by a rocket skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant