helly25 · Nov 3, 2026 · Nov 2, 2026 · Nov 2, 2026 · Nov 2, 2026 · Nov 2, 2026
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -24,7 +24,7 @@ repos:
     hooks:
       - id: check-added-large-files
         # Retain the exact lossless cross-platform performance evidence archives.
-        exclude: ^docs/benchmarks/((executor-copy-audit|bounded-read-ahead)-20261101|(directory-task-slots|packed-visit-snapshots|bounded-match-results|useful-match-workers|direct-evaluator-workers|derived-content-handoff|immutable-default-catalogs|prepared-diff-configuration|prepared-hash-operands|aggregate-comparison-totals|byte-shingle-representations|archive-input-buffers)-20261102|(digest-and-record-storage|caller-owned-streams)-20261103)\.json\.gz$
+        exclude: ^docs/benchmarks/((executor-copy-audit|bounded-read-ahead)-20261101|(directory-task-slots|packed-visit-snapshots|bounded-match-results|useful-match-workers|direct-evaluator-workers|derived-content-handoff|immutable-default-catalogs|prepared-diff-configuration|prepared-hash-operands|aggregate-comparison-totals|byte-shingle-representations|archive-input-buffers)-20261102|(digest-and-record-storage|caller-owned-streams|direct-source-materialization)-20261103)\.json\.gz$
       - id: check-merge-conflict
       - id: check-yaml
       # BINARY_TYPES below: archive containers are binary but can START with text (a phar begins with

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -3,6 +3,8 @@
 
 # 0.11.0
 
+- Fill bounded source ranges directly into final storage and reuse owned decoder blocks during materialization, preserving read limits, errors and budgets.
+
 - Reuse caller-owned input storage for memory-backed and nested archive-member streams while preserving read errors, sparse content and shared budgets.
 
 - Reduce cached-digest copies and plain/escaped output allocation churn while preserving snapshot lifetimes and exact output bytes.

diff --git a/TODO.md b/TODO.md
@@ -455,9 +455,10 @@ EP/JC/P work; candidates are not speedup claims until measured on both platforms
       nested containers, short reads, errors, cursor interleaving and budget/owner lifetimes. Keep an
       explicitly incapable legacy benchmark control and separately exercise actual memory/member sources.
       [Qualification, costs and rejected rewrite](docs/benchmarks/caller-owned-streams-20261103.md) retain all four variants and duplicate-baseline controls.
-- [ ] O19d: Qualify direct filling of materialized ranges/results through caller-owned reads, with
+- [x] O19d: Qualify direct filling of materialized ranges/results through caller-owned reads, with
       exact overflow/limit probes, EOF, changing capabilities and owned fallback. Keep range memory
       reservations and replay accounting; do not silently treat a zero-length read as proof of EOF.
+      Keep [qualification, rejected fallback and slower controls](docs/benchmarks/direct-source-materialization-20261103.md).
 - [ ] O20: Qualify optional bulk metadata by backend capability with precise fallback, while
       keeping metadata-free traversal free of extra probes and extended-stat work.
 - [x] O21a: Qualify staged one-word keys, borrowed normalized shingle windows and an isolated byte

diff --git a/bazelmod/clang_tidy_targets.bzl b/bazelmod/clang_tidy_targets.bzl
@@ -10,6 +10,7 @@ keeps the two sets identical.
 
 CLANG_TIDY_MANUAL_TARGETS = [
     "//tools:archive_source_benchmark",
+    "//tools:read_source_benchmark",
     "//xff/cli:globals_benchmark",
     "//xff/engine:read_accounting_test",
     "//xff/engine:compare_benchmark",

diff --git a/docs/benchmarks/direct-source-materialization-20261103.json.gz b/docs/benchmarks/direct-source-materialization-20261103.json.gz
diff --git a/docs/benchmarks/direct-source-materialization-20261103.md b/docs/benchmarks/direct-source-materialization-20261103.md
diff --git a/docs/performance-analysis.md b/docs/performance-analysis.md
@@ -3584,3 +3584,41 @@ all selected root/extension release tests and focused sanitizers remain part of
 The initial nested fixture command missed `--archive-depth=2`; its warmup oracle failed before
 accepting any native row. The corrected matrix uses explicit depth two. O19b seeking/index/replay
 and admission, and O19d direct final-buffer materialization, remain open.
+
+### O19d: fill final range and materialization storage
+
+`ReadSourceRange` already reserves its bounded result, but previously allocated owned intermediate
+blocks and copied them into that result. The selected path fills uninitialized final storage and
+commits only initialized bytes. It retains the exact range reservation, offset-overflow/64 MiB
+checks, zero-range behavior and sequential `OpenAt` replay accounting. Positive short reads
+continue; failures never become EOF.
+
+`ReadSourceBytes` uses the same checked span path when supported, reusing spare capacity and SSO.
+Incapable decoders keep direct owned reads and append; the first owned block transfers into the
+empty result, avoiding another allocation/copy. Observe capability once for each read, rather than
+caching it across cursor progress. At an exact limit, including zero, request one positive byte to
+distinguish EOF from overflow. Clamp logical length to the string maximum before arithmetic and
+reject impossible counts before committing string size. String capacity can still grow beyond
+logical size; this is not an RSS or total-process-memory limit.
+
+[The cross-platform qualification](benchmarks/direct-source-materialization-20261103.md) separates
+range-only, bytes-only and combined variants, direct-source and archive kernels, native commands,
+identical-baseline confirmation, fixed-iteration tiny cases, allocation diagnostics and footprint.
+Actual memory sources and explicitly incapable legacy sources are distinct. Caller-owned storage
+removes copies and intermediate allocations; virtual dispatch, cursor creation, archive indexing
+and decoder work remain. An SSO-sized first read may add a backend read for a larger source; its
+cost is included rather than assumed free.
+
+The first generalized span fallback regressed a legacy 4 KiB materialization by about 24% on Linux.
+It is rejected. Its complete source/data populations and the separate three-variant refinement
+pilot remain in the raw archive. The selected direct owned path and first-block transfer remove
+that regression. Keep slower tiny/native controls, their absolute times and reference-repeat
+movement visible; do not pool populations or infer universal speedups from favorable large cells.
+
+Nine committed cases cover changing capability, partial/early EOF, range reservation lifetime,
+exact/zero-limit positive probes, malformed counts from both read APIs, late/probe errors, maximum
+arithmetic and owned-only small/multiple-chunk results. The archive format probe now also fills
+reserved storage directly; its expected owned-read counter changes from one to zero. Sparse holes,
+parent lifetimes, content and shared-budget tests remain. No new public flag, external dependency,
+worker, lock, metadata probe or host-I/O bypass is introduced. O19b seeking/index/replay/admission
+remains open.
diff --git a/extra_modules/archive/archive_source_buffer_test.cc b/extra_modules/archive/archive_source_buffer_test.cc
@@ -169,7 +169,7 @@ TEST_F(ArchiveSourceBufferTest, ReusesTheCallerOwnedBufferForFullAndShortBlocks)
   const auto source =
       std::make_shared<CountingSource>(MakeTar(std::string(3 * kReadBlock, 'x')), ReadOptions{}, counts);
   EXPECT_THAT(ListMembersOfSource(source), IsOkAndHolds(ElementsAre(Field(&Member::path, EqualsText("data.txt")))));
-  EXPECT_THAT(counts->owned, Eq(1));  // Only the independent four-byte format probe owns its result.
+  EXPECT_THAT(counts->owned, Eq(0));  // The independent format probe also fills its reserved storage directly.
   EXPECT_THAT(counts->caller_owned, Gt(3));
   EXPECT_THAT(counts->stable_storage, IsTrue());
 }

diff --git a/release-site.json b/release-site.json
@@ -79,7 +79,8 @@
     "docs/benchmarks/byte-shingle-representations-20261102.md": "docs/benchmarks/byte-shingle-representations-20261102.html",
     "docs/benchmarks/archive-input-buffers-20261102.md": "docs/benchmarks/archive-input-buffers-20261102.html",
     "docs/benchmarks/digest-and-record-storage-20261103.md": "docs/benchmarks/digest-and-record-storage-20261103.html",
-    "docs/benchmarks/caller-owned-streams-20261103.md": "docs/benchmarks/caller-owned-streams-20261103.html"
+    "docs/benchmarks/caller-owned-streams-20261103.md": "docs/benchmarks/caller-owned-streams-20261103.html",
+    "docs/benchmarks/direct-source-materialization-20261103.md": "docs/benchmarks/direct-source-materialization-20261103.html"
   },
   "generated_html": {
     "XFF.md": {

diff --git a/tools/BUILD.bazel b/tools/BUILD.bazel
@@ -41,6 +41,21 @@ cc_binary(
     ],
 )
 
+cc_binary(
+    name = "read_source_benchmark",
+    srcs = ["read_source_benchmark.cc"],
+    tags = [
+        "clang-tidy",
+        "manual",
+    ],
+    deps = [
+        "@abseil-cpp//absl/status:statusor",
+        "@abseil-cpp//absl/strings",
+        "@com_github_google_benchmark//:benchmark",
+        "@xff_extras_api//:read_source_cc",
+    ],
+)
+
 py_library(
     name = "release_backfill_lib",
     srcs = ["release_backfill.py"],

diff --git a/tools/read_source_benchmark.cc b/tools/read_source_benchmark.cc
@@ -0,0 +1,139 @@
+// SPDX-FileCopyrightText: Copyright (c) M. Boerger, the MBO Works authors
+// SPDX-License-Identifier: Apache-2.0
+
+#include <algorithm>
+#include <array>
+#include <cstddef>
+#include <cstdint>
+#include <memory>
+#include <span>
+#include <string>
+#include <string_view>
+#include <utility>
+
+#include "absl/status/statusor.h"
+#include "absl/strings/str_cat.h"
+#include "benchmark/benchmark.h"
+#include "xff/vfs/read_source.h"
+
+namespace xff::vfs {
+namespace {
+constexpr std::size_t kChunk = 64UZ * 1'024;
+
+class BenchmarkStream final : public ReadStream {
+ public:
+  BenchmarkStream(std::shared_ptr<const std::string> bytes, bool caller_owned)
+      : bytes_(std::move(bytes)), caller_owned_(caller_owned) {}
+
+  absl::StatusOr<std::string> Read(std::size_t max_bytes) override {
+    const auto count = std::min({max_bytes, kChunk, bytes_->size() - offset_});
+    auto result = bytes_->substr(offset_, count);
+    offset_ += count;
+    return result;
+  }
+
+  bool SupportsReadInto() const noexcept override { return caller_owned_; }
+
+  absl::StatusOr<std::size_t> ReadInto(std::span<char> buffer) override {
+    const auto count = std::min({buffer.size(), kChunk, bytes_->size() - offset_});
+    std::ranges::copy(std::string_view(*bytes_).substr(offset_, count), buffer.begin());
+    offset_ += count;
+    return count;
+  }
+
+ private:
+  std::shared_ptr<const std::string> bytes_;
+  bool caller_owned_;
+  std::size_t offset_ = 0;
+};
+
+class BenchmarkSource final : public ReadSource {
+ public:
+  BenchmarkSource(std::string bytes, bool caller_owned)
+      : bytes_(std::make_shared<const std::string>(std::move(bytes))), caller_owned_(caller_owned) {}
+
+  absl::StatusOr<std::unique_ptr<ReadStream>> Open() const override {
+    return std::make_unique<BenchmarkStream>(bytes_, caller_owned_);
+  }
+
+ private:
+  std::shared_ptr<const std::string> bytes_;
+  bool caller_owned_;
+};
+
+enum class SourceKind { kLegacy, kCallerOwned, kMemory };
+
+SharedReadSource MakeSource(SourceKind kind, std::string bytes) {
+  if (kind == SourceKind::kMemory) {
+    return MemoryReadSource(std::move(bytes));
+  }
+  return std::make_shared<BenchmarkSource>(std::move(bytes), kind == SourceKind::kCallerOwned);
+}
+
+void Measure(benchmark::State& state, SourceKind kind, bool range) {
+  const auto count = static_cast<std::size_t>(state.range(0));
+  std::string expected(count, 'x');
+  if (!expected.empty()) {
+    expected.front() = 'a';
+    expected.back() = 'z';
+  }
+  // Sequential custom sources replay this small prefix; memory sources seek directly.
+  const auto source = MakeSource(kind, range ? absl::StrCat("prefix:", expected, ":suffix") : expected);
+  // NOLINTNEXTLINE(clang-analyzer-deadcode.DeadStores): benchmark controls the measured iterations.
+  for (const auto unused : state) {
+    if (range) {
+      auto result = ReadSourceRange(*source, 7, count);
+      if (!result.ok() || result->bytes != expected) {
+        state.SkipWithError("range differs from fixture");
+        return;
+      }
+      benchmark::DoNotOptimize(result);
+    } else {
+      auto result = ReadSourceBytes(*source, count);
+      if (!result.ok() || *result != expected) {
+        state.SkipWithError("materialization differs from fixture");
+        return;
+      }
+      benchmark::DoNotOptimize(result);
+    }
+  }
+  state.SetBytesProcessed(state.iterations() * static_cast<std::int64_t>(count));
+}
+
+void RegisterAll() {
+  struct SourceCase {
+    std::string_view name;
+    SourceKind kind;
+  };
+
+  constexpr auto kSources = std::to_array<SourceCase>({
+      {.name = "legacy", .kind = SourceKind::kLegacy},
+      {.name = "caller-owned", .kind = SourceKind::kCallerOwned},
+      {.name = "memory", .kind = SourceKind::kMemory},
+  });
+  constexpr auto kCounts = std::to_array<std::int64_t>({0, 10, 4'096, 65'536, 1'048'576, 16'777'216});
+  for (const auto& source : kSources) {
+    for (const bool range : {false, true}) {
+      const std::string name = absl::StrCat("source/", source.name, "/", range ? "range" : "bytes");
+      // XFF_ABI_POINTER: Google Benchmark returns a borrowed mutable registration object.
+      auto* registration = benchmark::RegisterBenchmark(name, Measure, source.kind, range);
+      for (const auto count : kCounts) {
+        registration->Arg(count);
+      }
+      registration->Unit(benchmark::kMicrosecond)->UseRealTime()->MinTime(0.05)->MinWarmUpTime(0.05)->Repetitions(9);
+    }
+  }
+}
+}  // namespace
+}  // namespace xff::vfs
+
+// XFF_ABI_POINTER: Google Benchmark's process argument interface.
+int main(int argc, char** argv) {
+  benchmark::Initialize(&argc, argv);
+  xff::vfs::RegisterAll();
+  if (benchmark::ReportUnrecognizedArguments(argc, argv)) {
+    return 1;
+  }
+  benchmark::RunSpecifiedBenchmarks();
+  benchmark::Shutdown();
+}
diff --git a/xff_extras_api/BUILD.bazel b/xff_extras_api/BUILD.bazel
@@ -404,6 +404,7 @@ cc_test(
         ":vfs_cc",
         "@abseil-cpp//absl/status",
         "@googletest//:gtest_main",
+        "@mboworks_mbo//mbo/status:status_macros_cc",
         "@mboworks_mbo//mbo/testing:matchers_cc",
         "@mboworks_mbo//mbo/testing:status_cc",
     ],

diff --git a/xff_extras_api/read_source.cc b/xff_extras_api/read_source.cc
@@ -8,6 +8,7 @@
 #include <unistd.h>
 
 #include <algorithm>
+#include <array>
 #include <cerrno>
 #include <limits>
 #include <memory>
@@ -21,6 +22,35 @@
 
 namespace xff::vfs {
 namespace {
+// Keep capability changes and owned-only decoders valid for every individual read.
+absl::StatusOr<std::size_t> ReadStreamInto(ReadStream& stream, std::span<char> buffer, bool caller_owned) {
+  if (caller_owned) {
+    MBO_ASSIGN_OR_RETURN(const auto count, stream.ReadInto(buffer));
+    if (count > buffer.size()) {
+      return absl::DataLossError("read source returned more bytes than requested");
+    }
+    return count;
+  }
+  MBO_ASSIGN_OR_RETURN(const auto bytes, stream.Read(buffer.size()));
+  if (bytes.size() > buffer.size()) {
+    return absl::DataLossError("read source returned more bytes than requested");
+  }
+  std::ranges::copy(bytes, buffer.begin());
+  return bytes.size();
+}
+
+absl::StatusOr<std::size_t> ReadStreamFullyInto(ReadStream& stream, std::span<char> buffer) {
+  std::size_t filled = 0;
+  while (filled < buffer.size()) {
+    MBO_ASSIGN_OR_RETURN(const auto count, ReadStreamInto(stream, buffer.subspan(filled), stream.SupportsReadInto()));
+    if (count == 0) {
+      break;
+    }
+    filled += count;
+  }
+  return filled;
+}
+
 class MemoryStream final : public ReadStream {
  public:
   MemoryStream(std::shared_ptr<const std::string> bytes, std::size_t offset)
@@ -192,29 +222,65 @@ absl::StatusOr<ReadBlock> ReadSourceRange(const ReadSource& source, std::uint64_
     return result;
   }
   MBO_ASSIGN_OR_RETURN(auto stream, source.OpenAt(offset));
-  result.bytes.reserve(count);
-  while (result.bytes.size() < count) {
-    MBO_ASSIGN_OR_RETURN(const auto bytes, stream->Read(count - result.bytes.size()));
-    if (bytes.empty()) {
-      break;
-    }
-    result.bytes += bytes;
-  }
+  absl::StatusOr<std::size_t> transferred = 0;
+  // XFF_ABI_POINTER: std::string::resize_and_overwrite requires this writable buffer callback.
+  result.bytes.resize_and_overwrite(count, [&](char* bytes, std::size_t) noexcept -> std::size_t {
+    transferred = ReadStreamFullyInto(*stream, std::span<char>(bytes, count));
+    // Callback errors remain in transferred; no uninitialized bytes become string content.
+    return transferred.value_or(0);
+  });
+  MBO_RETURN_IF_ERROR(transferred.status());
   return result;
 }
 
 absl::StatusOr<std::string> ReadSourceBytes(const ReadSource& source, std::size_t max_bytes) {
   MBO_ASSIGN_OR_RETURN(auto stream, source.Open());
   std::string result;
+  const std::size_t limit = std::min(max_bytes, result.max_size());
   while (true) {
-    MBO_ASSIGN_OR_RETURN(const std::string block, stream->Read(64UZ * 1'024));
-    if (block.empty()) {
-      return result;
-    }
-    if (block.size() > max_bytes - result.size()) {
+    if (result.size() == limit) {
+      // A positive-sized read distinguishes exact EOF from overflow, including a zero limit.
+      std::array<char, 1> probe;
+      MBO_ASSIGN_OR_RETURN(const auto count, ReadStreamInto(*stream, probe, stream->SupportsReadInto()));
+      if (count == 0) {
+        return result;
+      }
       return absl::ResourceExhaustedError("container exceeds materialization limit");
     }
-    result += block;
+    const std::size_t previous = result.size();
+    if (!stream->SupportsReadInto()) {
+      const std::size_t request = std::min(64UZ * 1'024, limit - previous);
+      MBO_ASSIGN_OR_RETURN(auto bytes, stream->Read(request));
+      if (bytes.size() > request) {
+        return absl::DataLossError("read source returned more bytes than requested");
+      }
+      if (bytes.empty()) {
+        return result;
+      }
+      if (previous == 0) {
+        result = std::move(bytes);
+        continue;
+      }
+      result += bytes;
+      continue;
+    }
+    std::size_t request = std::min(64UZ * 1'024, limit - previous);
+    if (previous < result.capacity()) {
+      // Reuse spare storage after a short read, including SSO; probing EOF must not grow it.
+      request = std::min(request, result.capacity() - previous);
+    }
+    absl::StatusOr<std::size_t> transferred = 0;
+    // XFF_ABI_POINTER: std::string::resize_and_overwrite requires this writable buffer callback.
+    result.resize_and_overwrite(previous + request, [&](char* bytes, std::size_t) noexcept -> std::size_t {
+      transferred = ReadStreamInto(*stream, std::span<char>(bytes, previous + request).subspan(previous), true);
+      // Preserve errors separately because this standard callback returns only the initialized size.
+      return previous + transferred.value_or(0);
+    });
+    MBO_ASSIGN_OR_RETURN(const auto count, std::move(transferred));
+    if (count == 0) {
+      return result;
+    }
   }
 }
+
 }  // namespace xff::vfs