Repository navigation
feat(cpp): add a RandomAccessWriteFile abstraction and in-memory write sink - #1094
Merged
ColinLeeo merged 1 commit intoNov 12, 2026
Merged
Conversation
…e sink Introduce RandomAccessWriteFile, the write-side counterpart of the existing RandomAccessReadFile byte source, and make the fd-backed WriteFile implement it. TsFileIOWriter, TsFileWriter, TsFileTableWriter, TsFileTreeWriter and RestorableTsFileIOWriter now reach their sink through the interface; path-based constructors, the C API and the Python binding are unchanged, and the local write path keeps the same syscall loop. Add MemoryRandomAccessWriteFile and MemoryRandomAccessReadFile so a complete TsFile can be written to memory and read straight back, which proves the write path is backend-independent. Recovery stays local-file only: self_check reads the existing bytes back by path. Replace the get_fd()-based liveness checks with is_opened() (fd >= 0), keeping file_opened() as an alias. Add unit and end-to-end tests (table, tree and aligned round trips, byte-for-byte parity between the memory sink and a local file) and a write_backend_benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
RandomAccessWriteFile, the write-side counterpart of the existingRandomAccessReadFilebyte source, and wires the C++ writers through it. Thefirst non-local backend is an in-memory sink that lets a complete TsFile be
written and read back without touching the file system.
Part of #1055 (C++ object storage read/write support). This PR is the
backend-independent write path; an object-storage sink is a follow-up.
Design
cpp/src/file/random_access_write_file.h— the interface:is_opened/file_path/write/get_position/sync/truncate/seek_to_end/close,in the project's error-code style. The header documents the contracts,
notably write-after-close (
E_FILE_WRITE_ERR, no-op),truncateextendingwith zeros and not moving the cursor (POSIX
ftruncatesemantics), and thatfile_path()may be a label for backends without a real path.WriteFile(the existing fd-backed local backend) now implements theinterface directly — no separate adapter class. This keeps the C API
(
typedef void* WriteFile), the Python binding and all path-basedconstructors working unchanged; the local write path keeps the same
syscall loop.
file_opened()stays as an alias ofis_opened(), andis_opened()isfd_ >= 0(the oldfd_ > 0check mis-treated fd 0 asclosed). The four
ASSERT(fd_ > 0)guards became error returns.MemoryRandomAccessWriteFile— growable buffer; constructing it opens it;close()keeps the bytes;data()/size()/release_data()expose them. Itcannot be reopened for append.
MemoryRandomAccessReadFile— aRandomAccessReadFileover those bytes soTsFileReader::open(std::unique_ptr<RandomAccessReadFile>)reads thein-memory file. The read side previously had no in-memory implementation;
it is required to verify the write path end to end.
TsFileIOWriter::init,TsFileWriter::init, theTsFileTableWriterandTsFileTreeWriterconstructors, andRestorableTsFileIOWriter::get_write_file()now take/returnRandomAccessWriteFile*. The threeget_fd()liveness checks inTsFileIOWriterbecameis_opened().Scope
RestorableTsFileIOWriter) remains local-file only:self_checkreads the existing bytes back by path through a separatehandle. Documented in both headers; a non-local sink cannot be recovered
in place.
Tests
New (all pass):
cpp/test/file/memory_random_access_write_file_test.cc— sequentialwrites and position tracking, truncate shrink/grow/zero-fill and cursor
semantics,
seek_to_end, write-after-close, invalid arguments,release_data, and the localWriteFiledriven through the interface.cpp/test/file/write_sink_e2e_test.cc— table-model, non-aligned tree andaligned tree round trips written to the memory sink and read back through
MemoryRandomAccessReadFile; byte-for-byte equality between the memorysink and a local file for the same workload; the restorable/recovery path
still works on a local file.
Verified on macOS arm64, Release:
TsFile_Test: 1467 passed, 0 failed, 3 pre-existing skips.DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib(macOS ASAN is unusableon the dev box).
clang-format17.0.6 (the version pinned inspotless) applied to everytouched file.
Benchmark
cpp/bench_mark/bench_mark_src/write_backend_benchmark.cc, built with-DBUILD_BENCHMARK=ON -DBUILD_TEST=OFF:Workload: table model, one table with 1 tag + 4 fields
(INT64/INT64/DOUBLE/INT32), 1M rows in 20 tablets of 50k rows, deterministic
LCG values (poorly compressible) and 1..100 ms timestamp gaps. Default
encodings/compression from
libtsfile_init(). macOS arm64, Release, pagecache warm.
WriteFileMemoryRandomAccessWriteFileThe memory sink is ~2.1× faster here because it replaces the write syscalls;
the encoded output is byte-for-byte identical to the local file (asserted by
MemorySinkMatchesLocalFileByteForByte).Local path: before/after. The same workload was compiled against
develop(same data generation, localWriteFileonly, same flags) and runalternately with this branch's local arm:
developbaselineThe difference is within the run-to-run spread of individual runs
(0.152–0.201 s). The only change on the local path is one virtual dispatch
per
TsFileIOWriter::flush_stream_to_filebuffer write; an isolated64 KiB-write microbenchmark on the in-memory sink (concrete call vs. the
interface pointer) shows a difference smaller than its own run-to-run noise
(~2–3 µs per call, ~3 GiB/s). The microbench is included in the benchmark
binary.
Follow-ups (not in this PR)
client belongs in the C++ core — this interface is the hook.
write_file_newstill creates local files only; expose a sinkhandle (including the memory sink) for embedders.
TsFileWriteraccepts a path only; a writable binary file-objectbridge mirroring
python_random_access_read_file.ccis the natural nextstep.
self_checkto read from thesink's matching read source (or a local spool).