Repository navigation
Description
Summary
Follow-up to #967 and the review discussion.
The bounded device-node LRU limits the number of retained devices, but repeated metadata lookups for a hot device still allocate into that cached node's PageArena. Its memory can therefore keep growing even when the cache contains only one entry and every query/scan iterator is released.
This allocation behavior predates #967; it is a remaining issue, rather than a regression introduced by the bounded LRU.
Reproduction and observed behavior
- Create a TsFile containing one non-aligned device,
root.followup.d0, with oneINT32measurement,value, and one row. - Keep one
TsFileIOReaderopen. Warm the device cache once. - Repeat
alloc_ssi()/revert_ssi()for that same series 10,000 times, resetting the caller's arena after each iteration.
Core test logic, after opening reader on the file:
auto device = std::make_shared<StringArrayDeviceID>("root.followup.d0");
PageArena query_pa;
query_pa.init(512, MOD_TSFILE_READER);
auto load = [&]() {
TsFileSeriesScanIterator* ssi = nullptr;
int ret = reader.alloc_ssi(device, "value", ssi, query_pa);
if (ssi) reader.revert_ssi(ssi);
query_pa.reset();
return ret;
};
ASSERT_EQ(load(), E_OK);
const auto before = reader.TEST_reader_memory_bytes();
for (size_t i = 0; i < 10000; ++i) ASSERT_EQ(load(), E_OK);
const auto after = reader.TEST_reader_memory_bytes();
EXPECT_EQ(reader.TEST_device_node_cache_size(), 1u); // passes
EXPECT_EQ(after, before); // failsWith a Debug build and ENABLE_TEST=ON, ENABLE_MEM_STAT=ON, the MOD_TSFILE_READER accounting reported:
| After warm-up | After 10,000 additional lookups | |
|---|---|---|
| Cache entries | 1 | 1 |
| Accounted bytes | 1,072 | 84,688 |
These are allocator accounting values, not process RSS. The reproduction was run on review revision 6a63ff35d9e94b889998dd69408ceecc1dad598d; the relevant reader and metadata allocation paths remain unchanged in the merged PR head, 1dfd2dc4d5daf91b9beca2da265a094126494633.
No row iteration is needed to trigger this: it happens during scan-iterator creation / measurement metadata lookup. Query teardown does not reclaim allocations retained by the cached arena.
Cause
A cache hit reuses the device's top measurement index node. load_timeseries_index_for_ssi() still resolves the measurement and reads/parses its timeseries metadata.
MetaIndexNode::binary_search_children() returns children_[l]->clone(pa_). For a MeasurementMetaIndexEntry, cloning duplicates the measurement name into the source node's arena. The binary search itself is not the retained allocation; the result copy is.
Those bytes are reclaimed when the owning arena is destroyed, rather than when the temporary result is released. A hot cache entry can stay resident indefinitely, so bounding entry count does not bound this growth.
Suggested fix and validation
- Give lookup-result copies an explicit query/result-owned destination arena, or use another representation with a safe ownership lifetime. Keep a published cached node's arena unchanged during lookups.
- Audit multi-level index traversal when changing this ownership. Some nodes use temporary arenas; simply returning a child pointer or removing cloning globally could create dangling references.
- Add a repeated-hot-series regression test: retained metadata memory should plateau after warm-up while cache size stays constant. Also cover multi-level indexes and active iterators surviving cache eviction. Guard accounting assertions with
ENABLE_MEM_STAT, while keeping functional checks enabled without it.
Optional subsequent performance improvement
Separately, consider lazily caching resolved, parsed (device, measurement) metadata with an owning handle, building on PreparedSeries / init_prepared(). A hit could avoid repeated measurement-index traversal and metadata I/O/deserialization. Filters, scan cursors and decoding buffers would remain query-local.
Such a cache should be scoped to the reader/file lifetime, have a memory budget, and share aligned time metadata where appropriate. The miss path would still need the allocation-lifetime fix above. Benchmark repeated and interleaved queries using metadata lookup/read/parse counts, query setup time and retained memory to quantify the benefit.
Activity