Skip to content

[Perf][C++] Load a device's timeseries indexes with one pread instead of per-measurement reads #1074

Closed

Description

@ColinLeeo

Problem

TsFileIOReader::get_timeseries_indexes handles each requested measurement
individually: binary search in the device index node, a separate
read_file_->read (PREAD backend, so one syscall each), deserialization into a
temporary arena, then a clone into the caller's arena. For a 4500-column table
that is ~4500 preads plus ~4500 deserialize+clone.

Profiling a wide-table read showed do_load_timeseries_index at ~7.6%.

Proposed changes

Collect every measurement leaf once (get_all_leaf), read the whole contiguous
timeseries-index range with a single pread, and deserialize each index directly
into the caller's arena; build a name -> index map and return the requested
ones.

Implemented in #1072; index loading dropped from ~7.8% to ~0.2% of the query.

Related: #1072

Activity

  1. added this to the 2.5.1 milestone on Nov 5, 2026
  2. ColinLeeo commented on Nov 16, 2026

    @ColinLeeo
    ContributorAuthor

    Resolved by #1095. For projections of 16 or more measurements, TsFileIOReader::get_timeseries_indexes() now walks the measurement index once (get_all_leaf()), loads the device's whole TimeseriesIndex range and maps the results by measurement name; do_load_all_timeseries_index() reads the contiguous [first start, last end) range with a single pread when the leaf entries tile it. Narrow projections keep the per-measurement path, where a full walk would deserialize every column.

    Closing as completed in develop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions