Skip to content

[Bug][C++] Aligned table query hangs with a small internal TsBlock capacity #1035

Closed

Description

@ColinLeeo

Problem

An aligned C++ table query can spin indefinitely inside the first ResultSet::next() when the internal TsBlock cannot hold the decoder's fixed batch of 129 rows. The query itself returns E_OK; reading its first result never returns.

A reproducer using g_config_value_.tsblock_max_memory_ = 128 hangs with no time or field predicates. The same file/query completes and returns all 63 rows with the default 2 MiB setting (2097152). This is the internal TsBlock memory capacity, not the result API's requested output batch size.

Reproduction conditions

  • Aligned table with TAG id and INT64 fields v, w, and ghost; PLAIN encoding and UNCOMPRESSED compression.
  • ghost is declared in the schema but is never written. v is absent for one device, and the written value columns also contain nulls.
  • Three devices, each with 21 timestamps written across three flushes; the page writer limit is 5 points.
  • Query {"v", "ghost", "w", "v"} with no predicates, offset 0, no limit, and serial reading. Missing requested measurements force the per-column aligned read path.
  • Set the internal TsBlock memory budget to 128 bytes before creating the reader, then call ResultSet::next().

Expected: return all 63 rows using appropriately sized internal batches. If a capacity is unsupported, return an explicit error rather than retrying without progress.

Actual: the first next() does not return; a bounded reproduction was terminated after 5 seconds.

Standalone C++ reproducer

Save this as small_tsblock_repro.cc and compile it against the built C++ library, for example from the repository root:

c++ -std=c++11 -I cpp/src small_tsblock_repro.cc \
  -L /path/to/build/lib -ltsfile -Wl,-rpath,/path/to/build/lib \
  -o small_tsblock_repro
#include <cstdio>
#include <cstdlib>
#include <iostream>
#include <string>
#include <vector>
#include "common/config/config.h"
#include "common/tablet.h"
#include "file/local_random_access_read_file.h"
#include "file/write_file.h"
#include "reader/table_query_executor.h"
#include "writer/tsfile_table_writer.h"
using namespace common;
using namespace storage;
void check(int ret) { if (ret != E_OK) { std::cerr << "error=" << ret << '\n'; std::exit(2); } }
int main(int argc, char** argv) {
  libtsfile_init();
  const auto saved = g_config_value_;
  g_config_value_.page_writer_max_point_num_ = 5;
  g_config_value_.tsblock_max_memory_ = argc > 2 ? std::atoi(argv[2]) : 128;
  g_config_value_.parallel_read_enabled_ = false;
  const std::string path = argv[1];
  {
    WriteFile file;
    check(file.create(path, O_WRONLY | O_CREAT | O_TRUNC, 0666));
    TableSchema schema("t", {
      {"id", STRING, UNCOMPRESSED, PLAIN, ColumnCategory::TAG},
      {"v", INT64, UNCOMPRESSED, PLAIN, ColumnCategory::FIELD},
      {"w", INT64, UNCOMPRESSED, PLAIN, ColumnCategory::FIELD},
      {"ghost", INT64, UNCOMPRESSED, PLAIN, ColumnCategory::FIELD}});
    TsFileTableWriter writer(&file, &schema);
    for (int chunk = 0; chunk < 3; ++chunk) for (int d = 0; d < 3; ++d) {
      std::vector<std::string> names{"id", "w"};
      std::vector<TSDataType> types{STRING, INT64};
      std::vector<ColumnCategory> categories{ColumnCategory::TAG, ColumnCategory::FIELD};
      if (d != 1) { names.push_back("v"); types.push_back(INT64); categories.push_back(ColumnCategory::FIELD); }
      Tablet tablet("t", names, types, categories, 7);
      for (int r = 0; r < 7; ++r) {
        int t = chunk * 7 + r;
        check(tablet.add_timestamp(r, t));
        const std::string id = "d" + std::to_string(d);
        check(tablet.add_value(r, "id", id.c_str()));
        if (t % 3 != 0) check(tablet.add_value(r, "w", int64_t(d * 1000 + t)));
        if (d != 1 && t % 4 != 0) check(tablet.add_value(r, "v", int64_t(d * 100 + t)));
      }
      check(writer.write_table(tablet));
      check(writer.flush());
    }
    check(writer.close());
  }
  std::cerr << "before query memory=" << g_config_value_.tsblock_max_memory_ << '\n';
  LocalRandomAccessReadFile file;
  check(file.open(path));
  TableQueryExecutor executor(&file, -1);
  ResultSet* result = nullptr;
  check(executor.query("t", {"v", "ghost", "w", "v"}, nullptr, nullptr, nullptr, 0, -1, result));
  std::cerr << "after query before first next\n";
  bool next = false;
  int ret = E_OK, count = 0;
  while ((ret = result->next(next)) == E_OK && next) {
    ++count;
    if (count > 200) return 3;
  }
  std::cerr << "after scan ret=" << ret << " count=" << count << " expected=63\n";
  executor.destroy_query_data_set(result);
  std::remove(path.c_str());
  g_config_value_ = saved;
  libtsfile_destroy();
  return ret == E_OK && count == 63 ? 0 : 4;
}

Run the two cases with an external 5-second timeout (e.g. timeout on Linux):

timeout 5s ./small_tsblock_repro /tmp/small-tsblock-128.tsfile 128
./small_tsblock_repro /tmp/small-tsblock-default.tsfile 2097152

The small-capacity case stops after after query before first next. The default-capacity case prints after scan ret=0 count=63 expected=63 and exits with code 0. Terminate the small-capacity process externally; an iteration counter around next() cannot interrupt this hang.

Cause

AlignedChunkReader::decode_tv_batch checks row_appender.remaining() < BATCH with BATCH = 129 and returns E_OVERFLOW before consuming any input. For the time-plus-INT64 internal block in this reproducer, 128 bytes only allow 8 rows; the condition remains true even on an empty block and even though a page has fewer than 129 points.

decode_time_value_buf_into_tsblock preserves the unfinished page and converts that overflow to E_OK. The series iterator receives an empty successful block, sees that the chunk still has data, and retries. Since neither the input cursor nor available block capacity changes, it repeats without progress.

The multi-column multi_DECODE_TV_BATCH path already caps its effective batch with min(BATCH, row_appender.remaining()). The single-column typed path still has the unconditional 129-row requirement.

Verification and scope

  • Reproduced on the base of [C++] Apply field predicates to aligned table query rows #1026, 1180b13675e11a0230708af2cc70d037e0e12623: timeout on the first next().
  • Reproduced again on standalone PR head c01ee105d397b1ed16a4daf335db9d8176036bac with a macOS arm64 Debug library: 128 bytes times out after 5 seconds; 2097152 bytes exits successfully with all 63 rows. No predicates are supplied.
  • Source inspection of current develop, eb733133b9eee0b1eb74f7671d809ecd730cedc3, confirms the same fixed-capacity guard remains. The runtime runs above used the PR head/base, not a separately rebuilt develop library.
  • This is a pre-existing reader issue found during review of [C++] Apply field predicates to aligned table query rows #1026. Track its fix separately; no fix is included in that PR. A released-version range has not been established.

Suggested regression coverage should run the sparse/missing-column query with internal capacities below, at, and above 129 rows, check forward progress with a bounded timeout, and compare complete results with the default-capacity query.

Activity

  1. added a commit that references this issue on Oct 29, 2026
    f2a71ef
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bot:triagedmanaged by TsFile maintainer bot

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions