Skip to content

benchmark: add metrics evaluator benchmark#831

Open
timothyw553 wants to merge 1 commit into
apache:mainfrom
timothyw553:tw/metrics-evaluator-benchmark
Open

benchmark: add metrics evaluator benchmark#831
timothyw553 wants to merge 1 commit into
apache:mainfrom
timothyw553:tw/metrics-evaluator-benchmark

Conversation

@timothyw553

Copy link
Copy Markdown
Contributor

Summary

This adds a focused benchmark for metrics evaluator performance now that the shared benchmark build scaffolding has landed. The benchmark covers evaluator construction separately from data-file evaluation so reviewers can see setup cost and per-file filtering cost independently.

  • Covers both InclusiveMetricsEvaluator and StrictMetricsEvaluator.
  • Exercises equality, range, starts-with, small IN, medium IN, and null-or-NaN predicates.
  • Scales evaluation across 1, 100, and 10,000 synthetic DataFile objects.
  • Registers the benchmark in both CMake and Meson benchmark builds.

Part of #690.

Test Plan

  • git diff --check origin/main..HEAD
  • Configured CMake benchmark build with -DICEBERG_BUILD_BENCHMARKS=ON
  • cmake --build build-bench-ci --target metrics_evaluator_benchmark
  • build-bench-ci/src/iceberg/benchmark/metrics_evaluator_benchmark --benchmark_filter=BM_InclusiveMetricsEvaluatorMake/equal_id --benchmark_min_time=0.01s
  • cmake --build build
  • ctest --test-dir build --output-on-failure
  • Meson benchmark configure was not run locally because meson is not installed on this machine.

Verification Commands

cmake -S . -B build-bench-ci -G Ninja \
  -DICEBERG_BUILD_BENCHMARKS=ON
cmake --build build-bench-ci --target metrics_evaluator_benchmark
build-bench-ci/src/iceberg/benchmark/metrics_evaluator_benchmark \
  --benchmark_filter=BM_InclusiveMetricsEvaluatorMake/equal_id \
  --benchmark_min_time=0.01s
cmake --build build
ctest --test-dir build --output-on-failure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant