Skip to content

Optimizer: auto-infer N_g from Prometheus #525

Open
1 of 1 issue completed
Open

Description

@milindsrivastava1997

Part of #523 (Track 2: non-empirical gap 3 + empirical gap 4).

Goal

Replace the optimizer cost model's hardcoded N_g = 1 with label-group cardinality inferred from live Prometheus data.

Scope

  • Add label_group_count to CandidateConfig, not AQE, so the profiled count belongs to the candidate configuration and remains compatible with future Phase 3 sharing.
  • Use N_g = 1 for subpopulation-aware sketches and the profiled distinct grouping-key count for non-subpopulation-aware sketches.
  • Apply the count to ingest memory and query CPU/memory costs; ingest CPU remains based on total arrival rate.
  • Profile once per profile key (metric, spatial_filter, grouping_labels) before candidate enumeration, and reuse the result across the AQE candidate grid.
  • Use the existing Prometheus /api/v1/series path and shared retry/fetch logic. Construct a selector from the metric and spatial filter, then count distinct grouping-label tuples client-side.
  • Cache series data so multiple AQEs with the same metric/filter do not repeat the expensive fetch.
  • An empty grouping-label set has N_g = 1. Empty or unavailable Prometheus data must never become N_g = 0.
  • If cardinality is unavailable, retain EXACT and subpopulation-aware candidates while dropping non-subpopulation-aware candidates, consistent with the missing-data policy in Optimizer: param-aware atomic costs from sketch-bench #524.

Follow-up

The offline, dataset-backed implementation is tracked separately in #693. Both implementations should feed the same optimizer profile interface; #693 should not require Prometheus.

Acceptance criteria

  • Prometheus-derived cardinality changes non-subpopulation-aware candidate costs proportionally.
  • Subpopulation-aware and EXACT candidates are unaffected.
  • Prometheus data takes precedence over any fallback metadata.
  • Selector construction, retries, caching, empty results, and distinct grouping-key counting are covered by tests.

Activity

  1. milindsrivastava1997 commented on Jul 13, 2026

    @milindsrivastava1997
    ContributorAuthor

    Implementation plan discussed:

    • No new PromQL query type needed: fetch_labels_for_metric already calls /api/v1/series with match[]=<metric> and gets back every matching series' full label set. Add profile_label_group_count(url, metric, spatial_filter, grouping_labels): same /api/v1/series call (match[] = metric + spatial_filter_normalized), group the returned series client-side by their grouping_labels values, count distinct groups. Refactor the shared HTTP-fetch-with-retry loop out so this doesn't duplicate the 503-retry logic. Empty grouping_labels → trivially 1 (matches today's hardcoded value, no regression for ungrouped queries).
    • Profiled once per AQE (batched in pipeline.rs's run_pipeline shell right after extract_aqes), threaded through as Vec<(AQE, u64)> (paired, not a separately-indexed parallel Vec) rather than re-profiled per candidate.
    • Total miss (no live Prometheus data AND no metrics: hint): drop candidates using non-subpopulation-aware sketches for that AQE, consistent with Optimizer: param-aware atomic costs from sketch-bench #524's "missing data → drop the candidate" policy — defaulting to 1.0 would silently understate a capacity-planning number. Subpopulation-aware candidates (CMS, MultipleSum, etc.) and EXACT are unaffected.
  2. changed the title [-]Optimizer: real N_g via Prometheus profiling, hints fallback[/-] [+]Optimizer: auto-infer N_g from Prometheus[/+] on Sep 3, 2026
  3. milindsrivastava1997 commented on Oct 26, 2026

    @milindsrivastava1997
    ContributorAuthor

    Re-scope note: with the MILP (#794), N_g is groups[].cardinality in the --workload-facts file. Inferring that file from Prometheus also needs each sketch-served grouping's data shape (zipf_s, distinct_keys, tail_index); for that we can use sketch-bench's data-shape fitter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions