Skip to content

refactor: tools and snapshots over the #1207 memo, edges and contexts (#1207 PR 9) - #1239

Draft
zzylol wants to merge 11 commits into
refactor/1207-accuracyfrom
refactor/1207-tools
Draft

zzylol wants to merge 11 commits into
refactor/1207-accuracyfrom
refactor/1207-tools

Conversation

@zzylol

@zzylol zzylol commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Last PR of the #1207 stack (on #1238, PR 8). Draft: the user reviews and merges the #1207 unit by hand; not for merge on its own.

Why

PRs 0–8 replaced the planner's interfaces (memo groups as expressions, candidates $(e_v, p^{\log}v)$ with $r^{\log}{c,i}$, physical candidates and edges with $\alpha$, contexts, constraint (K)). The tools still stopped at todo!("#1207 PR 9 …"): asap_planner --format text and --format json (the viewer document) could not print a memo, Stage 2's candidates or Stage 3's selection; planner_fuzz fed witnesses holding ASAP operators into Pass 1, which refuses them; the snapshots were still the plans of main before #1207. #1207's Order row 9 and #1205 ask for the tools to print every notation (memo groups with properties, every edge with $\alpha$ and its rejection reason, every stage's pruned candidates) and for every plan and cost change to be listed.

What

  • asap_planner --format text (planning_stages_text): each memo candidate with its operator over its input groups, θ, ω, fused groups and per-input parameter floors, marked new at the step that added it; Stage 2's physical candidates, each with every producer of each input, ✓ or ✗ with why ($\alpha^{\log}=0$: floor or metric rule; $\alpha^{\mathrm{phys}}=0$: timing/lifetime); Stage 2's and Stage 3's pruned candidates; Stage 3's selected DAG over physical candidates, its cost per realized node and each query's accepted accuracy.
  • Viewer document asap-stage-pipeline/v2 (producer asap_planner.rs and consumer tools/dag-viewer changed together, DAG viewer / stage_pipeline: print memo candidates with properties, every producer–consumer edge with its feasibility and rejection reason, and every stage’s pruned candidates #1205): memo candidates with logical_properties and input_requirements; stage2_physical_asap_optimization.physical_candidates with properties and per input every edge (satisfies_logical_requirement, satisfies_physical_requirement, rejection, selected); Stage 2 pruned_candidates with properties; Stage 3 pruned_candidates with dominated_by; memo.selected_candidates. The viewer shows them (group panel, candidate panel with ✓/✗ edges, pruned lists per stage). v1 documents are refused; every old-format reader is deleted.
  • planner_fuzz: witnesses are costed with asap_cost::plan_cost instead of being planned again.
  • Snapshots re-baselined (list below); a sketch workload whose selection fails now records its error (as a corpus query does) so re-baselining runs to the end.
  • Docs: planner-layering.md memo components, the worked example (numbers from asap_planner --format text), and the stage tables over today's memo; terminology sweep (consumer, materialized, cost) across crates, docs and the viewer.

How

  • Printers read the memo and Stage 2's output directly: a group's label is its expression's operator (operator_label, generic over Operator<C>); a candidate is one operator, so no node table is built (memo.nodes, MemoNodes, group_of_node are deleted). Edges come from PhysicalASAPMemoCandidates::edges() (every edge, admissible or not); the reason is derived by edge_rejection from the edge's two booleans and the two candidates' properties, mirroring logical_edges_with_capped_floors (floor first, else the metric rule).
  • Fuzz witness costing: Planner::plan records each realized node's assignment (timing, lifetime, θ, coverage) read back from the realized DAG (flatten order = export order) keyed by node address (the cached plans hold the nodes, so no address is reused). Planner::cost exports the witness roots with physical_asap_dag_with_assignments (planned nodes keep their assignment; derived nodes at query time, not materialized; for a witness, a not-materialized query-time node the derived root shares with another root is costed as OneEvaluation(Memory)), then plan_cost with the same arguments as select_plan. W's reference cost equals the search's (the_selected_plan_costs_what_the_search_selected_it_for).

Representation, rules, enumeration

Notation → code (what the tools print)

Notation Text (--format text) Document (--format json)
$G$, $e_v$ G<n>: <operator> <- G…, [i] <origin> <operator> <- G… memo.steps[].groups[]{operator, children}, candidates[]{operator, children}
$p^{\log}_v = (\boldsymbol\theta, \omega)$ θ=Kll(k=269) ω=tumbling 60000ms logical_properties{parameters, window_layout}
$r^{\log}_{c,i}$ floors=[…], requires θ ⪰ … input_requirements[], inputs[].requirement
$p^{\mathrm{phys}}_v = (\tau, \ell)$ C<n> = G<g>[<i>] (timing, lifetime) physical_candidates[]{group, index_in_group, properties}
$P_{c,i}$, $\alpha^{\log}$, $\alpha^{\mathrm{phys}}$ one line per producer, ✓ / ✗ α^… = 0: … inputs[].edges[]{producer, satisfies_logical_requirement, satisfies_physical_requirement, rejection}
$x_v$, $y_{u,c,i}$ node<n>: C<k> … <- node… stage3_plan_selection.selected_dag, edges[].selected, memo.selected_candidates
pruned per stage pruned by Stage 2 (n), pruned by Stage 3 (n) stage2…pruned_candidates[], stage3…pruned_candidates[]{dominated_by}

Concepts not in the formulation: origin (approved 2026-11-05); the rejection reason text.

Key code interfaces added / modified

crates/devtools/src/candidate_labels.rs

/// An operator's kind with its family, summary query, time range and window layout, without inputs.
pub fn operator_label<C>(operator: &Operator<C>) -> String; // was node_label(&FlatNode)
/// θ as text, e.g. `Kll(k=269)`.
pub fn parameters_label(parameters: &SketchParams) -> String; // new

crates/devtools/src/planning_stages_text.rs

/// Unchanged entry: the text of every stage the run ran.
pub fn planning_stages_text(stages: &PlanningStagesRun) -> String; // unchanged (bodies filled)
/// Why an edge is not admissible (α^log = 0: floor or metric rule; α^phys = 0), None when admissible.
pub fn edge_rejection(physical: &PhysicalASAPMemoCandidates, edge: &PhysicalEdge) -> Option<String>; // new
/// Whether two candidates of one group are the same (e_v, p^log_v): marks `new` / `added`.
pub fn same_candidate(left: &CandidateForMemoGroup, right: &CandidateForMemoGroup) -> bool; // new

crates/devtools/src/planner_fuzz/mod.rs

/// A plan costed outside the search.
pub struct Costed { pub selected: String, pub cost_per_second: f64, pub derived_root: PrivateNodes } // was Priced
impl Planner {
    /// `roots` costed by asap_cost::plan_cost with the plannings' node assignments; with
    /// `last_root_is_derived` (a witness), nodes shared with q' are costed OneEvaluation(Memory).
    pub fn cost(&self, workload: &FuzzWorkload, roots: &[QueryRoot], last_root_is_derived: bool)
        -> Result<Costed, String>; // was price(workload, roots, last_root_is_key_filter) via run_search_over
    /// `cost` of W's selected plan, cached per workload.
    pub fn reference_cost(&mut self, workload: &FuzzWorkload, roots: &[QueryRoot])
        -> Result<Costed, String>; // was reference_price
}

Viewer document: format asap-stage-pipeline/v2 (was v1), fields as in What; removed: memo.nodes, groups' and candidates' node, group_of_node, implements, accuracy_budget_parts, selected_choice, stage1…pruned_candidates, stage2…pruned.

Other: asap_cost::UnitPrice → UnitCost (pub in measured.rs, not re-exported); RejectionCause::failed_a_check removed (dead once the viewer's cause lists went).

Usage (the running example, checked at the head commit):

$ asap_planner --workload p99.json --run-until stage3      # format text by default
# p99.json: quantile_over_time(0.99, data[10m]) every 60 s at phase 0, rank 0.01, δ 0.01 per value;
#           1,000 series every 1 s, values_non_negative, ingesting; raw data retained (default)
G3:  SummaryAgg Kll, PerSubpopulationInstance       <- G1
    [3]   Pass1    SummaryAgg Kll, PerSubpopulationInstance <- G1  θ=Kll(k=269)
    [10]  TumblingWindows    SummaryAgg Kll, … <- G1  θ=Kll(k=269) ω=tumbling 60000ms
G4:  SummaryEstimate Quantile(q=0.99)               <- G3
    [0]  Pass1    SummaryEstimate Quantile(q=0.99) <- G3  exact  floors=[Kll(k=269)]      # Pass 2 output
    C32 = G4[0] (QueryTime, None)  SummaryEstimate Quantile(q=0.99)                        # Stage 2 output
        input 0 <- G3, requires θ ⪰ Kll(k=269):
            C12 = G3[3] (QueryTime, None)                            ✓
            C22 = G3[10] (IngestionTime, AcrossEvaluations(Memory))  ✓
    C6 = G2[0] (QueryTime, None)  Aggregate quantile(q=0.99)
        input 0 <- G1:
            C5 = G1[0] (IngestionTime, None)   ✗ α^phys = 0: consumer (QueryTime, None) cannot read producer (IngestionTime, None)
pruned by Stage 2 (104):
    G0[0]   (QueryTime, AcrossEvaluations(Memory))      AcrossEvaluationsWithoutWindowLayout
search: Optimal, 332 MILP variables, 420 constraints, 56 ms solving                          # Stage 3 output
node3:  C15 = G3[4] (QueryTime, OneEvaluation(Memory))  SummaryAgg Kll, PerSubpopulationInstance  <- node1
node4:  C32 = G4[0] (QueryTime, None)                   SummaryEstimate Quantile(q=0.99)          <- node3
cost_per_second: 0.062252
    q1: metric rank; accepted rank (Kll)
pruned by Stage 3 (0):

KLL k < 269 builds have no edge into G4 (Stage 2 pruned them: ReachesNoQueryOutput); k 547 is selected over 269 because sizes above the floor tie in the objective (#1238 spec question 5). --format json gives the same data to the viewer.

Major algorithm changes

Fuzz witness costing. Problem: a witness (W's selected plan plus q' derived over it) holds ASAP operators; after PR 3 Pass 1 refuses them, so 13 witness checks were inconclusive. Before: price ran Stages 0–3 over the witness roots (run_search_over). After: cost exports the roots with each planned node's realized assignment and runs plan_cost (steps in How). Correctness: W's own plan costs exactly what the search selected it for (test above); a witness differs from W only by derived nodes and the shared nodes it reads. Limits: the witness's accuracy is not checked (q' is derived at q's target or a looser one; another statistic from q's θ, unions and key filters are not re-checked); a LongerWindow witness's rewritten chain nodes are new nodes with no assignment (a sketch chain then has no θ and the check is inconclusive). Complexity: one plan_cost per witness instead of a full planning run.

Before this PR

asap_planner panicked at the todo!()s for any run past Stage 0 (text) or with candidates (JSON); the viewer read v1 fields no producer writes; fuzz witnesses were inconclusive; snapshots were main's plans before #1207.

After this PR

Every stage's output is printed and documented; the viewer shows the memo, the physical edges with $\alpha$ and the reasons, and the pruned candidates of Stage 2 and Stage 3. No todo!("#1207 PR 9 …") remains.

Snapshot plan and cost changes

Re-baseline (mechanical, redo after the pending lower-PR decisions): UPDATE_SNAPSHOTS=1 cargo test --release -p asap-devtools --test corpus_snapshots -- --ignored, then diff each changed snapshot's selected.cost.cost_per_second, node count and accepted[].sketches (or error) against the base. "Before" is the snapshot at the base (main's plan before #1207: PRs 0–8 did not re-baseline). Cost model 1 as each corpus/manifest declares.

Snapshot Before After
o11y_bench/q02 0.481567/s, 7 nodes, sketches none 2.87213/s, 6 nodes, sketches none
o11y_bench/q05 0.95544/s, 16 nodes, sketches none 5.25967/s, 13 nodes, sketches none
o11y_bench/q06 0.484617/s, 10 nodes, sketches none 2.8752/s, 8 nodes, sketches none
o11y_bench/q11 0.0063074/s, 7 nodes, sketches none 0.00630638/s, 5 nodes, sketches none
o11y_bench/q12 0.00691333/s, 2 nodes, sketches none 0.00691333/s, 2 nodes, sketches none
o11y_bench/q18 6.87727/s, 15 nodes, sketches none 62.9573/s, 13 nodes, sketches none
o11y_bench/q19 1.83727/s, 13 nodes, sketches none 62.9573/s, 11 nodes, sketches none
o11y_bench/q20 0.00630435/s, 4 nodes, sketches none 0.00630333/s, 3 nodes, sketches none
o11y_bench/q21 0.896918/s, 7 nodes, sketches none 2.38105/s, 6 nodes, sketches none
o11y_bench/q22 0.476918/s, 6 nodes, sketches none 2.38534/s, 5 nodes, sketches none
o11y_bench/q23 0.476918/s, 6 nodes, sketches none 2.38534/s, 5 nodes, sketches none
o11y_bench/q25 1.83727/s, 13 nodes, sketches none 62.9573/s, 11 nodes, sketches none
o11y_bench/q26 0.916235/s, 6 nodes, sketches none 22.6942/s, 5 nodes, sketches none
o11y_bench/q27 0.00630435/s, 4 nodes, sketches none 0.00630333/s, 3 nodes, sketches none
sketch_plan_snapshots/fuzz-issue903-raw-retained 1.86933e-09/s, 6 nodes, sketches ['Kll'] 3.23891e-09/s, 4 nodes, sketches none
sketch_plan_snapshots/nested-windows-15m-1h-mvp 3.99002e-06/s, 8 nodes, sketches ['DDSketch'] no plan: infeasible ("one of its inputs has no producer that can be costed")
sketch_plan_snapshots/quantile-grouped-mvp 1.966e-06/s, 12 nodes, sketches ['DDSketch'] no plan: infeasible ("one of its inputs has no producer that can be costed")
sketch_plan_snapshots/quantile-single-mvp 6.37724e-07/s, 5 nodes, sketches ['Kll'] no plan: infeasible ("one of its inputs has no producer that can be costed")
sketch_plan_snapshots/random-mixed-seed-862-long-w01-q3-mvp 1.90435e-05/s, 12 nodes, sketches ['DDSketch'] no plan: infeasible ("one of its inputs has no producer that can be costed")
sketch_plan_snapshots/random-mixed-seed-862-w01-q3-q8-mvp 6.43177e-06/s, 19 nodes, sketches ['DDSketch'] no plan: infeasible ("one of its inputs has no producer that can be costed")
sketch_plan_snapshots/random-mixed-seed-862-w02-q2-q7-mvp 6.44592e-06/s, 19 nodes, sketches ['DDSketch'] no plan: infeasible ("one of its inputs has no producer that can be costed")
sketch_plan_snapshots/ratio-of-quantiles-1h-mvp 6.48224e-06/s, 9 nodes, sketches ['DDSketch'] no plan: infeasible ("one of its inputs has no producer that can be costed")
sketch_plan_snapshots/shared-quantiles-1h-mvp 3.99011e-06/s, 8 nodes, sketches ['DDSketch'] no plan: infeasible ("one of its inputs has no producer that can be costed")
sketch_plan_snapshots/topk-rate-guarantees-ignored-mvp 2.91198e-08/s, 7 nodes, sketches ['CmsWithHeap'] no plan: infeasible ("one of its inputs has no producer that can be costed")

Causes: o11y_bench (all fixed_interval 60 s with no phase, 1 h range evaluation): delivers_on refuses a tumbling-window chain to a fixed interval without a phase, so the ingestion-time chains main selected are gone and each step recomputes its window (pending decision: fixed-interval phase, PR 4/6); q11, q20, q27: an exact Aggregate instead of exact accumulators (cheaper by cost). fuzz-issue903-raw-retained: KLL → exact plan: whole-result δ 0.001 over 10,000 series is below every listed size's per-value 0.01 (#1238 spec question 1). The nine -mvp sketch workloads (raw data not retained): no plan, Stage 3 infeasible because an input has no producer that can be costed (pending: PR 5, Stage 2 with raw data not retained).

Verification

  • cargo fmt --all; cargo clippy --workspace --all-features --all-targets -- -D warnings clean; crate_graph passes (6/6).
  • New/changed tests: devtools stage2_lists_every_edge_with_its_alpha_and_rejection, memo_section_lists_each_step_and_the_selected_candidates, the_selected_plan_costs_what_the_search_selected_it_for, pass2_holds_tumbling_windows_at_the_gcd_of_ranges_and_steps (was every divisor: the gcd decision of 2026-11-06), a_topk_heap_over_one_row_per_item_is_not_offered; viewer memo.test.mjs (14/14: θ/ω/floors, edges ✓/✗ with rejection, Stage 3 pruned with the dominating candidate), viewer_contract 2/2.
  • Independent review (fresh agent). Fixed: shared-node promotion applied to W's reference cost too (now witness only) + reference-cost test; terminology slips (remaining→in_output in Limit, leftover *_in_kept, usage_prices, UnitPrice, Priced test enum, "retained" where it meant materialized); a PR number in a design doc; dead RejectionCause::failed_a_check; the text section notes that #n cost rows are realized-node ids. Not taken: LongerWindow assignment carry-over and witness accuracy checks (limits above); an edges_into index in asap-types (the two printers regroup edges() themselves); a structured rejection instead of text and a smaller document (every edge is listed: DAG viewer / stage_pipeline: print memo candidates with properties, every producer–consumer edge with its feasibility and rejection reason, and every stage’s pruned candidates #1205 asks for it).

Test table (cargo test -p <crate> --all-features --no-fail-fast, at d61175c; the head commit after it only changes viewer wording and one comment; PR 8: 1350/97/9):

crate passed failed ignored
asap-executor 281 0 1
asap-types 197 0 0
asap-frontend-promql 218 0 0
asap-frontend-common 49 0 0
asap-planner 25 9 1
asap-accuracy 55 0 0
asap-cost 37 0 0
asap-frontend-metricsql 15 0 0
asap-frontend-sql 175 0 0
asap-sql-function-catalog 10 0 0
asap-stage0-memo 13 0 0
asap-stage1-pass1-per-computation-candidates 33 0 0
asap-stage1-pass2-reuse-candidates 20 0 0
asap-stage2-physical-optimization 15 0 0
asap-stage3-plan-selection 28 0 0
metricsql_common 3 0 0
asap-devtools 83 13 3
asap-integration-tests 139 30 4
total 1396 52 9

Viewer: memo.test.mjs 14/0; stages.test.mjs fails at load and test_viewer.py/test_server.py 92 run, 4 failures + 21 errors, all on test-fixtures/tumbling-windows-workload.json with raw data not retained, which does not plan (PR 5).

Remaining failures and causes (none stops at a todo!()):

  • plan_cost alignment (PR 7, pending), a not-materialized result shared by queries of one context is charged once per query by plan_cost, once by the objective: integration identical_exact_queries::{identical_exact_percentiles_share_work, different_exact_percentiles_share_the_input_window}, sharing_issue_regressions::{duplicate_exact_topk_shares_work, slower_repeat_reuses_faster_sum, sql_sum_shares_nested_windows_and_slower_readers, nested_exact_sum_reuses_newest_tumbling_windows}; planner summary_sharing::{identical_ungrouped_queries_share_their_producers, identical_sql_percentiles_share_one_producer}; devtools asap_planner::milp_search_objective_equals_plan_cost_on_example1_and_rangeheavy, fuzz exact_duplicate_of_a_shared_query_holds, finds_issue_902_… (the planner reuses the grouped result, but its scan is None and charged per query), finds_issue_903_… (violation found; W ∪ {q'} now 0.288/s, the test pins 0.166).
  • whole-result δ unreachable with the lists (refactor(stage3): constraint (K), accuracy at each query root (#1207 PR 8) #1238 spec question 1): planner columnar_batches::a_tumbling_window_is_built_…, runtime_install::{the_plan_carries_stage3s_predicted_usage, the_plan_file_carries_theta_and_coverage_and_reads_back, tumbling_windows_of_an_hour_share_one_definition_with_distinct_times}; installable::tests::view_refreshes_compile_only_the_nodes_below_them (whole-result δ 0.3: (K) infeasible); integration sharing_issue_regressions::grouped_filtered_quantile_plans, rollup_controlled::production_distinct_rollup_finds_shared_kmv.
  • fixed interval without a phase (pending, PR 4/6): devtools a_counter_function_over_a_declared_gauge_plans_with_a_warning (a windowless query-time scan cannot be costed without rows_held), fuzz exact_rollup_…, sketch_rollup_…, longer_window_rollup_… (W ∪ {q'} infeasible for the same scan), longer_window_witness_costs_over_a_lengthened_chain (no chain selected).
  • Stage 2/3 infeasible: an input with no costable producer, incl. raw data not retained (PR 5): devtools deployment_capabilities_flag_sets_the_capabilities_and_shows_them (no-raw part), the_nine_query_workload_{selects_a_valid_plan, lowers_and_installs}; integration joint_search_equivalence::different_windows_and_raw_retention_policies_produce_valid_plans; planner runtime_install::ignoring_accuracy_guarantees_plans_the_undeclared_p99_query; the viewer tests above.
  • executor-unsupported aggregates (PR 5, pending): planner runtime_install::check_compiles_reports_the_executors_error ("aggregate intent has no native implementation"); devtools workload_examples_select_valid_plans_with_linked_materialized_views (example1's exact top-k Aggregate has output_names: [], the executor needs one per measure: Pass 1 copies the frontend's group expression).
  • sum/avg error contract (PR 4, pending): integration stage1_logical_asap_optimization::{sql_avg_counts_its_own_input, sql_avg_from_sum_and_count_executes_as_avg, sql_avg_of_an_int64_column_keeps_the_exact_columns}, sql_to_post_asap::map_projection_export_keeps_the_map_over_an_avg_of_exact_states, per_computation_candidates_sql::a_sql_fold_over_estimates_leaves_its_row_count_unbound.
  • latency (L), out of scope: joint_search_equivalence::rootless_request_overhead_respects_capacity_and_latency.
  • tests that read a candidate through the selected plan (spec question 6): logical_candidates/explicit_*_candidate return the selected plan (L1) or a realized first-producer plan, and these tests pin a candidate the cost-optimal, (K)-checked selection no longer picks or Stage 2 no longer offers at query time: promql_to_post_asap::{promql_binary_arithmetic_keeps_an_unsupported_arm_exact, instant_topk_and_unsupported_child_remain_local_residuals, promql_quantile_of_rate_binds_kll_over_rate_accumulator, planner_heap_topk_reference_execution_matches_ground_truth}, sql_to_post_asap::{clickhouse_outer_sum_recursively_binds_inner_temporal_aggregate, sql_join_recursively_binds_both_temporal_aggregate_children, sql_full_query_retains_project_and_binds_inner_aggregate}, exact_composition::physical_export_carries_explicit_timing_for_a_composed_plan, promql_numeric_regressions::{sum_rate_and_increase_have_real_exact_accumulator_nodes, checked_ratio_must_not_certify_cross_zero_interpolation}, filtered_aggregates::{global_kll_quantile_over_no_row_is_null, kll_quantile_of_all_null_group_is_null, hand_filtered_kll_compiles_and_executes}, sketch_rollup::{distinct_counts_rolled_up_equal_the_direct_build, invalid_rollups_are_rejected}.

Compared with PR 8: +46 passing. Fixed here: every devtools todo!() (37), 6 fuzz witnesses, the CLI JSON tests, kll_tumbling_window_execution ×3 (θ on the hand-built builds), sql_to_post_asap ×2 (unqualified columns, PR 2b), viewer contract.

Deleted

Deleted Why
node_label, MemoNodes, group_node, memo.nodes and the node-table fields a candidate is one operator over groups (no node trees)
v1 viewer readers: upgradeTumblingWindowNames, upgradeMemoSteps, memoNodeDag, memoCandidateOwnNodes, memoPrunedText, selected_choice handling, reference-pricing/search_rejected selection rows, joint-search panel, stage2 pruned no producer writes them; no old-format parsing kept
enumeration.js, enumeration.test.mjs, memo-enumeration-small.json read memo.enumeration/memo.nodes, written by no producer since #1190
stage1_logical_asap_optimization.pruned_candidates (document) Pass 1 records nothing it leaves out since PR 0
Planner::price's last_root_is_key_filter, run_search_over witness path witnesses are costed, not planned
RejectionCause::failed_a_check no caller left
materialized_views::materialized_views_equal_the_stored_outputs_main_listed + stored_outputs_on_main.json pins main's pre-#970 DAGs in the old wire format (θ in operators and schemas, table qualifiers, no coverage on nodes)
tumbling_windows_reject_parameters_…'s relabelled-k batch assertion a sketch field type no longer carries θ (PR 0)
viewer_knows_every_view_refresh_and_rejection_cause's cause half the viewer's cause lists are deleted

New abstractions not in #1207 or the doc (pending approval)

  1. parameters_label, edge_rejection, same_candidate (devtools, pub because the asap_planner binary uses them).
  2. Planner::cost(…, last_root_is_derived) and the OneEvaluation(Memory) promotion of nodes a witness shares.
  3. Viewer document fields named in What (DAG viewer / stage_pipeline: print memo candidates with properties, every producer–consumer edge with its feasibility and rejection reason, and every stage’s pruned candidates #1205's shape, with rejection as text, see spec question 1).

Spec questions

  1. EdgeRejection (DAG viewer / stage_pipeline: print memo candidates with properties, every producer–consumer edge with its feasibility and rejection reason, and every stage’s pruned candidates #1205) names PhysicalRequirement, which no longer exists ($r^{\mathrm{phys}}$ is a function of the consumer's own properties). The tools derive the reason as text from the edge's $\alpha$ and the two candidates' properties; no type is added. OK, or add a structured type to asap-types?
  2. Pass 1 pruned list. DAG viewer / stage_pipeline: print memo candidates with properties, every producer–consumer edge with its feasibility and rejection reason, and every stage’s pruned candidates #1205 asks for every stage's pruned candidates; Pass 1 records none since PR 0 (it does not offer what it cannot compute). Add a list to Pass 1 (a PR 3 change), or keep none?
  3. Snapshots record a failed selection (the sketch harness used to fail on it), so a re-baseline runs to the end while lower-PR decisions are pending. Keep, or mark expected failures in the manifest?
  4. Witness accuracy is not checked (limits in Major algorithm changes). Add a (K)-style composed check over the witness DAG, or accept?
  5. Remaining stale-name hits: price_source / NodePriceSource (serialized in the plan cost and PredictedNodeUsage; a wire rename for its own PR?); "Materialization" as section titles in design docs (planner-layering.md still has the section); SQL text price; the Fargate pricing URL.
  6. Integration tests that pin a non-selected candidate (last group above): rewrite them to read memo candidates (a test-helper change across ~17 tests), or after the pending decisions?
  7. Document size: every edge of every input is listed (DAG viewer / stage_pipeline: print memo candidates with properties, every producer–consumer edge with its feasibility and rejection reason, and every stage’s pruned candidates #1205), quadratic in a group's candidates; fine for the examples (160 edges), large for Example 3 (972 candidates). Keep, or list only admissible and selected edges by default?

Lower-PR items (not changed)

  • No-raw-data and fixed-interval-without-phase workloads, sum/avg contracts, executor-unsupported aggregates, plan_cost alignment, whole-result δ: see the failure causes above (pending decisions).
  • PR 3/frontend: example1's exact top-k Aggregate candidate carries output_names: []; the executor refuses it (installable_plan_of → materialized_views_error).
  • planner-layering.md beyond the memo section and stage tables still describes pre-Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207 Stage 2/3 in places (e.g. physical-planning-and-deployment.md's "Stage 2 Materialization" naming).

@zzylol zzylol added dag-viewer DAG export and visualization tooling dev-tools Developer tools, diagnostics, and demos refactor testing Corpus, regression, differential, or benchmark tests labels Nov 7, 2026
…, ω and floors, physical edges with α, Stage 2 and Stage 3 pruned; delete old-format readers (#1207 PR 9)
…reference cost test, wording, delete dead failed_a_check (#1207 PR 9)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dag-viewer DAG export and visualization tooling dev-tools Developer tools, diagnostics, and demos refactor testing Corpus, regression, differential, or benchmark tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant