diff --git a/docs/mtp-design.md b/docs/mtp-design.md index 7c61eb2df..e8dbdf468 100644 --- a/docs/mtp-design.md +++ b/docs/mtp-design.md @@ -386,7 +386,10 @@ capture. | Grammar advancement, stop sequences, output presentation | Rust with ordered token-vector completions | Rust adapts among prepared widths from request-local prefix survival and measured -full-round costs. Every speculative graph is a complete proposal, verification, +full-round costs. A request's survival estimates start from the acceptance of +earlier requests' first rounds, as +[completed confidence and acceptance](scheduling-runtime-design.md#completed-confidence-and-acceptance) +specifies. Every speculative graph is a complete proposal, verification, and commit program. The controller learns from its emitted/offered/accepted records, or optional verification probabilities supplied by prepared graphs; graphs that declare `confidence_width` also return proposal-confidence diff --git a/docs/scheduling-runtime-design.md b/docs/scheduling-runtime-design.md index 6bae0de87..71e6db8cd 100644 --- a/docs/scheduling-runtime-design.md +++ b/docs/scheduling-runtime-design.md @@ -9,7 +9,7 @@ and selection), `utility.rs` (value arithmetic and cohort selection), `service.rs` (service prices), `cost.rs` (duration estimates), `refresh.rs` (timing-refresh and acceptance-learning credit), `learning.rs` (acceptance probes), `rounds.rs` (speculative round candidates and confidence forecasts), -and `speculation.rs` (acceptance estimates). +and `speculation.rs` (acceptance estimates and their population). [`scheduling-utility.md`](scheduling-utility.md) defines the utility and price flow. Python experiments are in `scripts/scheduling_sim/`; they do not run in Helios. The diagnostic `ignition debug-serve --direct` path has its own Python @@ -92,6 +92,24 @@ Unobserved suffixes retain their means. Ordinary selection uses the historical means; funded acceptance learning uses the bounded optimistic estimate `min(1, mean + sqrt(ln(20) / (2 * aged_weight)))`. +A request's acceptance estimates start from an acceptance population that the +scheduler keeps per proposal source and draft depth while the model is loaded. +Each population mean begins at `0.5^d` with the weight of eight observations. +A validated completion adds an acceptance observation, binary or from +`completion_speculative_survival`, when it is among the reporting request's +first eight at that depth. The first observations of a request describe the +rounds that an initial estimate governs. A population mean averages its +observations until its weight reaches 128 and then moves toward each +observation with weight 1/128. One request therefore supplies at most half of a +mean that held only the fixed start, and at most `1 - (127/128)^8`, about 6%, +of a mean at the weight cap. Ordinary and funded-probe rounds contribute alike; +width-one rounds offer no draft and contribute nothing, so requests admitted +before a drafting round completes start from `0.5^d` exactly. A request's +estimate takes the population mean at the initial weight of two, and two of +its own observations at a depth weigh as much as that start. When the catalog +declares device prefix allocation for learned proposals, selection reads +confidence forecasts and acceptance estimates keep the fixed `0.5^d` start. + A prepared program may also write `completion_speculative_survival`, cumulative products of `min(1, p(token)/q(token))` for evaluated prefixes. These replace binary acceptance observations at those depths. They affect future selection, @@ -594,7 +612,11 @@ includes its maximum packed token extent in capture headroom. measured synthetic catalogs: one, two, and sixteen long prefills; fitting and padded tails; high- and low-acceptance speculation; heterogeneous cohorts; padded batch capacities; long decode pairs; prefill/decode contention; graph -expiry and recovery; and output gaps. Mixed tests compare every legal cohort +expiry and recovery; and output gaps. +`helios/src/scheduler/acceptance_population_tests.rs` covers the acceptance +population: the first drafting round of a request after high- and +low-acceptance requests, the fixed start before any drafting round, and one +request's bounded contribution. Mixed tests compare every legal cohort against exhaustive useful-work/time enumeration, exercise fitting prompt tails, ordinary and speculative decode rows, malformed completions, independent shape expiry, and concurrent prompt/decode workloads. Native input tests check packed diff --git a/helios/README.md b/helios/README.md index 2920a0f22..4b916ecd0 100644 --- a/helios/README.md +++ b/helios/README.md @@ -392,7 +392,11 @@ requires confirmed stream drain before `release_after_shutdown` releases resources associated with the attempted submission. Each request owns conditional draft-acceptance statistics by depth and proposal -source. Validated speculative completions update only that request. The emitted +source. Validated speculative completions update that request. A request's +first eight observations at a depth also update the acceptance population from +which later requests' statistics start; see +[completed confidence and acceptance](../docs/scheduling-runtime-design.md#completed-confidence-and-acceptance). +The emitted accepted prefix supplies positive observations. When a correction token was emitted and an offered draft remained, the next draft supplies a rejection. Termination within an accepted prefix censors the remaining depths. Expected diff --git a/helios/src/scheduler.rs b/helios/src/scheduler.rs index 87aa18cd1..9fce35d34 100644 --- a/helios/src/scheduler.rs +++ b/helios/src/scheduler.rs @@ -665,6 +665,7 @@ pub struct RequestScheduler { proposal_source: speculation::ProposalSource, decoded_history: Option, forecasts: rounds::Forecasts, + acceptance_population: speculation::AcceptancePopulation, costs: cost::ActionCosts, refresh: refresh::RefreshState, tickets: HashMap, @@ -720,6 +721,7 @@ impl RequestScheduler { capabilities.page_tokens, )?; let forecasts = rounds::Forecasts::new(capabilities.completion_token_capacity); + let acceptance_population = speculation::AcceptancePopulation::new(capabilities.completion_token_capacity); Ok(Self { plan_owner: Arc::new(()), limits, @@ -756,6 +758,7 @@ impl RequestScheduler { proposal_source: speculation::ProposalSource::Mtp, decoded_history: None, forecasts, + acceptance_population, costs: cost::ActionCosts::default(), refresh: refresh::RefreshState::default(), tickets: HashMap::new(), @@ -1627,7 +1630,11 @@ impl RequestScheduler { window_snapshot_ready: false, window_restore: false, epoch: 0, - acceptance: speculation::AcceptanceProfile::new(self.capabilities.completion_token_capacity), + acceptance: if self.confidence_forecasts() { + speculation::AcceptanceProfile::new(self.capabilities.completion_token_capacity) + } else { + speculation::AcceptanceProfile::seeded(&self.acceptance_population) + }, accounting: self.accounting(), handle, state_slot: StateSlotId((1 + slot * execution_rows) as u32), @@ -2491,6 +2498,7 @@ impl RequestScheduler { let mut boundaries = Vec::new(); let mut captures = Vec::new(); let mut window_snapshots = Vec::new(); + let mut population_observations = Vec::new(); for (row, lane) in step.lanes.iter().enumerate() { let kind = step.lane_kind(row); let Ok(request) = self.request_mut(lane.request) else { @@ -2541,13 +2549,14 @@ impl RequestScheduler { { let vector = record.vector.as_ref().unwrap(); let offered = vector.offered_drafts as usize; - request.acceptance.observe( + request.acceptance.observe_shared( proposal_source, record.generated_tokens as u32, offered, vector.accepted_drafts as usize, vector.emitted_count > vector.accepted_drafts, survival.map(|rows| rows[lane.request.slot as usize].as_slice()), + Some(&mut |depth, value| population_observations.push((depth, value))), ); } if record.status == 3 { @@ -2631,6 +2640,9 @@ impl RequestScheduler { request.scheduled_consumed = request.consumed_tokens; } } + for (depth, value) in population_observations { + self.acceptance_population.observe(proposal_source, depth, value); + } for lane in &step.lanes { let pending = self.pending_token_ceiling(lane.request) as u64; if let Ok(request) = self.request_mut(lane.request) { diff --git a/helios/src/scheduler/acceptance_population_tests.rs b/helios/src/scheduler/acceptance_population_tests.rs new file mode 100644 index 000000000..9e687f3cf --- /dev/null +++ b/helios/src/scheduler/acceptance_population_tests.rs @@ -0,0 +1,249 @@ +/* + * SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-License-Identifier: Apache-2.0 + */ + +//! Seeding of a request's acceptance profile from the acceptance population. +//! +//! The fixtures are one request slot with host-allocated rounds of one to four +//! target rows lasting 28, 43.7, 48.5, and 53.2 ms, and two prefix-survival +//! curves: one at which the default policy drafts and one at which it does not. + +use super::*; +use speculation::{AcceptanceProfile, ProposalSource::{Copy, Mtp}}; + +const ROUND_NS: [u64; 4] = [28_000_000, 43_700_000, 48_500_000, 53_200_000]; +/// Expected output of 1.76, 2.32, and 2.74 tokens at two, three, and four rows. +const HIGH: [f32; 7] = [0.76, 0.56, 0.42, 0.30, 0.21, 0.15, 0.11]; +/// Expected output of 1.35, 1.47, and 1.51 tokens at two, three, and four rows. +const LOW: [f32; 7] = [0.35, 0.12, 0.04, 0.01, 0.0, 0.0, 0.0]; + +fn measured_engine() -> RequestScheduler { + let mut s = interactive_engine(1, true); + s.enable_speculation(&[1, 2, 3, 4]).unwrap(); + for (index, nanos) in ROUND_NS.into_iter().enumerate() { + set_cost(&mut s, GraphKind::Speculative, 1, index + 1, nanos); + } + s +} + +fn decoding(s: &mut RequestScheduler, id: u32, output: u32) -> RequestHandle { + let handle = add(s, id, 64, output); + ready(s, &[handle]); + handle +} + +/// Executes the selected plan. Each offered depth reports `curve` as its +/// survival and `accepted` drafts are emitted after the pending token. +fn round(s: &mut RequestScheduler, curve: &[f32; 7], accepted: usize) -> ScheduledPlan { + let plan = s.plan().expect("a decoding request must make progress"); + let fence = Fence(s.submitted.0 + 1); + s.commit(&plan, fence).unwrap(); + let duration = plan.estimated_duration_ns(); + s.advance_to(s.now_ns + duration).unwrap(); + let mut records = completion_records(s, &plan, fence, accepted); + let slot = plan.lanes[0].request.slot as usize; + let mut offered = 0; + if let Some(vector) = records[slot].vector.as_mut() { + offered = plan.lanes[0].tokens - 1; + vector.offered_drafts = offered as i64; + } + let mut survival = vec![-1.0; s.capabilities.completion_token_capacity - 1]; + survival[..offered].copy_from_slice(&curve[..offered]); + let output = s.complete_with_survival(fence, &records, gpu_time(duration), Some(&[survival])).unwrap(); + for token in output { + s.published_output(token.request, fence).unwrap(); + } + let request = plan.lanes[0].request; + if s.request(request).is_ok_and(|r| r.phase == RequestPhase::Finishing && r.pending == 0) { + s.finish(request).unwrap(); + } + plan +} + +/// Completes `requests` requests whose own estimates keep them drafting while +/// their rounds report `curve`. +fn drafting_requests(s: &mut RequestScheduler, curve: &[f32; 7], requests: u32, rounds: usize) { + for id in 0..requests { + let handle = decoding(s, 100 + id, rounds as u32 + 16); + for _ in 0..128 { + s.request_mut(handle).unwrap().acceptance.observe(Mtp, 0, 7, 7, true, None); + } + for _ in 0..rounds { + round(s, curve, 0); + } + s.cancel(handle).unwrap(); + } +} + +fn seed(s: &mut RequestScheduler) -> Vec { + let handle = decoding(s, 1, 4096); + let seed = (0..7).map(|depth| s.request(handle).unwrap().acceptance + .prefix_probability(Mtp, depth).unwrap()).collect(); + s.cancel(handle).unwrap(); + seed +} + +/// Index of the first drafting round among a request's first 64 decode +/// rounds, without acceptance probes. +fn first_drafting_round(s: &mut RequestScheduler) -> Option { + s.refresh.acceptance_credit_ns = f64::NEG_INFINITY; + (0..64).position(|_| round(s, &HIGH, 0).token_capacity > 1) +} + +#[test] +fn a_request_after_high_acceptance_requests_drafts_within_its_first_rounds() { + let mut fixed = measured_engine(); + decoding(&mut fixed, 1, 4096); + assert_eq!(first_drafting_round(&mut fixed), None); + + // A request whose own estimate already is the reported curve still spends + // its first two decode rounds at width one. + let mut known = measured_engine(); + let handle = decoding(&mut known, 1, 4096); + for _ in 0..128 { + known.request_mut(handle).unwrap().acceptance.observe(Mtp, 0, 7, 0, true, Some(&HIGH)); + } + assert_eq!(first_drafting_round(&mut known), Some(2)); + + for (requests, expected) in [(1, Some(14)), (3, Some(5)), (16, Some(3))] { + let mut seeded = measured_engine(); + drafting_requests(&mut seeded, &HIGH, requests, 8); + decoding(&mut seeded, 1, 4096); + assert_eq!(first_drafting_round(&mut seeded), expected, "{requests} requests"); + } +} + +#[test] +fn a_request_stays_at_width_one_after_low_acceptance_requests() { + let mut s = measured_engine(); + drafting_requests(&mut s, &LOW, 3, 8); + let handle = decoding(&mut s, 1, 4096); + let profile = s.request(handle).unwrap().acceptance(); + for depth in 0..3 { + assert!(profile.prefix_probability(Mtp, depth).unwrap() < 0.5f64.powi(depth as i32 + 1)); + } + for _ in 0..64 { + let plan = round(&mut s, &LOW, 0); + assert!(plan.token_capacity == 1 || plan.origin.is_acceptance()); + } +} + +#[test] +fn requests_start_from_the_fixed_prior_until_a_drafting_round_completes() { + let fixed = AcceptanceProfile::new(16); + let mut s = measured_engine(); + let handle = decoding(&mut s, 1, 4096); + s.refresh.acceptance_credit_ns = f64::NEG_INFINITY; + for _ in 0..64 { + assert_eq!(round(&mut s, &HIGH, 0).token_capacity, 1); + } + s.cancel(handle).unwrap(); + let handle = decoding(&mut s, 2, 4096); + let profile = s.request(handle).unwrap().acceptance(); + for source in [Mtp, Copy] { + for optimistic in [false, true] { + assert_eq!(profile.curve(source, 0, optimistic), fixed.curve(source, 0, optimistic)); + } + assert_eq!(profile.information(source, 0, 0), fixed.information(source, 0, 0)); + } +} + +#[test] +fn one_request_moves_the_population_by_at_most_its_share() { + let mut short = measured_engine(); + drafting_requests(&mut short, &[1.0; 7], 1, 8); + let mut long = measured_engine(); + drafting_requests(&mut long, &[1.0; 7], 1, 2000); + let expected = seed(&mut short); + // Eight observations of 1.0 against the fixed prior's weight of eight. + for depth in 0..3 { + assert_eq!(expected[depth], (0.5f64.powi(depth as i32 + 1) + 1.0) / 2.0); + } + for depth in 3..7 { + assert_eq!(expected[depth], 0.5f64.powi(depth as i32 + 1)); + } + assert_eq!(seed(&mut long), expected); + + // At the weight cap, a ninth observation moves a mean by 1/128 of its error. + let mut population = speculation::AcceptancePopulation::new(16); + for _ in 0..4096 { population.observe(Mtp, 0, 0.0); } + let before = population.mean(Mtp, 0); + let mut request = AcceptanceProfile::seeded(&population); + for _ in 0..2000 { + request.observe_shared(Mtp, 0, 1, 1, true, None, Some(&mut |depth, value| population.observe(Mtp, depth, value))); + } + let share = 1.0 - (1.0f64 - 1.0 / 128.0).powi(8); + assert!((population.mean(Mtp, 0) - (before + share * (1.0 - before))).abs() < 1e-12); + assert_eq!(population.mean(Copy, 0), 0.5); +} + +#[test] +fn a_low_acceptance_request_returns_to_width_one_after_a_high_acceptance_population() { + let mut s = measured_engine(); + drafting_requests(&mut s, &HIGH, 16, 8); + decoding(&mut s, 1, 4096); + s.refresh.acceptance_credit_ns = f64::NEG_INFINITY; + let widths: Vec<_> = (0..64).map(|_| round(&mut s, &LOW, 0).token_capacity).collect(); + let drafting = widths.iter().filter(|&&width| width > 1).count(); + assert!((1..=4).contains(&drafting), "{widths:?}"); + assert!(widths[32..].iter().all(|&width| width == 1), "{widths:?}"); +} + +#[test] +fn confidence_forecast_catalogs_keep_the_fixed_acceptance_prior() { + let fixture = measured_engine(); + let mut capabilities = fixture.capabilities.clone(); + capabilities.speculative_allocations = vec![(1, 7, 8, 8, crate::execution::PrefixAllocation::Gpu)]; + let mut s = RequestScheduler::new([0; 32], fixture.limits.clone(), capabilities).unwrap(); + for depth in 0..7 { + for _ in 0..64 { s.acceptance_population.observe(Mtp, depth, 1.0); } + } + let handle = add(&mut s, 1, 64, 64); + let fixed = AcceptanceProfile::new(16); + assert_eq!(s.request(handle).unwrap().acceptance().curve(Mtp, 0, false), fixed.curve(Mtp, 0, false)); +} + +/// Deterministic uniform draws for the simulated acceptance outcomes. +struct Draws(u64); + +impl Draws { + fn next(&mut self) -> f32 { + self.0 = self.0.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407); + (self.0 >> 40) as f32 / (1u64 << 24) as f32 + } +} + +/// Five consecutive 1,024-token requests on text with the `HIGH` curve. Each +/// round's accepted prefix is drawn from that curve. Prints, per request, the +/// width-one rounds before its first ordinary drafting round, its width-one +/// and total rounds, and its simulated decode time. +#[test] +fn simulated_requests_reach_drafting_sooner_as_the_population_fills() { + let mut s = measured_engine(); + let mut draws = Draws(1); + let mut before_drafting = Vec::new(); + let mut seconds = Vec::new(); + for id in 0..5 { + let handle = decoding(&mut s, id, 1024); + let start = s.now_ns; + let (mut first, mut narrow, mut rounds) = (None, 0, 0); + while s.request(handle).is_ok() { + let draw = draws.next(); + let width = s.plan().unwrap().token_capacity; + let accepted = HIGH[..width - 1].iter().take_while(|&&survival| draw < survival).count(); + let plan = round(&mut s, &HIGH, accepted); + if plan.token_capacity == 1 { narrow += 1; } + else if plan.origin == PlanOrigin::Ordinary { first.get_or_insert(rounds); } + rounds += 1; + } + let elapsed = (s.now_ns - start) as f64 / 1e9; + println!("request {id}: {} width-one rounds before ordinary drafting, {narrow} of {rounds} rounds at width one, {elapsed:.2} s, {:.1} tokens/s", + first.unwrap_or(rounds), 1024.0 / elapsed); + before_drafting.push(first.unwrap_or(rounds)); + seconds.push(elapsed); + } + assert!(before_drafting[0] > 100, "{before_drafting:?}"); + assert!(before_drafting[1..].iter().all(|&rounds| rounds <= 8), "{before_drafting:?}"); + assert!(seconds[1..].iter().all(|&elapsed| elapsed < seconds[0]), "{seconds:?}"); +} diff --git a/helios/src/scheduler/policy_tests.rs b/helios/src/scheduler/policy_tests.rs index f980e67be..d156ee1db 100644 --- a/helios/src/scheduler/policy_tests.rs +++ b/helios/src/scheduler/policy_tests.rs @@ -13,6 +13,9 @@ const MS: u64 = 1_000_000; #[path = "planning_tests.rs"] mod planning_tests; +#[path = "acceptance_population_tests.rs"] +mod acceptance_population_tests; + #[test] fn padding_pruning_keeps_three_requests_on_batch_four_and_two_requests_on_batch_two() { let mut s = engine(8, false); diff --git a/helios/src/scheduler/rounds.rs b/helios/src/scheduler/rounds.rs index 1bf21b386..f0fc3b1ce 100644 --- a/helios/src/scheduler/rounds.rs +++ b/helios/src/scheduler/rounds.rs @@ -72,9 +72,16 @@ impl RequestScheduler { .map(|lane| submission.lane_advance_ceiling(lane)).sum::()).sum() } + /// Learned proposals of a catalog with device prefix allocation are priced + /// from confidence forecasts; every other catalog is priced from verified + /// acceptance. + pub(super) fn confidence_forecasts(&self) -> bool { + self.proposal_source == speculation::ProposalSource::Mtp + && self.capabilities.speculative_allocations.iter().any(|shape| shape.4 == crate::execution::PrefixAllocation::Gpu) + } + pub(super) fn speculative_curve(&self, request: &LiveRequest, optimistic: bool) -> Vec { - if self.proposal_source == speculation::ProposalSource::Mtp - && self.capabilities.speculative_allocations.iter().any(|shape| shape.4 == crate::execution::PrefixAllocation::Gpu) { + if self.confidence_forecasts() { let forecast = self.forecasts.requests[request.handle.slot as usize].as_ref() .filter(|forecast| forecast.request == request.handle) .cloned().unwrap_or_else(|| Forecast::new(request.handle, &self.forecasts.population, request.generated_tokens)); diff --git a/helios/src/scheduler/speculation.rs b/helios/src/scheduler/speculation.rs index 31bdf90e8..c7b1fc311 100644 --- a/helios/src/scheduler/speculation.rs +++ b/helios/src/scheduler/speculation.rs @@ -86,10 +86,47 @@ impl PrefixEstimate { } } +/// Observations per draft depth that one request contributes to the acceptance +/// population. The fixed prior carries the same weight, so the first request +/// moves a population mean at most halfway. +const POPULATION_REQUEST_SHARE: u8 = 8; +/// Observation weight at which the population becomes a moving average. One +/// request then holds at most `1 - (1 - 1/128)^8`, about 6%, of a mean. +const POPULATION_WEIGHT_CAP: f64 = 128.0; + +/// Verified prefix survival by proposal source and draft depth across requests. +/// Each request contributes its first observations at a depth, which describe +/// the rounds a request's initial estimate governs. +pub(super) struct AcceptancePopulation { + /// Mean and observation weight per source and depth. + prefixes: [Vec<(f64, f64)>; 2], +} + +impl AcceptancePopulation { + pub fn new(token_capacity: usize) -> Self { + Self { prefixes: std::array::from_fn(|_| (0..token_capacity.saturating_sub(1)) + .map(|i| (0.5f64.powi(i as i32 + 1), f64::from(POPULATION_REQUEST_SHARE))).collect()) } + } + + #[cfg(test)] + pub fn mean(&self, source: ProposalSource, depth: usize) -> f64 { + self.prefixes[source as usize][depth].0 + } + + pub fn observe(&mut self, source: ProposalSource, depth: usize, value: f64) { + debug_assert!(value.is_finite() && (0.0..=1.0).contains(&value)); + let (mean, weight) = &mut self.prefixes[source as usize][depth]; + *weight = (*weight + 1.0).min(POPULATION_WEIGHT_CAP); + *mean += (value - *mean) / *weight; + } +} + /// Prefix survival of the complete proposal program, including copy overrides. #[derive(Clone)] pub struct AcceptanceProfile { prefixes: [Vec; 2], + /// Observations per source and depth already counted in the population. + shared: [Vec; 2], rounds: u64, } @@ -101,8 +138,18 @@ impl Default for AcceptanceProfile { impl AcceptanceProfile { pub fn new(token_capacity: usize) -> Self { - Self { prefixes: std::array::from_fn(|_| (0..token_capacity.saturating_sub(1)) - .map(|i| PrefixEstimate::new(0.5f64.powi(i as i32 + 1), 0)).collect()), rounds: 0 } + Self::seeded(&AcceptancePopulation::new(token_capacity)) + } + + /// A profile whose initial means are the population's, at the initial + /// weight of a `PrefixEstimate`. + pub(super) fn seeded(population: &AcceptancePopulation) -> Self { + Self { + prefixes: std::array::from_fn(|source| population.prefixes[source].iter() + .map(|&(mean, _)| PrefixEstimate::new(mean, 0)).collect()), + shared: std::array::from_fn(|source| vec![0; population.prefixes[source].len()]), + rounds: 0, + } } /// Depth zero is the first draft after the pending target token. @@ -132,7 +179,7 @@ impl AcceptanceProfile { self.prefixes[source as usize][depth].weight_at(generated) } - /// Rejection falsifies every longer offered prefix; termination censors its suffix. + #[cfg(test)] pub(super) fn observe( &mut self, source: ProposalSource, @@ -141,6 +188,22 @@ impl AcceptanceProfile { accepted: usize, rejected: bool, survival: Option<&[f32]>, + ) { + self.observe_shared(source, generated, offered, accepted, rejected, survival, None); + } + + /// Rejection falsifies every longer offered prefix; termination censors its suffix. + /// `population` receives the depth and value of each observation within + /// this request's population share. + pub(super) fn observe_shared( + &mut self, + source: ProposalSource, + generated: u32, + offered: usize, + accepted: usize, + rejected: bool, + survival: Option<&[f32]>, + mut population: Option<&mut dyn FnMut(usize, f64)>, ) { debug_assert!(accepted <= offered && offered <= self.prefixes[source as usize].len()); if offered == 0 { @@ -150,6 +213,11 @@ impl AcceptanceProfile { for depth in 0..observed { let value = survival.map_or(f64::from(depth < accepted), |p| f64::from(p[depth])); self.prefixes[source as usize][depth].observe(value, generated); + let shared = &mut self.shared[source as usize][depth]; + if let Some(population) = population.as_mut().filter(|_| *shared < POPULATION_REQUEST_SHARE) { + *shared += 1; + population(depth, value); + } } self.rounds = self.rounds.saturating_add(1); }