Skip to content

Use the intended fallback threshold in Algorithm D - #1022

Merged
devmotion merged 4 commits into
JuliaStats:masterfrom
Sinkleberg:codex/sequential-sampling-fallback
Nov 19, 2026
Merged

devmotion merged 4 commits into
JuliaStats:masterfrom
Sinkleberg:codex/sequential-sampling-fallback

Conversation

@Sinkleberg

@Sinkleberg Sinkleberg commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Algorithm D compared the remaining population with n / 13. That kept it from switching to Algorithm A when the remaining sample was dense. This uses the intended 13 * n threshold and passes the remaining population as a view.

This addresses the second suggestion in #998. The public algorithm selector is outside this change.

The latest cleanup clarifies which tests exercise D and separates the empty-sample RNG checks. All eight affected assertions passed on Julia 1.10.13. The boundary, handoff and full package tests passed before this test-only cleanup; the full suite was not rerun for it.

Developed with Codex and independently reviewed.

@devmotion
devmotion force-pushed the codex/sequential-sampling-fallback branch from e1b7267 to 51384eb Compare November 10, 2026 22:17

@devmotion devmotion left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! The threshold fix seems correct, I checked and it matches Vitter's pseudocode in both papers:

  • Vitter (1984), "Faster Methods for Random Sampling", CACM 27(7):703–718 (PDF). p. 715 says alpha_inverse (≈ 1/α, with α ≈ 0.07) and that threshold is "always equal to alpha_inverse × n". p. 716 has threshold := alpha_inverse × n; while (n > 1) and (threshold < N) … threshold := threshold − alpha_inverse. seqsample_d! is a line-by-line port of that p. 716 version.
  • Vitter (1987), "An Efficient Algorithm for Sequential Random Sampling", ACM TOMS 13(1):58–67 (PDF). In Appendix A2 (p. 66): negalphainv := −13; threshold := −negalphainv × n, with "we used α = 1/13". This is where the 13 comes from.

I also checked the sampler numerically with the PR's code. In the pure-D regime (N = 200, n = 3), the inclusion probabilities and the exact law of the first selected index match. In the mixed D→A regime, chi-square tests on inclusion counts are consistent with uniform. With dense samples from a Vector it's also faster, since A is used for the tail and the tail is no longer copied.

A few comments inline. The main one is that the existing statistical test for D no longer exercises D's rejection loop.

Comment thread test/sampling.jl Resolved
Comment thread src/sampling.jl Resolved
Comment thread src/sampling.jl Resolved
Comment thread test/sampling.jl Resolved
Comment thread test/sampling.jl Resolved
Comment thread test/sampling.jl Resolved
Comment thread test/sampling.jl Resolved
@devmotion
devmotion merged commit 26e7554 into JuliaStats:master Nov 19, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants