Skip to content

Kernel params and nonfinite - #45

Draft
beykyle wants to merge 11 commits into
rewritefrom
kernel-params-and-nonfinite
Draft

beykyle wants to merge 11 commits into
rewritefrom
kernel-params-and-nonfinite

Conversation

@beykyle

@beykyle beykyle commented Oct 25, 2026

Copy link
Copy Markdown
Owner

Summary

Three fixes:

  1. kernel(params=) is keyed by hyperparameter name. sklearn orders a kernel's hyperparameters alphabetically by attribute name, so a positional list for a kernel with two or more free hyperparameters could silently swap their priors.
  2. An error model that isn't finite at θ has zero density instead of raising. An overflowing kernel amplitude used to escape log_posterior as a scipy ValueError and crash the sampler.
  3. Faster marginal priors. norm, uniform and hypsecant use closed-form logpdf. A frozen scipy logpdf costs ~100 µs per call; a 10-parameter log_posterior drops from ~0.94 ms to ~0.25 ms.

Behaviour changes

params=

  • Takes a mapping keyed by kernel_param_names(kernel), e.g. params={"length_scale": ell, "periodicity": per}.
  • A positional list or tuple is accepted only for a kernel with a single free hyperparameter. Every existing caller is in that case.
  • Unordered containers (set, generator, a dict as a multi-element value) raise TypeError.
  • A key naming a fixed hyperparameter is reported as fixed.

Likelihood at an invalid θ (non-finite prediction or covariance): returns -inf, never raises.

  • The factorisations run unchecked. CompiledConstraint._stats treats a non-finite d2 or logdet as zero density. The hot path does no per-term scanning.
  • This also fixes a pre-existing Woodbury bug: an overflowing residual gave d2 = inf - inf = NaN, so log_posterior was nan and emcee raised.
  • Overflow no longer emits a numpy RuntimeWarning on every evaluation.
  • A rejected covariance containing NaN warns once per constraint and points to constraint.matrix(theta), which names the term.

Construction: constant terms are checked once, and a non-finite contribution raises ValueError naming the term. A NaN matrix term now gets that error instead of "not symmetric".

Predictives at an invalid θ: raise a public ValueError. Previously they returned NaN draws, which made the whole percentile band NaN.

  • Affected: predictive_draws (including model_only and given=), grid_draws, gp_predictive_draws and matrix().
  • The message names the sample row and its parameter values, and gives the likely causes: undiscarded burn-in or wrong column order; in a prior predictive, a prior covering a region where the error model is undefined.
  • A finite but singular covariance is still drawn from, because term subsets are legitimately rank-deficient.
  • heldout_log_predictive(given=) returns -inf at such a row, as it already did without given=.

Priors

  • norm and uniform are bit-identical to scipy wherever scipy is finite. They return -inf instead of raising where z overflows.
  • hypsecant stays finite in the tails, where scipy's overflows to -inf past |x| ≈ 710.
  • Families are matched by exact type; anything else falls back to dist.logpdf.

Other: KernelTerm uses Term's short repr. kernel_param_names is added to the API reference.

…l lists

sklearn orders a kernel's hyperparameters alphabetically by attribute name,
so a positional params= list for a kernel with two or more free
hyperparameters could silently swap their priors.  params= now accepts a
mapping keyed by kernel_param_names(kernel); a positional list is accepted
only when the kernel has a single free hyperparameter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A parametric term that overflowed (a huge kernel amplitude) put inf into the
covariance; scipy's check_finite then raised ValueError, which _factor did not
catch, so log_posterior raised instead of returning -inf.  Each term's value
on its active rows is now checked: at a theta a non-finite value is zero
density, and in a constant term it is a declaration error raised at
construction with the term named.  Overflow in the assembled sums is caught
at factorisation the same way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A frozen scipy logpdf spends ~100 us per call on argument handling, a
quarter of a cheap posterior evaluation with a dozen parameters.  The three
families get exact closed forms; hypsecant also stops overflowing to -inf
past |x| ~ 710 (log cosh via logaddexp), which made finite-difference
optimisers see inf - inf.  Everything else still calls dist.logpdf.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review findings on 2d5a05b.  The per-term finite check on the likelihood path
was redundant (scipy's check_finite already caught it), cost ~12% at N=1000,
and could not be tested.  Now the factorisations and solves run unchecked
(scipy returns NaN/inf factors rather than raising), and the compiled
constraint reads a non-finite d2 or logdet as zero density.  That one check
also fixes the Woodbury inf - inf = NaN on an overflowing residual, which
predates this branch.

- distance runs under np.errstate, so overflowing thetas no longer warn on
  every call (they did, and failed under -W error)
- a rejected covariance containing NaN warns once per constraint, pointing
  to constraint.matrix(theta) to name the term
- constant terms are checked once at construction, on their contribution
  (squares included), and a constant covariance with a non-finite logdet
  raises; a NaN matrix term reaches that check instead of failing as
  'not symmetric'

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review findings on 2d5a05b.  matrix() leaked the private
_SingularCovariance with a ~380-character dataclass repr, and on the base
branch a bad row gave NaN draws that turned the whole percentile band NaN.

- matrix() raises a plain ValueError naming the term (or, when only their
  sum overflows, saying so)
- predictive_draws (model_only and given= too), grid_draws and
  gp_predictive_draws check the final mean and covariance of each row and
  raise naming the row and its parameter values
- heldout_log_predictive(given=) returns -inf at such a row, as it already
  did without given=
- KernelTerm uses Term's short repr

A finite but singular covariance is still drawn from: term subsets are
legitimately rank-deficient.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review findings on bbecb86 and 3a644d6.

- norm: Python-float ** raised OverflowError past |z| ~ 1e154, crashing
  log_posterior where it used to return -inf; now z*z in scipy's operation
  order (bit-identical to scipy where finite)
- uniform: 'loc <= v <= loc+scale' disagreed with scipy by one ulp at the
  upper edge; now 0 <= (v-loc)/scale <= 1, as scipy computes it
- families are matched by exact type, so a user distribution named 'norm'
  falls back to its own logpdf
- params=: a set (or a dict as a multi-element value) silently reordered an
  anisotropic hyperparameter's priors; only a Parameter, list or tuple is
  accepted.  A key naming a fixed hyperparameter is reported as fixed.
- tests: the closed-form test now asserts the fast path is in use (it
  passed with the fast path disabled), and no longer asserts scipy's
  hypsecant overflow; kernel_param_names is in the API reference

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
matrix() raising when only a low-rank product overflows, and
predictive_draws raising on a non-finite prediction whose covariance is
finite.  Each test fails with its check removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@beykyle
beykyle changed the base branch from main to rewrite October 25, 2026 14:08
@beykyle
beykyle marked this pull request as draft October 25, 2026 14:09
The fast path for norm, uniform and hypsecant replaced the user's prior with
a hand-rolled copy, and needed its own exactness and overflow fixes.  Priors
are evaluated by dist.logpdf again, exactly as handed in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chol_logdet takes check_finite instead of having a private twin, and the
free-element count of a kernel is sklearn's kernel.n_dims rather than a
hand-written sum.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A line calibrated against data with a two-scale defect, with a
three-component spectral-mixture discrepancy written as a user sklearn
kernel and declared through T.kernel with a keyed params= mapping.

- periodogram-seeded multistart MAP inside problem.bounds (recipe 33)
- each component backed out on a fine grid with
  gp_predictive_draws(conditioned=True) (recipes 7, 40)
- the identifiability of theta: the L2 projection theta* is the only
  recoverable target, and Plumlee's orthogonal kernel, a second user
  kernel, makes it the estimand
- an emcee chain against each Laplace approximation: they agree for the
  orthogonal discrepancy; for Kennedy-O'Hagan the chain is several times
  wider, reaches theta_true, and the Laplace summary was wrong in both
  location and width

Registered in the notebook index test, the design document's table and
the examples toctree.  Runtime about 930 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- normalization_and_covariance_structure: the first section was an H3
  directly under the H1 title (myst.header, non-consecutive level)
- spectral_mixture_discrepancy: a relative link to gp_discrepancy.ipynb
  does not resolve through the docs/examples symlink
  (myst.xref_missing); name the notebook in backticks instead, as the
  docs do elsewhere

Checked with sphinx-build -W --keep-going, intersphinx aside (its
inventories are unreachable from the build machine used).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants