diff --git a/AGENTS.md b/AGENTS.md index 1c885c6..1bebf58 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -90,6 +90,14 @@ This section documents the reasons for each entry in the `stubtest-allowlist.txt **Reason:** This entry is necessary because the `MethodName` type alias is defined as a `typing.Literal` in the implementation, but it is simplified to `str` in the corresponding stub file (`.pyi`). This simplification is intentional to avoid duplicating the literal values in the stub, which would make it harder to maintain. (An equivalent entry for the `ellphi.solver` re-export was needed with older mypy versions but is reported as unused since mypy 1.20.) +* `ellphi._minimax_python.MethodName` + +**Reason:** The stub declares the same seven-value `typing.Literal` alias as the implementation, but stubtest cannot verify multi-value `Literal` aliases: it expands the stub alias into a union of single-value literals and reports the runtime alias as "not a Union". Simplifying the stub to `str` does not avoid the report either, so the entry keeps the precise stub type. + +* `ellphi.cech.MethodName` + +**Reason:** The stub declares the precise eight-value `typing.Literal` alias, including the public `"auto"` route, but stubtest cannot verify multi-value `Literal` aliases and reports the runtime alias as "not a Union". The precise alias is retained for type checking. + #### Managing the Allowlist When adding a new entry to the `stubtest-allowlist.txt` file, you must also update the "Allowlist Justification" section in this document to include a clear and concise explanation for why the entry is needed. This ensures that the allowlist remains transparent and easy to manage. diff --git a/README.md b/README.md index 588d95c..60eb5fe 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,13 @@ Starting from an ordinary point cloud, it estimates local covariance, inflates * - **N-Dimensional Support**: Works with n-dimensional ellipsoids, allowing for analysis in higher-dimensional spaces. - **Visualization**: Comes with helper functions to quickly visualize ellipse clouds using Matplotlib. +For simplex-level anisotropic filtrations, `ellphi.cech(coefs)` computes the +Čech time for several ellipsoids, while `ellphi.cech_grad(coefs)` also returns +the gradient with respect to the packed coefficients. The default `method="auto"` +tries the robust many-body route and records the selected method, convergence +status, gap, iteration count, and any fallback stages in the result's `info` +field. + ## Installation Install from PyPI: diff --git a/docs/guide/cech_filtration.md b/docs/guide/cech_filtration.md new file mode 100644 index 0000000..39fe573 --- /dev/null +++ b/docs/guide/cech_filtration.md @@ -0,0 +1,111 @@ +# Čech filtration + +The public `ellphi.cech` API computes the anisotropic Čech filtration value for +a simplex of packed ellipsoid coefficient vectors. For +`f_i(x)` describing the growing ellipsoids +`E_i(t) = {f_i <= t^2}`, `cech(coefs)` returns the Čech filtration time +`t`, where + +`t^2 = alpha(sigma) = min_x max_i f_i(x)` + +is the least squared scale at which all bodies share a common point. + +The result fields are ordered as +`(t, point, mu, support, active_set, info)`. The gradient result fields are +ordered as `(t, point, mu, dt_dcoef, support, active_set, info)`. + +- `t`: the Čech filtration time, with `t^2 = alpha(sigma)`. +- `point`: the common intersection point `x*` at the critical scale. +- `mu`: the Lagrange multipliers, or dual weights on the simplex. +- `support`: the thresholded weight support + `{i : mu_i > weight_tol}`. +- `active_set`: the tight-constraint set at the returned point, + `I = {i : t^2 - f_i(point) <= active_tol * max(1, t^2)}`; if `alpha` is + clipped at zero, the actual clipping shift `t^2 - alpha` is added to this + threshold. +- `info`: a `CechInfo` record containing `requested_method`, `method_used`, + `converged`, the final `gap`, the selected stage's `n_iter`, and its + `stages` tuple. Each `CechStage` records `method`, `converged`, `status`, + `gap`, and `n_iter`. + +The public `cech` and `cech_grad` signatures intentionally do not expose +matrix stabilization controls such as `regularization`, +`condition_number_limit`, or `max_conditioning_steps`. Stabilized solves are +available only through the private `ellphi._minimax_python` engine for +research use, and the public coefficient-space gradient carries no guarantee +for those stabilized solves. + +At the critical scale the intersection is the single point `x*`. The active +ellipsoid boundaries pass through `x*` and satisfy +`sum_i mu_i grad f_i(x*) = 0`, so their normals are positively dependent; +the boundaries are not tangent to each other. Pairwise tangency is only the +`k = 2` special case: for `k = 2` the Čech time equals the tangency time +computed by `tangency()`, and `cech()` agrees with it. + +Up to numerical tolerance, `support` is a subset of `active_set`; under +strict complementarity (ND1), they coincide. For three unit balls centred at +`(0, 0)`, `(2, 0)`, and `(0, 2)`, the point is `(1, 1)`, support has +two indices, and `active_set` has three. + +Every solver method uses the same accepted-result gap contract: +`gap <= tol * max(1, abs(dual value))`. Thus `tol` is an all-method relative +duality-gap tolerance, not a pairwise-FW-only control or a Newton residual +tolerance. The Frank-Wolfe description below refers only to the first stage +of the default route. + +Failure diagnostics depend on the route. A named method runs one stage and +raises `RuntimeError` with that stage's method, status, iteration count, final +gap, `tol`, and relative-gap scale. The `auto` route lists both attempted +stages, including each method, status, final gap, and iteration or evaluation +count; it does not collapse them into a single named-method diagnostic. + +## Default route + +The default `method="auto"` is a declared two-stage route. Stage 1 runs +`fw+brentq+newton` with the caller's `tol`, `max_iter`, and `newton_tol`. If it +does not converge, stage 2 runs the gap-enforced dual-weight solver +`scipy-slsqp` from uniform weights with the same `tol`. `info.stages` records +both stages when the fallback is used, including the first stage's status and +final gap. The selected stage is reported by `info.method_used`, and the +requested route remains visible in `info.requested_method`. + +The route never reports a non-converged result as success. If both stages fail, +`cech` raises `RuntimeError` with each stage's status, final gap, and iteration +or evaluation count. A named method runs only that method and raises on +failure; named methods never fall back. + +Each Frank-Wolfe step in stage 1 selects the best vertex (largest constraint +value) and the worst active vertex (smallest constraint value), then swaps mass +along `e_s - e_v` with step at most `mu[v]`. Its gap tolerance is relative to +`max(1, |dual value|)`. + +The private engine's internal field named `active_set` is the weight support +(historical ellcech naming) and is not part of the public API. + +If `unpack_conic` returns `(A_i, b_i, c_i)`, the center is +`xbar_i = -A_i^{-1} b_i` and the completed-square constant is +`delta_i = c_i - b_i^T A_i^{-1} b_i`. The public API passes this computed +`delta_i` to the many-body engine, including when it is small, so +general independent constants are supported. + +A packed row encodes its constant as +`c_i = xbar_i^T A_i xbar_i + delta_i`. For `|xbar_i|^2 >> 1`, the computed +`delta_i` can lose precision in the centered quadratic term and its +subtraction from `c_i`. The implementation reports a per-row rounding +estimate based on the solve residual and the dot-product/subtraction +dimension; this is an estimate, not a rigorous bound. Callers needing exact +normalization at large translations should center their data first. Packed +input requires `d >= 2`; d=1 packed conics are not supported and raise +`ValueError`. + +The many-body engine uses the completed-square offsets above. Thus +`cech_grad(coefs)` returns `dt_dcoef`, the gradient for arbitrary +independent packed-coordinate perturbations, including the constant component. +For the upper-triangular packed basis +`[x_i**2, 2*x_i*x_j (i < j), 2*x_i, 1]`, the formula is +`mu_i * basis(point) / (2*t)`. It assumes a non-degenerate input, at least +two indices in `active_set`, multipliers supported on `active_set`, and an +exact weighted linear solve. Convergence or a small residual alone is not a +derivative certificate, and the gradient is undefined at `t == 0`. +For far-translated inputs, the uncentred packed basis used by this derivative +can be poorly conditioned. diff --git a/mkdocs.yml b/mkdocs.yml index 5503958..94b460e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -69,6 +69,7 @@ nav: - User Guide: - TDA Workflow: guide/tda_workflow.md - Differentiable Tangency: guide/differentiable.md + - Čech Filtration: guide/cech_filtration.md - Notebook Examples: guide/notebooks.md - API Reference: - Overview: reference/index.md diff --git a/scripts/compare_minimax_engine.py b/scripts/compare_minimax_engine.py new file mode 100644 index 0000000..cb2b8d0 --- /dev/null +++ b/scripts/compare_minimax_engine.py @@ -0,0 +1,406 @@ +"""Compare the private minimax engine with its ellcech source revision. + +This is an opt-in fidelity check, not part of the default pytest run. Generate +the shared inputs and the two engine records in separate processes, then +compare them:: + + uv run --no-sync python scripts/compare_minimax_engine.py \ + inputs --output /tmp/minimax-inputs.json + /path/to/ellcech/.venv/bin/python scripts/compare_minimax_engine.py \ + generate --engine legacy --revision 82d13e3 \ + --inputs /tmp/minimax-inputs.json --output /tmp/legacy.json + /path/to/ellcech/.venv/bin/python scripts/compare_minimax_engine.py \ + generate --engine candidate --revision HEAD \ + --source-root . --inputs /tmp/minimax-inputs.json \ + --output /tmp/candidate.json + uv run --no-sync python scripts/compare_minimax_engine.py \ + compare /tmp/legacy.json /tmp/candidate.json + +Only cases for which the legacy engine reports ``converged=True`` are fidelity +requirements. ``METHOD_TOLERANCES`` are empirical comparison thresholds, not +solver settings; each method has a rationale recorded in ``METHOD_REASONS``. +For every method, records with ``alpha_legacy <= 1`` use the strict empirical +threshold, while records with larger alpha use a scale-aware threshold based on +``max(1, abs(alpha_legacy))``. That scale keeps the comparison proportional to +the magnitude of the legacy value while avoiding a division by a small value. +Circumcenter and weight deviations remain reported diagnostics, and the +candidate must also report convergence. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import platform +import sys +from pathlib import Path +from typing import Any, Callable + +import numpy as np + + +METHODS = ( + "fw+bisect", + "fw+brentq", + "fw+bisect+newton", + "fw+brentq+newton", + "fw+bisect+damped-newton", + "scipy-slsqp", + "newton-cold", +) +BASE_SEED = 260_027 +INSTANCES_PER_SHAPE = 20 +METHOD_TOLERANCES = { + "fw+bisect": 1e-12, + "fw+brentq": 1e-12, + "fw+bisect+newton": 1e-11, + "fw+brentq+newton": 1e-11, + "fw+bisect+damped-newton": 1e-11, + "scipy-slsqp": 1e-6, + "newton-cold": 1e-12, +} +METHOD_REASONS = { + "fw+bisect": "52-step bisection line search and relative-gap contract", + "fw+brentq": "Brent line search and relative-gap contract", + "fw+bisect+newton": "bisection warm start with Newton polishing", + "fw+brentq+newton": "Brent warm start with damped Newton polishing", + "fw+bisect+damped-newton": "bisection warm start with damped Newton polishing", + "scipy-slsqp": "dual-weight SLSQP with shared gap enforcement and polishing", + "newton-cold": "uniform-start Newton stability baseline", +} + + +def _solver(engine: str, source_root: Path | None) -> Callable[..., Any]: + if engine == "legacy": + from ellphi_alpha.minimax import solve_minimax + else: + if source_root is not None: + sys.path.insert(0, str(source_root.resolve() / "src")) + from ellphi._minimax_python import solve_minimax + + return solve_minimax + + +def _random_instance(k: int, d: int, instance: int) -> tuple[np.ndarray, np.ndarray]: + rng = np.random.default_rng(np.random.SeedSequence([BASE_SEED, k, d, instance])) + matrices = [] + for _ in range(k): + q, _ = np.linalg.qr(rng.standard_normal((d, d))) + eigenvalues = np.exp(rng.uniform(-1.0, 1.0, d)) + matrix = (q * eigenvalues) @ q.T + matrices.append(0.5 * (matrix + matrix.T)) + return np.stack(matrices), rng.standard_normal((k, d)) + + +def make_inputs(output: Path) -> None: + """Write the exact deterministic inputs shared by both implementations.""" + instances = [] + for k in range(1, 7): + for d in range(1, 5): + for instance in range(INSTANCES_PER_SHAPE): + matrices, centers = _random_instance(k, d, instance) + instances.append( + { + "k": k, + "d": d, + "instance": instance, + "matrices": matrices.tolist(), + "centers": centers.tolist(), + } + ) + payload = { + "base_seed": BASE_SEED, + "instances_per_shape": INSTANCES_PER_SHAPE, + "instances": instances, + } + output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n") + print(f"wrote {len(instances)} inputs to {output}") + + +def _sha256(path: Path) -> str: + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def generate( + engine: str, + revision: str, + inputs: Path, + output: Path, + source_root: Path | None, +) -> None: + """Generate deterministic solver records for one implementation.""" + solve_minimax = _solver(engine, source_root) + input_payload = json.loads(inputs.read_text()) + records = [] + for item in input_payload["instances"]: + matrices = np.asarray(item["matrices"], dtype=float) + centers = np.asarray(item["centers"], dtype=float) + for method in METHODS: + result = solve_minimax(matrices, centers, method=method) + records.append( + { + "k": item["k"], + "d": item["d"], + "instance": item["instance"], + "method": method, + "alpha": result.alpha, + "circumcenter": result.circumcenter.tolist(), + "weights": result.weights.tolist(), + "converged": result.converged, + } + ) + + payload = { + "manifest": { + "engine": engine, + "revision": revision, + "source_root": str(source_root.resolve()) if source_root else None, + "entry_point": "scripts/compare_minimax_engine.py generate", + "configuration": { + "methods": METHODS, + "k": [1, 6], + "d": [1, 4], + "instances_per_shape": INSTANCES_PER_SHAPE, + "comparison_thresholds": METHOD_TOLERANCES, + }, + "random_seed": BASE_SEED, + "input_identity": { + "path": str(inputs), + "sha256": _sha256(inputs), + "generator": "SeedSequence([base_seed, k, d, instance])", + }, + "output": str(output), + "execution_backend": "local Python/NumPy/SciPy", + "python": platform.python_version(), + "numpy": np.__version__, + "fallback_status": "not-applicable", + "status": "complete", + }, + "records": records, + } + output.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n") + print(f"wrote {len(records)} records to {output}") + + +def _record_key(record: dict[str, Any]) -> tuple[int, int, int, str]: + return ( + record["k"], + record["d"], + record["instance"], + record["method"], + ) + + +def _expected_keys(payload: dict[str, Any]) -> set[tuple[int, int, int, str]]: + """Return the complete deterministic grid declared by a payload manifest.""" + try: + configuration = payload["manifest"]["configuration"] + methods = tuple(configuration["methods"]) + k_min, k_max = configuration["k"] + d_min, d_max = configuration["d"] + instances_per_shape = configuration["instances_per_shape"] + except (KeyError, TypeError, ValueError) as exc: + raise ValueError("payload manifest does not declare a complete grid") from exc + return { + (k, d, instance, method) + for k in range(k_min, k_max + 1) + for d in range(d_min, d_max + 1) + for instance in range(instances_per_shape) + for method in methods + } + + +def _records_by_key( + payload: dict[str, Any], label: str +) -> dict[tuple[int, int, int, str], dict[str, Any]]: + """Validate uniqueness and completeness before indexing records.""" + records = payload.get("records") + if not isinstance(records, list): + raise ValueError(f"{label} payload has no records list") + indexed: dict[tuple[int, int, int, str], dict[str, Any]] = {} + duplicates = [] + for record in records: + key = _record_key(record) + if key in indexed: + duplicates.append(key) + indexed[key] = record + if duplicates: + raise ValueError(f"{label} payload has duplicate record keys: {duplicates[:3]}") + expected = _expected_keys(payload) + missing = sorted(expected - indexed.keys()) + extra = sorted(indexed.keys() - expected) + if missing or extra: + raise ValueError( + f"{label} payload has incomplete grid: " + f"missing={missing[:3]}, extra={extra[:3]}" + ) + return indexed + + +def compare(legacy_path: Path, candidate_path: Path) -> int: + """Compare candidate outputs where the legacy engine converged.""" + legacy_payload = json.loads(legacy_path.read_text()) + candidate_payload = json.loads(candidate_path.read_text()) + legacy = _records_by_key(legacy_payload, "legacy") + candidate = _records_by_key(candidate_payload, "candidate") + if legacy.keys() != candidate.keys(): + missing = sorted(legacy.keys() - candidate.keys()) + extra = sorted(candidate.keys() - legacy.keys()) + raise ValueError( + f"record-key mismatch: missing={missing[:3]}, extra={extra[:3]}" + ) + + converged = 0 + mismatches = [] + strict_tolerance_violations = [] + converged_by_method = {method: 0 for method in METHODS} + per_method = { + method: { + "legacy_converged": 0, + "mismatches": 0, + "max_abs_dalpha": 0.0, + "max_abs_dx": 0.0, + "max_abs_dmu": 0.0, + "alpha_le_one": { + "count": 0, + "max_abs_deviation": 0.0, + "max_abs_dalpha": 0.0, + "max_abs_dx": 0.0, + "max_abs_dmu": 0.0, + "tolerance": METHOD_TOLERANCES[method], + }, + "alpha_gt_one": { + "count": 0, + "max_abs_deviation": 0.0, + "max_relative_deviation": 0.0, + "max_relative_dalpha": 0.0, + "max_abs_dalpha": 0.0, + "max_abs_dx": 0.0, + "max_abs_dmu": 0.0, + "tolerance": METHOD_TOLERANCES[method], + }, + "declared_tolerance": METHOD_TOLERANCES[method], + **({"reason": METHOD_REASONS[method]} if method in METHOD_REASONS else {}), + } + for method in METHODS + } + for key, old in legacy.items(): + if not old["converged"]: + continue + converged += 1 + method = key[-1] + converged_by_method[method] += 1 + stats = per_method[method] + stats["legacy_converged"] += 1 + new = candidate[key] + dalpha = abs(float(new["alpha"] - old["alpha"])) + dx = float( + np.max( + np.abs( + np.asarray(new["circumcenter"]) - np.asarray(old["circumcenter"]) + ) + ) + ) + dmu = float( + np.max(np.abs(np.asarray(new["weights"]) - np.asarray(old["weights"]))) + ) + stats["max_abs_dalpha"] = max(stats["max_abs_dalpha"], dalpha) + stats["max_abs_dx"] = max(stats["max_abs_dx"], dx) + stats["max_abs_dmu"] = max(stats["max_abs_dmu"], dmu) + deviation = max(dalpha, dx, dmu) + alpha_scale = max(1.0, abs(float(old["alpha"]))) + if old["alpha"] <= 1: + partition = stats["alpha_le_one"] + tolerance = METHOD_TOLERANCES[method] + if dalpha > tolerance: + strict_tolerance_violations.append( + { + "record": key, + "alpha_legacy": old["alpha"], + "tolerance": tolerance, + "dalpha": dalpha, + "dx": dx, + "dmu": dmu, + } + ) + else: + partition = stats["alpha_gt_one"] + tolerance = METHOD_TOLERANCES[method] + tolerance = max(tolerance, 1e-9 * alpha_scale) + partition["max_relative_deviation"] = max( + partition["max_relative_deviation"], deviation / alpha_scale + ) + partition["max_relative_dalpha"] = max( + partition["max_relative_dalpha"], dalpha / alpha_scale + ) + partition["count"] += 1 + partition["max_abs_deviation"] = max(partition["max_abs_deviation"], deviation) + partition["max_abs_dalpha"] = max(partition["max_abs_dalpha"], dalpha) + partition["max_abs_dx"] = max(partition["max_abs_dx"], dx) + partition["max_abs_dmu"] = max(partition["max_abs_dmu"], dmu) + partition["tolerance"] = tolerance + same = new["converged"] and dalpha <= tolerance + if not same: + stats["mismatches"] += 1 + mismatches.append(key) + + summary = { + "records_total": len(legacy), + "records_checked": converged, + "records_skipped": len(legacy) - converged, + "records_compared": converged, + "legacy_converged": converged, + "legacy_converged_by_method": converged_by_method, + "mismatches": len(mismatches), + "first_mismatches": mismatches[:10], + "strict_tolerance_violations": strict_tolerance_violations[:10], + "per_method": per_method, + "status": ( + "complete" + if converged and not mismatches and not strict_tolerance_violations + else "failed" + ), + } + print(json.dumps(summary, indent=2, sort_keys=True)) + return 1 if not converged or mismatches or strict_tolerance_violations else 0 + + +def main() -> int: + parser = argparse.ArgumentParser() + subparsers = parser.add_subparsers(dest="command", required=True) + + inputs_parser = subparsers.add_parser("inputs") + inputs_parser.add_argument("--output", type=Path, required=True) + + generate_parser = subparsers.add_parser("generate") + generate_parser.add_argument( + "--engine", choices=("legacy", "candidate"), required=True + ) + generate_parser.add_argument("--revision", required=True) + generate_parser.add_argument("--source-root", type=Path) + generate_parser.add_argument("--inputs", type=Path, required=True) + generate_parser.add_argument("--output", type=Path, required=True) + + compare_parser = subparsers.add_parser("compare") + compare_parser.add_argument("legacy", type=Path) + compare_parser.add_argument("candidate", type=Path) + + args = parser.parse_args() + if args.command == "inputs": + make_inputs(args.output) + return 0 + if args.command == "generate": + generate( + args.engine, + args.revision, + args.inputs, + args.output, + args.source_root, + ) + return 0 + return compare(args.legacy, args.candidate) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/ellphi/__init__.py b/src/ellphi/__init__.py index fe3e6f0..9bc822a 100644 --- a/src/ellphi/__init__.py +++ b/src/ellphi/__init__.py @@ -14,6 +14,7 @@ two ellipses. - `ellphi.visualization`: Helper functions for visualizing ellipses and ellipse clouds. +- `ellphi.cech`: Anisotropic Čech filtration values and gradients. """ import numpy as np @@ -46,6 +47,16 @@ from .grad import TangencyGrad, tangency_grad, pdist_tangency_grad, coef_from_cov_grad +# anisotropic Čech filtration +from .cech import ( + CechInfo, + CechResult, + CechStage, + CechGrad, + cech, + cech_grad, +) + FloatArray = NDArray[np.float64] @@ -76,6 +87,13 @@ "tangency_grad", "pdist_tangency_grad", "coef_from_cov_grad", + # anisotropic Čech filtration + "CechStage", + "CechInfo", + "CechResult", + "CechGrad", + "cech", + "cech_grad", "__version__", "version_info", ] diff --git a/src/ellphi/__init__.pyi b/src/ellphi/__init__.pyi index e608f45..1868a11 100644 --- a/src/ellphi/__init__.pyi +++ b/src/ellphi/__init__.pyi @@ -32,6 +32,14 @@ from .grad import ( pdist_tangency_grad as pdist_tangency_grad, coef_from_cov_grad as coef_from_cov_grad, ) +from .cech import ( + CechInfo as CechInfo, + CechGrad as CechGrad, + CechResult as CechResult, + CechStage as CechStage, + cech as cech, + cech_grad as cech_grad, +) from ._version import __version__ as __version__ def version_info() -> str: ... @@ -59,6 +67,12 @@ __all__ = [ "tangency_grad", "pdist_tangency_grad", "coef_from_cov_grad", + "CechResult", + "CechGrad", + "CechStage", + "CechInfo", + "cech", + "cech_grad", "FloatArray", "__version__", "version_info", diff --git a/src/ellphi/_minimax_grad_python.py b/src/ellphi/_minimax_grad_python.py new file mode 100644 index 0000000..ec4f676 --- /dev/null +++ b/src/ellphi/_minimax_grad_python.py @@ -0,0 +1,163 @@ +"""Gradient of the many-body minimax filtration value. + +Provenance +---------- +Adapted from uda-lab/ellcech ``src/ellphi_alpha/minimax_grad.py`` at commit +82d13e3e174f4903cdcd6adcee1f469348361c91, by Tomoki Uda (the author of +EllPHi). The only later change to that file on ellcech ``main`` (commit +ebe63a42060ac6420ad42d90b3ffe942969d54ec) is documentation-only. This +docstring is written for EllPHi rather than copied from either revision: it +adopts the ebe63a4 remark that zero weights give zero gradient blocks, but +not its characterization of differentiability. EllPHi adds the offset +gradient ``d alpha / d delta_i = mu_i``; the original center and matrix +gradient code is unchanged. Distributed as part of EllPHi under its MIT +license; the ellcech source is also distributed under its MIT license. + +This numerical engine is internal. The public interface is :mod:`ellphi.cech`. + +Formula +------- +Given a :class:`~ellphi._minimax_python.MinimaxResult` with weights ``mu`` and +circumcenter ``x*``, the envelope theorem applied to the dual gives + + d alpha / d xbar_i = 2 mu_i A_i (xbar_i - x*) + d alpha / d A_i = mu_i outer(xbar_i - x*, xbar_i - x*) + d alpha / d delta_i = mu_i + +with centers ``xbar_i`` and matrices ``A_i`` treated as independent +parameters. For ``k = 2`` these are the pairwise gradients of ``t ** 2``. +Indices with zero weight get zero blocks; a zero weight alone is not +evidence that ``alpha`` fails to be differentiable. + +Hypotheses +---------- +The returned arrays are the formula evaluated at the solver output. They +approximate the true gradient of ``alpha`` only under the following +hypotheses: + +* the input configuration is non-degenerate; +* at least two indices are active; +* the returned weights are close to the optimal multipliers and are + supported on the active set; +* the weighted linear solve for ``x*`` is exact. A floating-point solve + leaves an additional residual. + +"Active" in these hypotheses refers to the exact optimization problem, not +to the thresholded ``result.active_set``. + +``MinimaxResult.converged`` and the value of ``alpha`` do not certify these +hypotheses. The formula scales each block by ``result.weights[i]`` directly; +it does not use ``result.active_set`` or ``weight_tol``. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Sequence + +import numpy as np + +from ._minimax_python import MinimaxResult + +__all__ = [ + "GradientResult", + "compute_gradient", +] + + +@dataclass +class GradientResult: + """Gradient of ``alpha`` with respect to centers and metric matrices. + + Attributes: + d_xbar: Gradients with respect to the centers ``xbar_i``, one array + of shape ``(d,)`` per vertex. + d_A: Gradients with respect to the matrices ``A_i``, one symmetric + array of shape ``(d, d)`` per vertex. + d_offsets: Gradient with respect to additive offsets ``delta_i``, + shape ``(k,)``. + alpha: The filtration value from the forward pass. + circumcenter: The circumcenter ``x*`` from the forward pass. + weights: The weights ``mu`` from the forward pass. + """ + + d_xbar: list[np.ndarray] + d_A: list[np.ndarray] + d_offsets: np.ndarray + alpha: float + circumcenter: np.ndarray + weights: np.ndarray + + +def compute_gradient( + result: MinimaxResult, + centers: np.ndarray | Sequence[np.ndarray], + matrices: np.ndarray | Sequence[np.ndarray], +) -> GradientResult: + """Evaluate the envelope-theorem gradient of ``alpha`` at a solver result. + + Computes ``d alpha / d xbar_i = 2 mu_i A_i (xbar_i - x*)``, + ``d alpha / d A_i = mu_i outer(xbar_i - x*, xbar_i - x*)``, and + ``d alpha / d delta_i = mu_i``. See the module docstring for the hypotheses + under which these approximate the true gradient. + + Args: + result: A MinimaxResult from + :func:`ellphi._minimax_python.solve_minimax` + computed for the same ``centers`` and ``matrices``. + centers: Centers ``xbar_i``, shape ``(k, d)``. + matrices: SPD matrices ``A_i``, shape ``(k, d, d)``. + + Returns: + GradientResult with ``d_xbar`` and ``d_A`` lists of length ``k`` and + ``d_offsets`` of shape ``(k,)``. + + Raises: + ValueError: If the shapes of ``centers``, ``matrices`` and + ``result.weights`` are inconsistent. + """ + centers = np.asarray(centers, dtype=float) + matrices = np.asarray(matrices, dtype=float) + + if centers.ndim == 1: + centers = centers[np.newaxis] + matrices = matrices[np.newaxis] + + k, d = centers.shape + if matrices.shape != (k, d, d): + raise ValueError( + f"matrices shape {matrices.shape} inconsistent with " + f"centers shape {centers.shape}" + ) + if len(result.weights) != k: + raise ValueError( + f"result.weights length {len(result.weights)} does not match k={k}" + ) + + xstar = result.circumcenter # shape (d,) + mu = result.weights # shape (k,) + + d_xbar = [] + d_A = [] + + for i in range(k): + diff = centers[i] - xstar # xbar_i - x*, shape (d,) + mu_i = float(mu[i]) + + # d alpha / d xbar_i = 2 mu_i A_i (xbar_i - x*) + grad_xbar_i = 2.0 * mu_i * (matrices[i] @ diff) # shape (d,) + + # d alpha / d A_i = mu_i outer(xbar_i - x*, xbar_i - x*) + grad_A_i = mu_i * np.outer(diff, diff) # shape (d, d) + + d_xbar.append(grad_xbar_i) + d_A.append(grad_A_i) + + return GradientResult( + d_xbar=d_xbar, + d_A=d_A, + d_offsets=mu.copy(), + alpha=result.alpha, + circumcenter=xstar.copy(), + weights=mu.copy(), + ) diff --git a/src/ellphi/_minimax_grad_python.pyi b/src/ellphi/_minimax_grad_python.pyi new file mode 100644 index 0000000..5249e81 --- /dev/null +++ b/src/ellphi/_minimax_grad_python.pyi @@ -0,0 +1,26 @@ +from dataclasses import dataclass +from typing import Sequence + +import numpy as np + +from ._minimax_python import MinimaxResult + +__all__ = [ + "GradientResult", + "compute_gradient", +] + +@dataclass +class GradientResult: + d_xbar: list[np.ndarray] + d_A: list[np.ndarray] + d_offsets: np.ndarray + alpha: float + circumcenter: np.ndarray + weights: np.ndarray + +def compute_gradient( + result: MinimaxResult, + centers: np.ndarray | Sequence[np.ndarray], + matrices: np.ndarray | Sequence[np.ndarray], +) -> GradientResult: ... diff --git a/src/ellphi/_minimax_python.py b/src/ellphi/_minimax_python.py new file mode 100644 index 0000000..1ec6d46 --- /dev/null +++ b/src/ellphi/_minimax_python.py @@ -0,0 +1,2063 @@ +"""Many-body minimax solver for anisotropic filtration values. + +Provenance +---------- +Adapted from uda-lab/ellcech ``src/ellphi_alpha/minimax.py`` at commit +82d13e3e174f4903cdcd6adcee1f469348361c91, by Tomoki Uda (the author of +EllPHi). That file is byte-identical at ellcech ``main`` +893056409db56865b7432274562db2d863e0abce. This engine deliberately deviates +from ellcech 82d13e3 in the following ways: + +* Armijo-rejected Newton steps are never applied. The best iterate by dual + value is retained, the status is ``"armijo_rejected"``, and + ``converged=False``. +* After an accepted warm-start Newton step, the weight-thresholded face is + re-identified before the next step; the cold-start baseline keeps its full + initial face during Newton. +* Undamped Newton polishing stops when the reduced Hessian is ill-conditioned + beyond the configured limit, with status ``"ill_conditioned_hessian"``. + Damped Newton uses diagonal regularization at that threshold instead. +* ``newton-cold`` starts at the uniform weights. If Newton does not converge, + it reports the Newton status without running a second method. +* Newton failure statuses force ``converged=False``; these statuses are + ``"empty_face"``, ``"armijo_rejected"``, ``"factorization_failed"``, + ``"ill_conditioned_hessian"``, ``"linear_solve_failed"``, + ``"nonfinite_residual"``, ``"nonfinite_step"``, and + ``"projection_failed"``. A full Newton step may additionally be accepted + for roundoff when + ``g_trial + 8 * eps * max(1, abs(g_current), abs(g_trial), + abs(armijo_target)) >= armijo_target``; this allowance applies only to the + unbacktracked full step. +* The objective supports additive per-constraint offsets, including general + packed conic constants. +* For stabilization, the SLSQP and damped-Newton objectives are the + regularized dual rather than ``dot(mu, f)``; this keeps their objective and + Jacobian consistent with the conditioned matrix. +* Under ``condition_number_limit``, the Newton Hessian omits the derivative + of the mu-dependent epsilon-floor shift, so the Newton step uses an + approximate Hessian. The residual test uses the exact stabilised gradient; + this can increase the step count without changing the accepted result. +* A post-polish Cholesky factorization failure forces ``converged=False``; + ellcech 82d13e3 preserved the prior convergence flag in this case. +* Direct ``scipy-slsqp`` is gap-enforced and Newton-polished when its initial + result misses the requested global gap. +* The default method is ``fw+brentq+newton``; the source uses a different + default dispatch. +* Frank-Wolfe gap convergence uses ``gap <= tol * max(1, abs(dual_value))``; + ``gap = max_i gradient_i - dot(mu, gradient)`` and ``dual_value`` is the + stabilized dual value when conditioning or regularization is active; + ellcech uses an absolute gap instead. +* The ``fw+brentq`` line search uses Brent with a bisection safeguard when the + bracket is rejected, and records each safeguard in its metadata. +The list above exhausts numerical and solver-behavior deviations from ellcech. +The remaining differences are packaging-only (the intra-package import of +``unpack_conic`` and type annotations) or documentation. Distributed as part +of EllPHi under its MIT license; the ellcech source is also distributed under +its MIT license. + +This numerical engine is internal. The public interface is :mod:`ellphi.cech`. + +Mathematical background +----------------------- +Given vertices ``i = 0, ..., k-1`` with centers ``x_i`` in R^d and SPD +matrices ``A_i`` and offsets ``delta_i``, the exact filtration value is + + alpha = min_x max_i f_i(x), + f_i(x) = (x - x_i)^T A_i (x - x_i) + delta_i. + +``alpha`` is a *squared* scale. In the pairwise case ``k = 2`` the exact +value equals ``ellphi.tangency(p, q).t ** 2``; the tangency time is +``sqrt(alpha)``. Every solver output is a finite-precision, finite-iteration +approximation of these mathematical quantities (within the tolerances and +subject to ``converged``). An *unadjusted* solve (``regularization == 0`` +and ``condition_number_limit is None``) approximates the exact problem +directly; passing ``condition_number_limit`` always applies at least a +machine-epsilon diagonal shift, and any positive ``regularization`` shifts +``A(mu)``, so adjusted solves approximate a stabilised problem instead. + +By strong duality, ``alpha = max_{mu in Delta^{k-1}} g(mu)`` with + + A(mu) = sum_i mu_i A_i + b(mu) = sum_i mu_i A_i x_i + g(mu) = sum_i mu_i (x_i^T A_i x_i + delta_i) + - b(mu)^T A(mu)^{-1} b(mu) + = sum_i mu_i f_i(x*(mu)), + +where ``x*(mu) = A(mu)^{-1} b(mu)`` is the circumcenter. The dual gradient is +``dg/dmu_i = f_i(x*(mu))`` and the Hessian is +``d2g/dmu_i dmu_j = -2 (x* - x_i)^T A_i A(mu)^{-1} A_j (x* - x_j)``. +These identities describe the unadjusted problem; with regularization or +conditioning the solver works on a stabilised ``A(mu)`` and its outputs +approximate that stabilised problem. + +The FW runners use ellcech's original pairwise Frank-Wolfe step on the dual +simplex: ``s`` is the best vertex (largest constraint value), ``v`` is the +worst active vertex (smallest constraint value), and mass is transferred along +``e_s - e_v`` by ``0 <= gamma <= mu[v]``. This pairwise step is not a +deviation from ellcech and is therefore not listed above. + +Weight support versus tight set +------------------------------- +``MinimaxResult.active_set`` is the *weight support* +``{i : weights[i] > weight_tol}``. It is not the tight constraint set +``{i : f_i(circumcenter) == alpha}``. At an exact optimum the support of the +weights lies inside the tight set, but the tight set can be strictly larger: +for three unit balls centered at ``(0, 0)``, ``(2, 0)`` and ``(0, 2)``, all +three constraints are tight at the circumcenter ``(1, 1)`` with +``alpha = 2``, while the unique optimal weights are ``(0, 1/2, 1/2)``. + +Value versus derivative reliability +----------------------------------- +``MinimaxResult.converged`` reports whether the chosen method met its own +stopping test for the value. It is not a certificate that ``alpha`` is +differentiable at the input or that the returned weights are accurate enough +for derivatives; see :mod:`ellphi._minimax_grad_python` for the hypotheses under +which the gradient formula applies. + +Correspondence with the pairwise solver +--------------------------------------- +For ``k = 2`` the pairwise solver finds the root of +``F(mu) = f_i(x*(mu)) - f_j(x*(mu))``, while this module maximises ``g`` +over ``Delta^1``; both give the same ``x*`` and ``dg/dmu = -F(mu)``. The +pairwise scalar ``mu`` and the vector weights here are related primitives +with different result contracts. + +Packed coefficients store the linear term as ``b = -A x_bar``, so the center +is ``x_bar = -A^{-1} b`` and ``delta = c - x_bar^T A x_bar``; +:func:`solve_minimax_from_coefs` handles both conversions. +""" + +from __future__ import annotations + +import warnings +from typing import Any, Literal, NamedTuple + +import numpy as np +import numpy.typing as npt +from scipy import linalg + +from .geometry import unpack_conic + +__all__ = [ + "MethodName", + "MinimaxResult", + "solve_minimax", + "solve_minimax_from_coefs", +] + +MethodName = Literal[ + "fw+bisect", + "fw+brentq", + "fw+bisect+newton", + "fw+brentq+newton", + "fw+bisect+damped-newton", + "scipy-slsqp", + "newton-cold", +] + +_DEFAULT_TOL = 1e-9 +_DEFAULT_MAX_ITER = 2000 +_DEFAULT_WEIGHT_TOL = 1e-10 +_DEFAULT_MAX_COND_STEPS = 8 +_N_BISECT = 52 # ~machine precision for double + +_NEWTON_TOL = 1e-14 +_NEWTON_MAX_ITER = 20 +_NEWTON_HESSIAN_COND_LIMIT = 1e12 + +# Brentq line search constants (analogous to ellphi _DEFAULT_HYBRID_BRACKET_MAXITER) +_DEFAULT_BRENTQ_MAXITER = 28 +_BRENTQ_XTOL = 1e-12 +_BRENTQ_RTOL = 4.0 * np.finfo(float).eps + +# Armijo backtracking constants +_ARMIJO_BETA = 0.5 +_ARMIJO_SIGMA = 1e-4 +_ARMIJO_MAX_BACKTRACK = 10 + +_VALID_METHODS: tuple[str, ...] = ( + "fw+bisect", + "fw+brentq", + "fw+bisect+newton", + "fw+brentq+newton", + "fw+bisect+damped-newton", + "scipy-slsqp", + "newton-cold", +) + +# Legacy alias +_METHOD_ALIASES: dict[str, str] = { + "fw+newton": "fw+bisect+newton", +} + + +class MinimaxResult(NamedTuple): + """Result of the minimax solver. + + Attributes: + alpha: Filtration value ``max_i f_i(circumcenter)``, a squared scale. + A numerical approximation of the exact minimax value within the + solver tolerances (see ``converged``). For an unadjusted solve + (``regularization == 0`` and ``condition_number_limit is None``) + the target is the exact problem; otherwise it is the stabilised + problem. In the pairwise case the exact value is + ``ellphi.tangency(p, q).t ** 2``. + circumcenter: Point ``x* = A(mu)^{-1} b(mu)`` for the returned + weights, computed from the unadjusted ``A(mu)`` for an unadjusted + solve and from the stabilised ``A(mu)`` otherwise. + weights: Returned dual weights ``mu`` in the simplex, shape ``(k,)``. + active_set: Weight support ``{i : weights[i] > weight_tol}``. This is + not the tight constraint set ``{i : f_i(x*) == alpha}``, which can + be strictly larger (see the module docstring). + converged: Whether the method met its stopping test for the value. + It is not a derivative certificate. + n_iter: Number of iterations performed. + method: Canonical solver method used (e.g. ``"fw+bisect"``). + metadata: Optional dict with method-specific diagnostics + (``fw_iters``, ``newton_iters``, ``hessian_cond``, and line-search + safeguard counts where applicable). + """ + + alpha: float + circumcenter: np.ndarray + weights: np.ndarray + active_set: list[int] + converged: bool + n_iter: int + method: str = "fw+bisect" + metadata: dict[str, Any] | None = None + + +# --------------------------------------------------------------------------- +# Low-level helpers (shared across methods) +# --------------------------------------------------------------------------- + + +def _condition_matrix( + A: np.ndarray, + *, + regularization: float, + condition_number_limit: float | None, + max_conditioning_steps: int, +) -> np.ndarray: + """Optionally regularise A to improve conditioning.""" + if regularization < 0.0: + raise ValueError("regularization must be non-negative") + if condition_number_limit is not None and condition_number_limit <= 1.0: + raise ValueError("condition_number_limit must be > 1 when provided") + if max_conditioning_steps < 0: + raise ValueError("max_conditioning_steps must be non-negative") + + if regularization == 0.0 and condition_number_limit is None: + return A + + eye = np.eye(A.shape[0], dtype=A.dtype) + if condition_number_limit is None: + return A + regularization * eye + + spectral_scale = float(np.linalg.norm(A, ord=2)) + eps_floor = np.finfo(A.dtype).eps * max(1.0, spectral_scale) + reg = max(regularization, eps_floor) + A_reg = A + reg * eye + + cond = np.linalg.cond(A_reg) + if np.isfinite(cond) and cond <= condition_number_limit: + return A_reg + + # Strict cap: perform at most max_conditioning_steps escalations. + for _ in range(max_conditioning_steps): + reg *= 10.0 + A_reg = A + reg * eye + cond = np.linalg.cond(A_reg) + if np.isfinite(cond) and cond <= condition_number_limit: + return A_reg + return A_reg + + +def _conditioning_shift_gradient( + mu: np.ndarray, + matrices: np.ndarray, + A_mu: np.ndarray, + *, + regularization: float, + condition_number_limit: float | None, + max_conditioning_steps: int, +) -> np.ndarray: + """Differentiate the active scalar conditioning shift with respect to mu.""" + if condition_number_limit is None: + return np.zeros(mu.shape, dtype=float) + + spectral_scale = float(np.linalg.norm(A_mu, ord=2)) + if spectral_scale < 1.0: + return np.zeros(mu.shape, dtype=float) + eps_floor = np.finfo(A_mu.dtype).eps * spectral_scale + base_shift_from_floor = regularization < eps_floor + if not base_shift_from_floor: + base_shift = regularization + base_gradient = np.zeros(mu.shape, dtype=float) + else: + base_shift = eps_floor + base_gradient = np.empty(mu.shape, dtype=float) + A_used = _condition_matrix( + A_mu, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + effective_shift = float(np.trace(A_used - A_mu) / A_mu.shape[0]) + escalation = effective_shift / base_shift + _, eigenvectors = np.linalg.eigh(A_mu) + leading_vector = eigenvectors[:, -1] + if base_shift_from_floor: + spectral_gradient = np.einsum( + "i,kij,j->k", leading_vector, matrices, leading_vector + ) + base_gradient = np.finfo(A_mu.dtype).eps * spectral_gradient + if spectral_scale == 1.0: + base_gradient *= 0.5 + return escalation * base_gradient + + +def _validate_tolerance( + name: str, value: float, *, strictly_positive: bool = False +) -> None: + """Validate a finite numerical tolerance or regularization value.""" + if not np.isfinite(value) or (value <= 0.0 if strictly_positive else value < 0.0): + qualifier = "finite and > 0" if strictly_positive else "finite and non-negative" + raise ValueError(f"{name} must be {qualifier}") + + +def _validate_positive_int(name: str, value: int) -> None: + """Validate a positive integer solver parameter.""" + if ( + isinstance(value, (bool, np.bool_)) + or not isinstance(value, (int, np.integer)) + or value <= 0 + ): + raise ValueError(f"{name} must be a positive integer") + + +def _validate_non_negative_int(name: str, value: int) -> None: + """Validate a non-negative integer solver parameter.""" + if ( + isinstance(value, (bool, np.bool_)) + or not isinstance(value, (int, np.integer)) + or value < 0 + ): + raise ValueError(f"{name} must be a non-negative integer") + + +def _cholesky_solve( + A: np.ndarray, + b: np.ndarray, + *, + regularization: float = 0.0, + condition_number_limit: float | None = None, + max_conditioning_steps: int = _DEFAULT_MAX_COND_STEPS, +) -> np.ndarray: + """Solve A x = b with optional conditioning safeguards.""" + A = _condition_matrix( + A, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + chol = linalg.cho_factor(A, check_finite=False) + return linalg.cho_solve(chol, b, check_finite=False) + + +def _exact_linear_solve(A: np.ndarray, b: np.ndarray) -> np.ndarray: + """Solve A x = b without stability regularization adjustments.""" + try: + chol = linalg.cho_factor(A, check_finite=False) + return linalg.cho_solve(chol, b, check_finite=False) + except linalg.LinAlgError: + return np.linalg.solve(A, b) + + +def _eval_f( + mu: np.ndarray, + matrices: np.ndarray, + Ax: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray | None, + *, + regularization: float, + condition_number_limit: float | None, + max_conditioning_steps: int, +) -> tuple[np.ndarray, np.ndarray]: + """Compute circumcenter x*(mu) and f_i(x*(mu)) for all i. + + For an unadjusted solve, ``g(mu) = sum_i mu_i f_i(x*(mu))``. Stabilized + solves use the regularized dual value for the scale in the gap test. + """ + A_mu = np.einsum("k,kij->ij", mu, matrices) + b_mu = np.einsum("k,ki->i", mu, Ax) + xstar = _cholesky_solve( + A_mu, + b_mu, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + diff = xstar[np.newaxis, :] - centers + f = np.einsum("ki,kij,kj->k", diff, matrices, diff) + if offsets is not None: + f = f + offsets + return xstar, f + + +def _dual_gradient( + mu: np.ndarray, + matrices: np.ndarray, + Ax: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray | None, + xstar: np.ndarray, + f: np.ndarray, + *, + regularization: float, + condition_number_limit: float | None, + max_conditioning_steps: int, +) -> np.ndarray: + """Return the gradient of the dual represented by ``xstar`` and ``f``.""" + if regularization == 0.0 and condition_number_limit is None: + return f + + A_mu = np.einsum("k,kij->ij", mu, matrices) + shift_gradient = _conditioning_shift_gradient( + mu, + matrices, + A_mu, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + return f + shift_gradient * float(np.dot(xstar, xstar)) + + +def _fw_gap_and_scale( + mu: np.ndarray, + gradient: np.ndarray, + *, + dual_value: float | None = None, +) -> tuple[float, float]: + """Return the Frank-Wolfe gap and its relative dual-value scale.""" + gradient_value = float(np.dot(mu, gradient)) + if dual_value is None: + dual_value = gradient_value + gap = float(np.max(gradient) - gradient_value) + scale = ( + float(max(1.0, abs(dual_value))) if np.isfinite(dual_value) else float("nan") + ) + return gap, scale + + +def _fw_gap_converged( + mu: np.ndarray, + gradient: np.ndarray, + tol: float, + *, + dual_value: float | None = None, +) -> bool: + """Return whether the relative Frank-Wolfe gap meets ``tol``.""" + gap, scale = _fw_gap_and_scale(mu, gradient, dual_value=dual_value) + return bool(np.isfinite(gap) and np.isfinite(scale) and gap <= tol * scale) + + +def _regularized_dual_value( + mu: np.ndarray, + matrices: np.ndarray, + Ax: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray | None, + xstar: np.ndarray, +) -> float: + """Evaluate the dual value for the matrix used to obtain ``xstar``.""" + c = np.einsum("ki,kij,kj->k", centers, matrices, centers) + if offsets is not None: + c = c + offsets + b_mu = np.einsum("k,ki->i", mu, Ax) + return float(np.dot(mu, c) - np.dot(b_mu, xstar)) + + +def _exact_line_search( + s: int, + v: int, + mu: np.ndarray, + matrices: np.ndarray, + Ax: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray | None, + gamma_max: float, + *, + regularization: float, + condition_number_limit: float | None, + max_conditioning_steps: int, +) -> float: + """Maximise ``g`` along a dual-simplex pairwise Frank-Wolfe swap. + + The direction is ``e_s - e_v``, where ``s`` is the best vertex (largest + constraint value) and ``v`` is the worst active vertex (smallest + constraint value). The line search is restricted to + ``0 <= gamma <= gamma_max <= mu[v]``. + + Bisects on the stabilized dual directional derivative + ``h(gamma) = grad_s - grad_v``. + """ + if gamma_max <= 0.0: + return 0.0 + + def h(gamma: float) -> float: + mu_g = mu.copy() + mu_g[s] += gamma + mu_g[v] -= gamma + x_g, f_g = _eval_f( + mu_g, + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + gradient_g = _dual_gradient( + mu_g, + matrices, + Ax, + centers, + offsets, + x_g, + f_g, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + return float(gradient_g[s] - gradient_g[v]) + + h0 = h(0.0) + if h0 <= 0.0: + return 0.0 # already at or past optimum in this direction + + h_max = h(gamma_max) + if h_max >= 0.0: + return gamma_max # optimum is at the boundary + + lo, hi = 0.0, gamma_max + for _ in range(_N_BISECT): + mid = 0.5 * (lo + hi) + if h(mid) > 0.0: + lo = mid + else: + hi = mid + return 0.5 * (lo + hi) + + +def _ensure_finite(value: float, label: str) -> float: + """Raise RuntimeError if *value* is NaN or Inf (ellphi pattern).""" + if not np.isfinite(value): + raise RuntimeError(f"Non-finite {label} value: {value}") + return value + + +def _brentq_line_search( + s: int, + v: int, + mu: np.ndarray, + matrices: np.ndarray, + Ax: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray | None, + gamma_max: float, + *, + regularization: float, + condition_number_limit: float | None, + max_conditioning_steps: int, +) -> tuple[float, int, bool]: + """Brentq line search for a dual-simplex pairwise Frank-Wolfe swap. + + The direction is ``e_s - e_v``, with ``s`` the best vertex (largest + constraint value) and ``v`` the worst active vertex (smallest constraint + value). The line search is restricted to + ``0 <= gamma <= gamma_max <= mu[v]``. + + Returns: + (gamma, n_fevals, used_bisection): Optimal step, function evaluations, + and whether the declared bisection safeguard was used. + """ + if gamma_max <= 0.0: + return 0.0, 0, False + + n_fevals = [0] + + def h(gamma: float) -> float: + n_fevals[0] += 1 + mu_g = mu.copy() + mu_g[s] += gamma + mu_g[v] -= gamma + x_g, f_g = _eval_f( + mu_g, + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + gradient_g = _dual_gradient( + mu_g, + matrices, + Ax, + centers, + offsets, + x_g, + f_g, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + return float(gradient_g[s] - gradient_g[v]) + + h0 = h(0.0) + if h0 <= 0.0: + return 0.0, n_fevals[0], False + + h_max = h(gamma_max) + if h_max >= 0.0: + return gamma_max, n_fevals[0], False + + try: + from scipy.optimize import brentq + + gamma = brentq( + h, + 0.0, + gamma_max, + xtol=_BRENTQ_XTOL, + rtol=_BRENTQ_RTOL, + maxiter=_DEFAULT_BRENTQ_MAXITER, + ) + return float(gamma), n_fevals[0], False + except (ValueError, RuntimeError): + # Declared safeguard for a rejected Brent bracket. + lo, hi = 0.0, gamma_max + for _ in range(_N_BISECT): + mid = 0.5 * (lo + hi) + if h(mid) > 0.0: + lo = mid + else: + hi = mid + return 0.5 * (lo + hi), n_fevals[0], True + + +# --------------------------------------------------------------------------- +# Per-method internal runners +# --------------------------------------------------------------------------- + + +def _run_fw_bisect( + matrices: np.ndarray, + Ax: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray | None, + mu: np.ndarray, + *, + tol: float, + max_iter: int, + weight_tol: float, + regularization: float, + condition_number_limit: float | None, + max_conditioning_steps: int, +) -> tuple[np.ndarray, bool, int]: + """Pairwise Frank-Wolfe with exact bisection line search. + + Each swap chooses the best vertex (largest constraint value) and the + worst active vertex (smallest constraint value), then transfers mass along + ``e_s - e_v`` with step at most ``mu[v]``. + + Returns: + (mu, converged, n_iter) + """ + converged = False + n_iter = 0 + + for n_iter in range(1, max_iter + 1): + xstar, f = _eval_f( + mu, + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + + gradient = _dual_gradient( + mu, + matrices, + Ax, + centers, + offsets, + xstar, + f, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + + s = int(np.argmax(gradient)) + active_mask = mu > weight_tol + if not np.any(active_mask): + active_mask = np.ones_like(mu, dtype=bool) + gradient_active = np.where(active_mask, gradient, np.inf) + v = int(np.argmin(gradient_active)) + if s == v: + positive_mask = mu > 0.0 + positive_mask[s] = False + if np.any(positive_mask): + v = int(np.argmin(np.where(positive_mask, gradient, np.inf))) + + dual_value = ( + float(np.dot(mu, f)) + if regularization == 0.0 and condition_number_limit is None + else _regularized_dual_value(mu, matrices, Ax, centers, offsets, xstar) + ) + if _fw_gap_converged(mu, gradient, tol, dual_value=dual_value): + converged = True + break + + gamma_max = float(mu[v]) + gamma = _exact_line_search( + s, + v, + mu, + matrices, + Ax, + centers, + offsets, + gamma_max, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + + mu = mu.copy() + mu[s] += gamma + mu[v] -= gamma + mu = np.clip(mu, 0.0, None) + mu /= mu.sum() + + if not converged: + xstar, f = _eval_f( + mu, + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + gradient = _dual_gradient( + mu, + matrices, + Ax, + centers, + offsets, + xstar, + f, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + dual_value = ( + float(np.dot(mu, f)) + if regularization == 0.0 and condition_number_limit is None + else _regularized_dual_value(mu, matrices, Ax, centers, offsets, xstar) + ) + converged = _fw_gap_converged(mu, gradient, tol, dual_value=dual_value) + + return mu, converged, n_iter + + +def _run_fw_brentq( + matrices: np.ndarray, + Ax: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray | None, + mu: np.ndarray, + *, + tol: float, + max_iter: int, + weight_tol: float, + regularization: float, + condition_number_limit: float | None, + max_conditioning_steps: int, +) -> tuple[np.ndarray, bool, int, dict]: + """Pairwise Frank-Wolfe with brentq line search. + + Each swap chooses the best vertex (largest constraint value) and the + worst active vertex (smallest constraint value), then transfers mass along + ``e_s - e_v`` with step at most ``mu[v]``. + + Returns: + (mu, converged, n_iter, metadata) + """ + converged = False + n_iter = 0 + total_line_search_evals = 0 + total_fevals = 0 + line_search_bisection_safeguards = 0 + + for n_iter in range(1, max_iter + 1): + xstar, f = _eval_f( + mu, + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + total_fevals += 1 + + gradient = _dual_gradient( + mu, + matrices, + Ax, + centers, + offsets, + xstar, + f, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + + s = int(np.argmax(gradient)) + active_mask = mu > weight_tol + if not np.any(active_mask): + active_mask = np.ones_like(mu, dtype=bool) + gradient_active = np.where(active_mask, gradient, np.inf) + v = int(np.argmin(gradient_active)) + if s == v: + positive_mask = mu > 0.0 + positive_mask[s] = False + if np.any(positive_mask): + v = int(np.argmin(np.where(positive_mask, gradient, np.inf))) + + dual_value = ( + float(np.dot(mu, f)) + if regularization == 0.0 and condition_number_limit is None + else _regularized_dual_value(mu, matrices, Ax, centers, offsets, xstar) + ) + if _fw_gap_converged(mu, gradient, tol, dual_value=dual_value): + converged = True + break + + gamma_max = float(mu[v]) + gamma, ls_evals, used_bisection = _brentq_line_search( + s, + v, + mu, + matrices, + Ax, + centers, + offsets, + gamma_max, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + total_line_search_evals += ls_evals + line_search_bisection_safeguards += int(used_bisection) + + mu = mu.copy() + mu[s] += gamma + mu[v] -= gamma + mu = np.clip(mu, 0.0, None) + mu /= mu.sum() + + if not converged: + xstar, f = _eval_f( + mu, + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + gradient = _dual_gradient( + mu, + matrices, + Ax, + centers, + offsets, + xstar, + f, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + dual_value = ( + float(np.dot(mu, f)) + if regularization == 0.0 and condition_number_limit is None + else _regularized_dual_value(mu, matrices, Ax, centers, offsets, xstar) + ) + converged = _fw_gap_converged(mu, gradient, tol, dual_value=dual_value) + + metadata = { + "fw_iters": n_iter, + "n_fevals": total_fevals + total_line_search_evals, + "line_search_evals": total_line_search_evals, + "line_search_bisection_safeguards": line_search_bisection_safeguards, + } + return mu, converged, n_iter, metadata + + +def _hessian_g_negative( + active: list[int], + xstar: np.ndarray, + matrices: np.ndarray, + Amu_inv: np.ndarray, + centers: np.ndarray, +) -> np.ndarray: + """Negative Hessian of g on the active face (positive semidefinite). + + neg_H[l, l'] = 2 (x* - x_{i_l})^T A_{i_l} A(mu)^{-1} A_{i_{l'}} (x* - x_{i_{l'}}) + + Equivalently neg_H = 2 * Ad @ Amu_inv @ Ad.T + where Ad[l] = A_{i_l} (x* - x_{i_l}) (shape m x d). + """ + diff = xstar[np.newaxis, :] - centers[active] # (m, d) + Ad = np.einsum("lij,lj->li", matrices[active], diff) # (m, d) + return 2.0 * (Ad @ Amu_inv @ Ad.T) # (m, m) + + +def _newton_polish( + mu: np.ndarray, + active_set: list[int], + matrices: np.ndarray, + Ax: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray | None, + *, + weight_tol: float = _DEFAULT_WEIGHT_TOL, + max_iter: int = _NEWTON_MAX_ITER, + tol: float = _NEWTON_TOL, + regularization: float = 0.0, + condition_number_limit: float | None = None, + max_conditioning_steps: int = _DEFAULT_MAX_COND_STEPS, +) -> tuple[np.ndarray, int, dict]: + """Newton polishing on the dual restricted to the current active face. + + Solves the KKT conditions using the stabilised gradient via the reduced + system: + + neg_H_r delta = r + neg_H_r[l, l'] = neg_H[l,l'] - neg_H[l, m-1] - neg_H[m-1, l'] + neg_H[m-1, m-1] + + When ``condition_number_limit`` is set, the epsilon-floor shift depends on + ``mu`` through ``||A(mu)||_2``. The Hessian above omits the derivative of + that shift, so this is a quasi-Newton step with an approximate Hessian; + the residual test still uses the exact stabilised gradient. The omitted + terms can increase the step count without changing the accepted result. + + Then mu_{i_l} += delta[l], mu_{i_{m-1}} -= sum(delta), + followed by simplex projection. + + Returns: + Updated weights, number of Newton steps, and diagnostics. + """ + m = len(active_set) + if m == 0: + return ( + mu, + 0, + { + "newton_iters": 0, + "hessian_cond": 0.0, + "newton_status": "empty_face", + }, + ) + if m <= 1: + return ( + mu, + 0, + { + "newton_iters": 0, + "hessian_cond": 0.0, + "newton_status": "singleton_face", + }, + ) + + mu = mu.copy() + n_iter = 0 + max_hessian_cond = 0.0 + newton_status = "max_iter" + + for n_iter in range(1, max_iter + 1): + m = len(active_set) + if m == 0: + newton_status = "empty_face" + break + if m <= 1: + newton_status = "singleton_face" + break + + A_mu = np.einsum("k,kij->ij", mu, matrices) + b_mu = np.einsum("k,ki->i", mu, Ax) + A_used = _condition_matrix( + A_mu, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + + try: + chol = linalg.cho_factor(A_used, check_finite=False) + xstar = linalg.cho_solve(chol, b_mu, check_finite=False) + d = A_used.shape[0] + Amu_inv = linalg.cho_solve(chol, np.eye(d), check_finite=False) + except linalg.LinAlgError: + newton_status = "factorization_failed" + break + + diff = xstar[np.newaxis, :] - centers[active_set] # (m, d) + f = np.einsum("ki,kij,kj->k", diff, matrices[active_set], diff) # (m,) + if offsets is not None: + f = f + offsets[active_set] + + if regularization == 0.0 and condition_number_limit is None: + # Keep the unregularized residual path bitwise identical. + r = f[:-1] - f[-1] + else: + diff_all = xstar[np.newaxis, :] - centers + f_all = np.einsum("ki,kij,kj->k", diff_all, matrices, diff_all) + if offsets is not None: + f_all = f_all + offsets + gradient = _dual_gradient( + mu, + matrices, + Ax, + centers, + offsets, + xstar, + f_all, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + r = gradient[active_set[:-1]] - gradient[active_set[-1]] + if float(np.max(np.abs(r))) < tol: + newton_status = "converged" + break + + neg_H = _hessian_g_negative( + active_set, xstar, matrices, Amu_inv, centers + ) # (m, m) + + # Reduced system (eliminate last variable via sum=1 constraint) + neg_H_r = ( + neg_H[:-1, :-1] - neg_H[:-1, -1:] - neg_H[-1:, :-1] + neg_H[-1, -1] + ) # (m-1, m-1) + + hess_cond = float(np.linalg.cond(neg_H_r)) + max_hessian_cond = max(max_hessian_cond, hess_cond) + if not np.isfinite(hess_cond) or hess_cond > _NEWTON_HESSIAN_COND_LIMIT: + newton_status = "ill_conditioned_hessian" + break + + try: + delta = np.linalg.solve(neg_H_r, r) + except np.linalg.LinAlgError: + newton_status = "linear_solve_failed" + break + + mu_new = mu.copy() + for pos, idx in enumerate(active_set[:-1]): + mu_new[idx] += delta[pos] + mu_new[active_set[-1]] -= float(delta.sum()) + + # Project back to simplex + mu_new = np.clip(mu_new, 0.0, None) + s = mu_new.sum() + if s <= 0.0: + newton_status = "projection_failed" + break + mu_new /= s + mu = mu_new + active_set = [i for i in range(len(mu)) if mu[i] > weight_tol] + + metadata = { + "newton_iters": n_iter, + "hessian_cond": max_hessian_cond, + "newton_status": newton_status, + } + return mu, n_iter, metadata + + +def _damped_newton_polish( + mu: np.ndarray, + active_set: list[int], + matrices: np.ndarray, + Ax: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray | None, + *, + weight_tol: float = _DEFAULT_WEIGHT_TOL, + reidentify_face: bool = True, + max_iter: int = _NEWTON_MAX_ITER, + tol: float = _NEWTON_TOL, + regularization: float = 0.0, + condition_number_limit: float | None = None, + max_conditioning_steps: int = _DEFAULT_MAX_COND_STEPS, +) -> tuple[np.ndarray, int, dict]: + """Newton polishing with Armijo backtracking and conditioning safeguards. + + Like ``_newton_polish`` but adds: + - Condition number check on the reduced Hessian (regularize if > 1e12) + - Armijo backtracking line search on the Newton step + - ``_ensure_finite`` guards against NaN propagation + + Under ``condition_number_limit``, the epsilon-floor shift depends on + ``mu`` through ``||A(mu)||_2``. The Newton Hessian omits its derivative, + making this a quasi-Newton step with an approximate Hessian; the residual + test uses the exact stabilised gradient. The omitted terms may increase + the step count without changing the accepted result. + + Returns: + (mu, n_iter, metadata) + """ + m = len(active_set) + if m == 0: + return ( + mu, + 0, + { + "newton_iters": 0, + "hessian_cond": 0.0, + "newton_status": "empty_face", + }, + ) + if m <= 1: + return ( + mu, + 0, + { + "newton_iters": 0, + "hessian_cond": 0.0, + "newton_status": "singleton_face", + }, + ) + + mu = mu.copy() + best_mu = mu.copy() + best_g = -np.inf + n_iter = 0 + max_hessian_cond = 0.0 + newton_status = "max_iter" + rejected_step_diagnostics: dict[str, float] = {} + roundoff_accepts = 0 + max_armijo_roundoff_shortfall = 0.0 + + for n_iter in range(1, max_iter + 1): + m = len(active_set) + if m == 0: + newton_status = "empty_face" + break + if m <= 1: + newton_status = "singleton_face" + break + + A_mu = np.einsum("k,kij->ij", mu, matrices) + b_mu = np.einsum("k,ki->i", mu, Ax) + A_used = _condition_matrix( + A_mu, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + + try: + chol = linalg.cho_factor(A_used, check_finite=False) + xstar = linalg.cho_solve(chol, b_mu, check_finite=False) + d = A_used.shape[0] + Amu_inv = linalg.cho_solve(chol, np.eye(d), check_finite=False) + except linalg.LinAlgError: + newton_status = "factorization_failed" + break + + diff = xstar[np.newaxis, :] - centers[active_set] + f = np.einsum("ki,kij,kj->k", diff, matrices[active_set], diff) + if offsets is not None: + f = f + offsets[active_set] + + if regularization == 0.0 and condition_number_limit is None: + # Keep the unregularized residual path bitwise identical. + r = f[:-1] - f[-1] + else: + diff_all = xstar[np.newaxis, :] - centers + f_all = np.einsum("ki,kij,kj->k", diff_all, matrices, diff_all) + if offsets is not None: + f_all = f_all + offsets + gradient = _dual_gradient( + mu, + matrices, + Ax, + centers, + offsets, + xstar, + f_all, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + r = gradient[active_set[:-1]] - gradient[active_set[-1]] + try: + _ensure_finite(float(np.max(np.abs(r))), "Newton residual") + except RuntimeError: + newton_status = "nonfinite_residual" + break + if float(np.max(np.abs(r))) < tol: + newton_status = "converged" + break + + neg_H = _hessian_g_negative(active_set, xstar, matrices, Amu_inv, centers) + + neg_H_r = neg_H[:-1, :-1] - neg_H[:-1, -1:] - neg_H[-1:, :-1] + neg_H[-1, -1] + + # Conditioning check + hess_cond = float(np.linalg.cond(neg_H_r)) + max_hessian_cond = max(max_hessian_cond, hess_cond) + if hess_cond > _NEWTON_HESSIAN_COND_LIMIT: + # Diagonal regularization + reg = float(np.trace(neg_H_r)) / neg_H_r.shape[0] * 1e-10 + reg = max(reg, 1e-14) + neg_H_r = neg_H_r + reg * np.eye(neg_H_r.shape[0]) + + try: + delta = np.linalg.solve(neg_H_r, r) + except np.linalg.LinAlgError: + newton_status = "linear_solve_failed" + break + + if not np.all(np.isfinite(delta)): + newton_status = "nonfinite_step" + break + + # Current objective: g(mu) = dot(mu, f_full) + _, f_full = _eval_f( + mu, + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + if regularization == 0.0 and condition_number_limit is None: + g_current = float(np.dot(mu, f_full)) + else: + g_current = _regularized_dual_value( + mu, matrices, Ax, centers, offsets, xstar + ) + if g_current > best_g: + best_g = g_current + best_mu = mu.copy() + + # Armijo backtracking + step = 1.0 + accepted = False + for backtrack in range(_ARMIJO_MAX_BACKTRACK): + mu_trial = mu.copy() + for pos, idx in enumerate(active_set[:-1]): + mu_trial[idx] += step * delta[pos] + mu_trial[active_set[-1]] -= step * float(delta.sum()) + + # Project to simplex + mu_trial = np.clip(mu_trial, 0.0, None) + s = mu_trial.sum() + if s <= 0.0: + step *= _ARMIJO_BETA + continue + mu_trial /= s + + xstar_trial, f_trial = _eval_f( + mu_trial, + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + if regularization == 0.0 and condition_number_limit is None: + g_trial = float(np.dot(mu_trial, f_trial)) + else: + g_trial = _regularized_dual_value( + mu_trial, matrices, Ax, centers, offsets, xstar_trial + ) + armijo_target = g_current + _ARMIJO_SIGMA * step * float(r @ delta) + armijo_roundoff = ( + 8.0 + * np.finfo(float).eps + * max(1.0, abs(g_current), abs(g_trial), abs(armijo_target)) + ) + if backtrack == 0: + rejected_step_diagnostics = { + "armijo_g_current": g_current, + "armijo_full_step_g": g_trial, + "armijo_full_step_target": armijo_target, + "armijo_residual": float(np.max(np.abs(r))), + } + + # Armijo sufficient increase (maximizing g) + roundoff_accepted = ( + backtrack == 0 and g_trial + armijo_roundoff >= armijo_target + ) + if g_trial >= armijo_target or roundoff_accepted: + if roundoff_accepted and g_trial < armijo_target: + roundoff_accepts += 1 + max_armijo_roundoff_shortfall = max( + max_armijo_roundoff_shortfall, armijo_target - g_trial + ) + mu = mu_trial + if g_trial > best_g: + best_g = g_trial + best_mu = mu.copy() + accepted = True + break + step *= _ARMIJO_BETA + + if not accepted: + newton_status = "armijo_rejected" + mu = best_mu + break + + if reidentify_face: + active_set = [i for i in range(len(mu)) if mu[i] > weight_tol] + + metadata = { + "newton_iters": n_iter, + "hessian_cond": max_hessian_cond, + "newton_status": newton_status, + "armijo_roundoff_accepts": roundoff_accepts, + "armijo_roundoff_shortfall": max_armijo_roundoff_shortfall, + } + if newton_status == "armijo_rejected": + metadata.update(rejected_step_diagnostics) + return mu, n_iter, metadata + + +def _simplex_residual(mu: npt.NDArray[np.float64]) -> float: + """Equality constraint ``sum(mu) - 1`` for SLSQP.""" + return float(mu.sum() - 1.0) + + +def _slsqp_objective_and_gradient( + mu: npt.NDArray[np.float64], + matrices: np.ndarray, + Ax: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray | None, + *, + regularization: float, + condition_number_limit: float | None, + max_conditioning_steps: int, +) -> tuple[float, np.ndarray]: + """Return SLSQP's negative dual objective and its negative gradient.""" + xstar, f = _eval_f( + mu, + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + gradient = _dual_gradient( + mu, + matrices, + Ax, + centers, + offsets, + xstar, + f, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + if regularization == 0.0 and condition_number_limit is None: + # Keep the unregularized objective path bitwise identical. + return -float(np.dot(mu, f)), -gradient + + # The stabilized dual is g_r(mu) = sum_i mu_i c_i - + # b(mu)^T A_used(mu)^(-1) b(mu). Its gradient includes the derivative + # of any conditioning shift that depends on A(mu). + g_regularized = _regularized_dual_value(mu, matrices, Ax, centers, offsets, xstar) + return -g_regularized, -gradient + + +def _run_scipy_slsqp( + matrices: np.ndarray, + Ax: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray | None, + k: int, + *, + tol: float, + regularization: float, + condition_number_limit: float | None, + max_conditioning_steps: int, +) -> tuple[np.ndarray, bool, int]: + """Maximize g(mu) over the simplex using scipy SLSQP. + + Returns: + (mu, converged, n_iter) + """ + from scipy.optimize import minimize + + n_eval = [0] + + def neg_g_and_grad(mu: npt.NDArray[np.float64]) -> tuple[float, np.ndarray]: + n_eval[0] += 1 + return _slsqp_objective_and_gradient( + mu, + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + + x0 = np.full(k, 1.0 / k) + bounds = [(0.0, None)] * k + + res = minimize( + neg_g_and_grad, + x0, + jac=True, + method="SLSQP", + bounds=bounds, + constraints={"type": "eq", "fun": _simplex_residual}, + options={"ftol": 1e-14, "maxiter": 500}, + ) + + mu = np.clip(res.x, 0.0, None) + s = mu.sum() + if s > 0.0: + mu /= s + + xstar, f = _eval_f( + mu, + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + gradient = _dual_gradient( + mu, + matrices, + Ax, + centers, + offsets, + xstar, + f, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + dual_value = ( + float(np.dot(mu, f)) + if regularization == 0.0 and condition_number_limit is None + else _regularized_dual_value(mu, matrices, Ax, centers, offsets, xstar) + ) + converged = bool(res.success) and _fw_gap_converged( + mu, gradient, tol, dual_value=dual_value + ) + return mu, converged, n_eval[0] + + +# --------------------------------------------------------------------------- +# Public API +# --------------------------------------------------------------------------- + + +def _check_newton_convergence( + mu: np.ndarray, + matrices: np.ndarray, + Ax: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray | None, + newton_status: str, + *, + tol: float, + regularization: float = 0.0, + condition_number_limit: float | None = None, + max_conditioning_steps: int = _DEFAULT_MAX_COND_STEPS, +) -> bool: + """Apply the shared gap contract after Newton polishing. + + Newton's residual tolerance controls its own iterations; it is not an + additional accepted-result criterion. Fatal Newton statuses remain + non-convergent even when the final global gap can be evaluated. + """ + failure_statuses = { + "empty_face", + "armijo_rejected", + "factorization_failed", + "ill_conditioned_hessian", + "linear_solve_failed", + "nonfinite_residual", + "nonfinite_step", + "projection_failed", + } + if newton_status in failure_statuses: + return False + try: + A_mu = np.einsum("k,kij->ij", mu, matrices) + b_mu = np.einsum("k,ki->i", mu, Ax) + A_used = _condition_matrix( + A_mu, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + chol = linalg.cho_factor(A_used, check_finite=False) + xstar_check = linalg.cho_solve(chol, b_mu, check_finite=False) + d_check = xstar_check[np.newaxis, :] - centers + f_check = np.einsum("ki,kij,kj->k", d_check, matrices, d_check) + if offsets is not None: + f_check = f_check + offsets + gradient_check = _dual_gradient( + mu, + matrices, + Ax, + centers, + offsets, + xstar_check, + f_check, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + dual_value = ( + float(np.dot(mu, f_check)) + if regularization == 0.0 and condition_number_limit is None + else _regularized_dual_value( + mu, matrices, Ax, centers, offsets, xstar_check + ) + ) + if not _fw_gap_converged(mu, gradient_check, tol, dual_value=dual_value): + return False + return True + except linalg.LinAlgError: + return False + + +def _solve_minimax( + matrices: np.ndarray, + centers: np.ndarray, + *, + offsets: np.ndarray | None = None, + method: MethodName | str = "fw+brentq+newton", + tol: float = _DEFAULT_TOL, + max_iter: int = _DEFAULT_MAX_ITER, + weight_tol: float = _DEFAULT_WEIGHT_TOL, + regularization: float = 0.0, + condition_number_limit: float | None = None, + max_conditioning_steps: int = _DEFAULT_MAX_COND_STEPS, + newton_tol: float = _NEWTON_TOL, + newton_max_iter: int = _NEWTON_MAX_ITER, +) -> MinimaxResult: + """Compute a numerical approximation of the filtration value. + + The mathematical target is ``alpha = max_{mu in Delta} g(mu)``. The + returned value and circumcenter are finite-iteration approximations + within the solver tolerances; check ``converged``. For an unadjusted + solve (``regularization == 0`` and ``condition_number_limit is None``) + the target is the exact problem. Otherwise ``A(mu)`` is stabilised + (``condition_number_limit`` alone already applies at least a + machine-epsilon diagonal shift) and the outputs approximate the + stabilised problem. + + Seven solver back-ends are available via ``method``: + + * ``"fw+bisect"``: Pairwise FW with 52-step bisection line search. + * ``"fw+brentq"``: Pairwise FW with adaptive brentq line search. + * ``"fw+bisect+newton"``: Pairwise FW(bisect) warm-start + Newton polishing. + * ``"fw+brentq+newton"``: Pairwise FW(brentq) warm-start + damped Newton polishing. + * ``"fw+bisect+damped-newton"``: Pairwise FW(bisect) + Armijo-damped Newton. + * ``"scipy-slsqp"``: Direct dual-weight SLSQP solve via scipy, with Newton + accuracy polishing if SLSQP's result misses the requested global gap. + * ``"newton-cold"``: Newton from uniform mu=1/k (stability baseline). + + The legacy name ``"fw+newton"`` is accepted as an alias for + ``"fw+bisect+newton"`` with a deprecation warning. + + Args: + matrices: SPD matrices ``A_i``, shape ``(k, d, d)``, or ``(d, d)`` for + a single vertex. + centers: Centers ``x_i``, shape ``(k, d)``, or ``(d,)`` for a single + vertex. + offsets: Optional additive constants ``delta_i``, shape ``(k,)``. + ``None`` is the original zero-offset problem. + method: One of the seven canonical method names above. + tol: Shared accepted-result duality-gap tolerance for every method, + relative to ``max(1, abs(dual value))``; it is not Newton's + residual tolerance. + max_iter: Maximum number of Frank-Wolfe iterations. + weight_tol: Threshold defining the weight support ``active_set`` and + the face selected by Newton polishing; it does not change the + global gap acceptance test. + regularization: Non-negative diagonal shift applied to ``A(mu)`` in + Frank-Wolfe evaluations and line searches, SLSQP evaluations, + Newton polishing, and the final evaluation. + condition_number_limit: Optional target for ``cond(A(mu))``. If the + target is exceeded, the diagonal shift is escalated by factors of + 10 for at most ``max_conditioning_steps`` attempts; the target is + not guaranteed after the cap. + max_conditioning_steps: Maximum number of attempted conditioning + escalations. + newton_tol: Residual tolerance for Newton's own polishing iterations, + including SLSQP's Newton polish; it does not add an acceptance + criterion after polishing. + newton_max_iter: Maximum number of Newton steps. + + Returns: + MinimaxResult with ``.method`` recording the canonical solver name + and ``.metadata`` containing method-specific diagnostics. ``alpha`` + and ``circumcenter`` approximate the exact minimax value and + circumcenter for an unadjusted solve (``regularization == 0`` and + ``condition_number_limit is None``), and the stabilised problem + otherwise; in the pairwise case the exact value is + ``ellphi.tangency(p, q).t ** 2``. For ``k = 1`` the result uses the + same conditioned evaluation as the non-singleton paths and keeps + the trivial converged, zero-iteration semantics. + + Raises: + ValueError: For an unknown method, an empty simplex or invalid + conditioning parameters. + """ + # Handle legacy alias + if method in _METHOD_ALIASES: + canonical = _METHOD_ALIASES[method] + warnings.warn( + f"Method {method!r} is deprecated, use {canonical!r} instead.", + DeprecationWarning, + stacklevel=2, + ) + method = canonical + + if method not in _VALID_METHODS: + raise ValueError(f"Unknown method {method!r}. Valid methods: {_VALID_METHODS}") + + matrices = np.asarray(matrices, dtype=float) + centers = np.asarray(centers, dtype=float) + # Accept single-simplex input (2D matrix, 1D center) + if matrices.ndim == 2: + matrices = matrices[np.newaxis] + centers = centers[np.newaxis] + + k, d = centers.shape + + _validate_tolerance("tol", tol, strictly_positive=True) + _validate_tolerance("newton_tol", newton_tol, strictly_positive=True) + _validate_tolerance("regularization", regularization) + _validate_tolerance("weight_tol", weight_tol) + _validate_positive_int("max_iter", max_iter) + _validate_positive_int("newton_max_iter", newton_max_iter) + _validate_non_negative_int("max_conditioning_steps", max_conditioning_steps) + if condition_number_limit is not None and ( + not np.isfinite(condition_number_limit) or condition_number_limit <= 1.0 + ): + raise ValueError("condition_number_limit must be finite and > 1 when provided") + + if k == 0: + raise ValueError("simplex must contain at least one vertex (k=0 given)") + if weight_tol >= 1.0 / k: + raise ValueError("weight_tol must satisfy 0 <= weight_tol < 1/k") + if offsets is not None: + offsets = np.asarray(offsets, dtype=float) + if offsets.shape != (k,): + raise ValueError(f"offsets shape {offsets.shape} inconsistent with k={k}") + if not np.all(np.isfinite(offsets)): + raise ValueError("offsets must be finite") + + # Precompute A_i x_bar_i (shape: k x d) + Ax = np.einsum("kij,kj->ki", matrices, centers) + + # --- Trivial case: single point --- + if k == 1: + xstar, f = _eval_f( + np.array([1.0]), + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + return MinimaxResult( + alpha=float(f[0]), + circumcenter=xstar, + weights=np.array([1.0]), + active_set=[0], + converged=True, + n_iter=0, + method=method, + ) + + # Uniform initialisation for FW-based methods + mu_init = np.full(k, 1.0 / k) + + _fw_kwargs: dict[str, Any] = dict( + tol=tol, + max_iter=max_iter, + weight_tol=weight_tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + + metadata: dict[str, Any] = {} + + # --- Dispatch --- + if method == "fw+bisect": + mu, converged, n_iter = _run_fw_bisect( + matrices, Ax, centers, offsets, mu_init, **_fw_kwargs + ) + metadata["fw_iters"] = n_iter + + elif method == "fw+brentq": + mu, converged, n_iter, meta_fw = _run_fw_brentq( + matrices, Ax, centers, offsets, mu_init, **_fw_kwargs + ) + metadata.update(meta_fw) + + elif method == "fw+bisect+newton": + mu, converged, n_iter_fw = _run_fw_bisect( + matrices, Ax, centers, offsets, mu_init, **_fw_kwargs + ) + active_set_fw = [i for i in range(k) if mu[i] > weight_tol] + mu, n_iter_newton, meta_newton = _newton_polish( + mu, + active_set_fw, + matrices, + Ax, + centers, + offsets, + weight_tol=weight_tol, + max_iter=newton_max_iter, + tol=newton_tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + n_iter = n_iter_fw + n_iter_newton + converged = _check_newton_convergence( + mu, + matrices, + Ax, + centers, + offsets, + meta_newton["newton_status"], + tol=tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + metadata["fw_iters"] = n_iter_fw + metadata.update(meta_newton) + + elif method == "fw+brentq+newton": + mu, converged, n_iter_fw, meta_fw = _run_fw_brentq( + matrices, Ax, centers, offsets, mu_init, **_fw_kwargs + ) + active_set_fw = [i for i in range(k) if mu[i] > weight_tol] + mu, n_iter_newton, meta_newton = _damped_newton_polish( + mu, + active_set_fw, + matrices, + Ax, + centers, + offsets, + weight_tol=weight_tol, + max_iter=newton_max_iter, + tol=newton_tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + n_iter = n_iter_fw + n_iter_newton + converged = _check_newton_convergence( + mu, + matrices, + Ax, + centers, + offsets, + meta_newton["newton_status"], + tol=tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + metadata.update(meta_fw) + metadata.update(meta_newton) + + elif method == "fw+bisect+damped-newton": + mu, converged, n_iter_fw = _run_fw_bisect( + matrices, Ax, centers, offsets, mu_init, **_fw_kwargs + ) + active_set_fw = [i for i in range(k) if mu[i] > weight_tol] + mu, n_iter_newton, meta_newton = _damped_newton_polish( + mu, + active_set_fw, + matrices, + Ax, + centers, + offsets, + weight_tol=weight_tol, + max_iter=newton_max_iter, + tol=newton_tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + n_iter = n_iter_fw + n_iter_newton + converged = _check_newton_convergence( + mu, + matrices, + Ax, + centers, + offsets, + meta_newton["newton_status"], + tol=tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + metadata["fw_iters"] = n_iter_fw + metadata.update(meta_newton) + + elif method == "newton-cold": + # Newton from uniform start (no FW warm-up) + active_set_cold = list(range(k)) + mu, n_iter_cold, meta_cold = _damped_newton_polish( + mu_init.copy(), + active_set_cold, + matrices, + Ax, + centers, + offsets, + weight_tol=weight_tol, + reidentify_face=False, + max_iter=newton_max_iter, + tol=newton_tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + n_iter = n_iter_cold + converged = _check_newton_convergence( + mu, + matrices, + Ax, + centers, + offsets, + meta_cold["newton_status"], + tol=tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + metadata.update(meta_cold) + + elif method == "scipy-slsqp": + mu, converged, n_iter = _run_scipy_slsqp( + matrices, + Ax, + centers, + offsets, + k, + tol=tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + metadata["n_fevals"] = n_iter + if not converged: + active_set_slsqp = [i for i in range(k) if mu[i] > weight_tol] + mu, n_iter_newton, meta_newton = _newton_polish( + mu, + active_set_slsqp, + matrices, + Ax, + centers, + offsets, + weight_tol=weight_tol, + max_iter=newton_max_iter, + tol=newton_tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + n_iter += n_iter_newton + converged = _check_newton_convergence( + mu, + matrices, + Ax, + centers, + offsets, + meta_newton["newton_status"], + tol=tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + metadata["accuracy_polish"] = "newton" + metadata.update(meta_newton) + + # --- Final evaluation --- + xstar, f = _eval_f( + mu, + matrices, + Ax, + centers, + offsets, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + ) + alpha = float(np.max(f)) + if not (np.isfinite(alpha) and np.all(np.isfinite(xstar))): + converged = False + active_set = [i for i in range(k) if mu[i] > weight_tol] + + return MinimaxResult( + alpha=alpha, + circumcenter=xstar, + weights=mu, + active_set=active_set, + converged=converged, + n_iter=n_iter, + method=method, + metadata=metadata or None, + ) + + +def solve_minimax( + matrices: np.ndarray, + centers: np.ndarray, + *, + offsets: np.ndarray | None = None, + method: MethodName | str = "fw+brentq+newton", + tol: float = _DEFAULT_TOL, + max_iter: int = _DEFAULT_MAX_ITER, + weight_tol: float = _DEFAULT_WEIGHT_TOL, + regularization: float = 0.0, + condition_number_limit: float | None = None, + max_conditioning_steps: int = _DEFAULT_MAX_COND_STEPS, + newton_tol: float = _NEWTON_TOL, + newton_max_iter: int = _NEWTON_MAX_ITER, +) -> MinimaxResult: + """Compute a minimax result, reporting factorization failure as non-convergence.""" + try: + return _solve_minimax( + matrices, + centers, + offsets=offsets, + method=method, + tol=tol, + max_iter=max_iter, + weight_tol=weight_tol, + regularization=regularization, + condition_number_limit=condition_number_limit, + max_conditioning_steps=max_conditioning_steps, + newton_tol=newton_tol, + newton_max_iter=newton_max_iter, + ) + except linalg.LinAlgError: + centers_array = np.asarray(centers, dtype=float) + if centers_array.ndim == 1: + centers_array = centers_array[np.newaxis] + k, d = centers_array.shape + canonical_method = _METHOD_ALIASES.get(str(method), str(method)) + weights = np.full(k, 1.0 / k) if k else np.empty(0, dtype=float) + return MinimaxResult( + alpha=float("nan"), + circumcenter=np.full(d, np.nan), + weights=weights, + active_set=list(range(k)), + converged=False, + n_iter=0, + method=canonical_method, + metadata={"solver_status": "factorization_failed"}, + ) + + +solve_minimax.__doc__ = _solve_minimax.__doc__ + + +def solve_minimax_from_coefs( + coefs: np.ndarray, + **kwargs: Any, +) -> MinimaxResult: + """Convenience wrapper: accept ellphi packed conic coefficient vectors. + + Converts ellphi's packed representation (A, b_ellphi, c) to + (A_i, x_bar_i, delta_i) and calls solve_minimax. + + Sign convention: + ellphi stores ``b_ellphi = -A x_bar`` (linear term of + ``x^T A x + 2 b^T x + c``), so ``x_bar = -A^{-1} b_ellphi``. + Completing the square gives + ``delta = c - x_bar^T A x_bar``. + + Value correspondence: + In the pairwise case ``result.alpha`` approximates + ``ellphi.tangency(coefs[0], coefs[1]).t ** 2`` (exactly equal in the + mathematical problem; numerically within solver tolerances for an + unadjusted solve, i.e. ``regularization == 0`` and + ``condition_number_limit is None``). + + Args: + coefs: Packed conic coefficient array, shape (k, m) or (m,) for k=1. + **kwargs: Forwarded to solve_minimax (method, tol, max_iter, weight_tol, + regularization, condition_number_limit, max_conditioning_steps, + newton_tol, newton_max_iter). + + Returns: + MinimaxResult. + """ + coefs = np.asarray(coefs, dtype=float) + if coefs.ndim == 1: + coefs = coefs[np.newaxis] + + A_arr, b_arr, c_arr = unpack_conic(coefs) # (k,d,d), (k,d), (k,) + + # x_bar_i = -A_i^{-1} b_i (ellphi sign convention: b = -A x_bar) + k = A_arr.shape[0] + centers = np.empty_like(b_arr) + for i in range(k): + centers[i] = _exact_linear_solve(A_arr[i], -b_arr[i]) + + centered_constants = np.einsum("ki,kij,kj->k", centers, A_arr, centers) + offsets = c_arr - centered_constants + return solve_minimax(A_arr, centers, offsets=offsets, **kwargs) diff --git a/src/ellphi/_minimax_python.pyi b/src/ellphi/_minimax_python.pyi new file mode 100644 index 0000000..7b81a62 --- /dev/null +++ b/src/ellphi/_minimax_python.pyi @@ -0,0 +1,53 @@ +from typing import Any, Literal, NamedTuple, TypeAlias + +import numpy as np + +__all__ = [ + "MethodName", + "MinimaxResult", + "solve_minimax", + "solve_minimax_from_coefs", +] + +MethodName: TypeAlias = Literal[ + "fw+bisect", + "fw+brentq", + "fw+bisect+newton", + "fw+brentq+newton", + "fw+bisect+damped-newton", + "scipy-slsqp", + "newton-cold", +] + +class MinimaxResult(NamedTuple): + alpha: float + circumcenter: np.ndarray + weights: np.ndarray + active_set: list[int] + converged: bool + n_iter: int + method: str = "fw+bisect" + metadata: dict[str, Any] | None = None + +def solve_minimax( + matrices: np.ndarray, + centers: np.ndarray, + *, + offsets: np.ndarray | None = None, + method: MethodName | str = "fw+brentq+newton", + tol: float = 1e-09, + max_iter: int = 2000, + weight_tol: float = 1e-10, + regularization: float = 0.0, + condition_number_limit: float | None = None, + max_conditioning_steps: int = 8, + newton_tol: float = 1e-14, + newton_max_iter: int = 20, +) -> MinimaxResult: + """Solve the dual; FW methods use pairwise swaps on the dual simplex.""" + ... + +def solve_minimax_from_coefs( + coefs: np.ndarray, + **kwargs: Any, +) -> MinimaxResult: ... diff --git a/src/ellphi/cech.py b/src/ellphi/cech.py new file mode 100644 index 0000000..9eaa5e1 --- /dev/null +++ b/src/ellphi/cech.py @@ -0,0 +1,719 @@ +"""Anisotropic Čech filtration API. + +The :func:`cech` function computes the anisotropic Čech filtration time for +one or more packed ellipsoid coefficient vectors while keeping the many-body +numerical engine internal. For ``k = 2`` the Čech time equals the tangency +time computed by :func:`ellphi.tangency`, and ``cech`` agrees with it. +""" + +from __future__ import annotations + +from typing import Literal, NamedTuple, cast + +import numpy as np + +from ._minimax_python import MethodName as EngineMethodName +from ._minimax_python import MinimaxResult, solve_minimax +from .geometry import unpack_conic + +__all__ = [ + "MethodName", + "CechStage", + "CechInfo", + "CechResult", + "CechGrad", + "cech", + "cech_grad", +] + + +MethodName = Literal[ + "fw+bisect", + "fw+brentq", + "fw+bisect+newton", + "fw+brentq+newton", + "fw+bisect+damped-newton", + "scipy-slsqp", + "newton-cold", + "auto", +] + +_NAMED_METHODS = ( + "fw+bisect", + "fw+brentq", + "fw+bisect+newton", + "fw+brentq+newton", + "fw+bisect+damped-newton", + "scipy-slsqp", + "newton-cold", + "fw+newton", +) + + +_NEGATIVE_ALPHA_ROUNDING_FACTOR = 64.0 +_MACHINE_EPSILON = np.finfo(float).eps + + +class CechStage(NamedTuple): + """Diagnostics for one public Čech solver stage.""" + + method: str + converged: bool + status: str + gap: float + n_iter: int + + +class CechInfo(NamedTuple): + """Traceability information for a public Čech solve. + + ``n_iter`` is the iteration or evaluation count reported by the selected + stage. The counts and outcomes for every attempted stage are in + ``stages``. + """ + + requested_method: str + method_used: str + converged: bool + gap: float + n_iter: int + stages: tuple[CechStage, ...] + + +class CechResult(NamedTuple): + """Result of an anisotropic Čech filtration calculation. + + For a simplex ``sigma`` of ellipsoids with quadratic functions ``f_i``, + ``t**2 = alpha(sigma) = min_x max_i f_i(x)`` is the least squared scale at + which all growing ellipsoids ``E_i(t) = {f_i <= t**2}`` share a point. + At the critical scale their intersection is the single point ``x*``; + active boundaries pass through ``x*`` and satisfy + ``sum_i mu_i grad f_i(x*) = 0`` with positive weights, but the boundaries + are not tangent to each other. Pairwise tangency is only the ``k = 2`` + special case. + + ``support`` is the thresholded weight support ``{i: mu[i] > weight_tol}``. + ``active_set`` is the tight-constraint set at the returned point, + ``I = {i : t**2 - f_i(point) <= active_tol * max(1, t**2)}``; when + ``alpha`` is clipped at zero, the actual shift ``t**2 - alpha`` is added + to this threshold. Up to numerical tolerance, ``support`` is a subset of + ``active_set``; under strict complementarity (ND1), they coincide. In the + degenerate example + of unit balls centred at ``(0, 0)``, ``(2, 0)``, and ``(0, 2)``, support + has two indices while ``active_set`` has three. + + The private engine's internal field named ``active_set`` is the weight + support (historical ellcech naming) and is not part of the public API. + + The fields are ordered as ``t, point, mu, support, active_set, info``. + + Attributes: + t: Čech filtration time; ``t**2 = alpha(sigma)``. + point: Common intersection point ``x*`` at the critical scale, shape + ``(d,)``. + mu: Lagrange multipliers (dual weights on the simplex), shape ``(k,)``. + support: Indices with multiplier greater than ``weight_tol``. + active_set: Tight constraints satisfying the numerical active-set + test with ``active_tol``. + info: Requested method, selected method, final gap and per-stage + convergence diagnostics. + """ + + t: float + point: np.ndarray + mu: np.ndarray + support: tuple[int, ...] + active_set: tuple[int, ...] + info: CechInfo + + +class CechGrad(NamedTuple): + """Čech filtration result and coefficient-space gradient. + + ``support`` and ``active_set`` have the definitions documented for + :class:`CechResult`. + + The fields are ordered as + ``t, point, mu, dt_dcoef, support, active_set, info``. + + Attributes: + t: Čech filtration time; ``t**2 = alpha(sigma)``. + point: Common intersection point ``x*`` at the critical scale, shape + ``(d,)``. + mu: Lagrange multipliers (dual weights on the simplex), shape ``(k,)``. + dt_dcoef: Gradient of ``t`` for independent packed coefficient + perturbations, shape ``(k, m)``. + support: Indices with multiplier greater than ``weight_tol``. + active_set: Tight constraints at ``point`` under ``active_tol``. + info: Requested method, selected method, final gap and per-stage + convergence diagnostics. + """ + + t: float + point: np.ndarray + mu: np.ndarray + dt_dcoef: np.ndarray + support: tuple[int, ...] + active_set: tuple[int, ...] + info: CechInfo + + +def _coefficient_basis(point: np.ndarray) -> np.ndarray: + """Return the packed conic monomial basis at ``point``.""" + tri_i, tri_j = np.triu_indices(point.shape[0]) + quadratic = np.where( + tri_i == tri_j, + point[tri_i] ** 2, + 2.0 * point[tri_i] * point[tri_j], + ) + return np.concatenate((quadratic, 2.0 * point, np.array([1.0]))) + + +def _validate_tolerance( + name: str, value: float, *, strictly_positive: bool = False +) -> None: + """Validate a finite numerical tolerance.""" + if not np.isfinite(value) or (value <= 0.0 if strictly_positive else value < 0.0): + qualifier = "finite and > 0" if strictly_positive else "finite and non-negative" + raise ValueError(f"{name} must be {qualifier}") + + +def _validate_positive_int(name: str, value: int) -> None: + """Validate a positive integer solver parameter.""" + if ( + isinstance(value, (bool, np.bool_)) + or not isinstance(value, (int, np.integer)) + or value <= 0 + ): + raise ValueError(f"{name} must be a positive integer") + + +def _validate_solver_parameters( + k: int, + *, + tol: float, + max_iter: int, + weight_tol: float, + newton_tol: float, + newton_max_iter: int, +) -> None: + """Validate public solver controls before dispatching to the engine.""" + _validate_tolerance("tol", tol, strictly_positive=True) + _validate_tolerance("newton_tol", newton_tol, strictly_positive=True) + _validate_tolerance("weight_tol", weight_tol) + _validate_positive_int("max_iter", max_iter) + _validate_positive_int("newton_max_iter", newton_max_iter) + if weight_tol >= 1.0 / k: + raise ValueError("weight_tol must satisfy 0 <= weight_tol < 1/k") + + +def _validate_quadratic_matrices(matrices: np.ndarray) -> np.ndarray: + """Validate symmetry and positive definiteness before solving centers.""" + lambda_minima = np.empty(matrices.shape[0], dtype=float) + for row, matrix in enumerate(matrices): + if not np.all(np.isfinite(matrix)): + raise ValueError(f"coefficient row {row} has a non-finite quadratic matrix") + + condition_number = float(np.linalg.cond(matrix, 2)) + if not np.isfinite(condition_number): + condition_number = 1.0 + symmetry_rtol = ( + _NEGATIVE_ALPHA_ROUNDING_FACTOR + * _MACHINE_EPSILON + * max(1.0, condition_number) + ) + matrix_scale = float(np.linalg.norm(matrix, ord=np.inf)) + symmetry_error = float(np.linalg.norm(matrix - matrix.T, ord=np.inf)) + if symmetry_error > symmetry_rtol * matrix_scale: + raise ValueError( + f"coefficient row {row} has a non-symmetric quadratic matrix" + ) + + try: + eigenvalues = np.linalg.eigvalsh(matrix) + except np.linalg.LinAlgError as exc: + raise ValueError( + f"coefficient row {row} has a non-positive definite quadratic matrix" + ) from exc + lambda_min = float(eigenvalues[0]) + if not np.isfinite(lambda_min) or lambda_min <= 0.0: + raise ValueError( + f"coefficient row {row} has a non-positive definite quadratic matrix" + ) + lambda_minima[row] = lambda_min + return lambda_minima + + +def _prepare_coefs( + coefs: np.ndarray, +) -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]: + """Reconstruct centers and estimate completed-square roundoff. + + A packed row encodes the constant as + ``c = xbar.T @ A @ xbar + delta``. Computing ``delta`` from a + far-translated row can therefore lose precision in the centered quadratic + term and its subtraction from ``c``. The returned per-row rounding + estimates use the computed solve residual and the dot-product/subtraction + dimension; the minimax value is a convex combination of row offsets. + Callers needing exact normalization at large translations should center + their data first. + """ + matrices, linear, constants = unpack_conic(coefs) + lambda_minima = _validate_quadratic_matrices(matrices) + centers = np.empty_like(linear) + offsets = np.empty_like(constants) + row_roundoff_estimates = np.empty_like(constants) + for row, (matrix, vector, constant) in enumerate(zip(matrices, linear, constants)): + try: + y_hat = np.linalg.solve(matrix, vector) + except np.linalg.LinAlgError as exc: + raise ValueError( + f"coefficient row {row} has a singular quadratic matrix" + ) from exc + + residual = vector - matrix @ y_hat + centered_constant = float(vector @ y_hat) + delta = float(constant) - centered_constant + solve_estimate = float( + np.linalg.norm(vector, ord=2) + * np.linalg.norm(residual, ord=2) + / lambda_minima[row] + ) + rounding_estimate = ( + (matrix.shape[0] + 2) + * _MACHINE_EPSILON + * (abs(float(constant)) + abs(centered_constant)) + ) + row_roundoff = solve_estimate + rounding_estimate + if not np.isfinite(delta) or not np.isfinite(row_roundoff): + raise ValueError( + f"coefficient row {row} has a non-finite normalization residual" + ) + centers[row] = -y_hat + offsets[row] = delta + row_roundoff_estimates[row] = row_roundoff + return matrices, centers, offsets, row_roundoff_estimates + + +def _centered_constraint_values( + point: np.ndarray, + matrices: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray, +) -> np.ndarray: + """Evaluate all centered constraints with the engine's offsets.""" + differences = point[np.newaxis, :] - centers + values = np.einsum("ki,kij,kj->k", differences, matrices, differences) + return values + offsets + + +class _StageRun(NamedTuple): + result: MinimaxResult | None + stage: CechStage + values: np.ndarray | None + dual_value: float + gap_scale: float + error: str | None + + +def _stage_status( + result: MinimaxResult, + *, + accepted: bool, + finite: bool, + finite_values: bool, +) -> str: + """Return a stable public status for one engine result.""" + if accepted: + return "converged" + if result.metadata is not None: + solver_status = result.metadata.get("solver_status") + if isinstance(solver_status, str) and solver_status != "converged": + return solver_status + newton_status = result.metadata.get("newton_status") + if isinstance(newton_status, str) and newton_status != "converged": + return newton_status + if not finite or not finite_values: + return "nonfinite" + return "gap_not_met" + + +def _run_stage( + matrices: np.ndarray, + centers: np.ndarray, + offsets: np.ndarray, + *, + method: str, + tol: float, + max_iter: int, + weight_tol: float, + newton_tol: float, + newton_max_iter: int, +) -> _StageRun: + """Run one engine stage and retain its public convergence diagnostics.""" + try: + result = solve_minimax( + matrices, + centers, + offsets=offsets, + method=cast(EngineMethodName, method), + tol=tol, + max_iter=max_iter, + weight_tol=weight_tol, + newton_tol=newton_tol, + newton_max_iter=newton_max_iter, + ) + except Exception as exc: + stage = CechStage( + method=method, + converged=False, + status=f"exception:{type(exc).__name__}", + gap=float("nan"), + n_iter=0, + ) + return _StageRun(None, stage, None, float("nan"), float("nan"), str(exc)) + + finite = bool( + np.isfinite(result.alpha) + and np.all(np.isfinite(result.circumcenter)) + and np.all(np.isfinite(result.weights)) + ) + values = _centered_constraint_values( + result.circumcenter, matrices, centers, offsets + ) + finite_values = bool(np.all(np.isfinite(values))) + final_gap = ( + float(np.max(values) - np.dot(result.weights, values)) + if finite and finite_values + else float("nan") + ) + dual_value = ( + float(np.dot(result.weights, values)) + if finite and finite_values + else float("nan") + ) + gap_scale = max(1.0, abs(dual_value)) if np.isfinite(dual_value) else float("nan") + accepted = bool( + result.converged + and finite + and finite_values + and np.isfinite(final_gap) + and final_gap <= tol * gap_scale + ) + status = _stage_status( + result, + accepted=accepted, + finite=finite, + finite_values=finite_values, + ) + stage = CechStage( + method=result.method, + converged=accepted, + status=status, + gap=final_gap, + n_iter=result.n_iter, + ) + return _StageRun(result, stage, values, dual_value, gap_scale, None) + + +def _stage_diagnostics(run: _StageRun) -> str: + """Format one stage for a failure message.""" + details = ( + f"method={run.stage.method!r}, status={run.stage.status!r}, " + f"final_gap={run.stage.gap:.17g}, n_iter={run.stage.n_iter}" + ) + if run.result is not None and run.result.metadata is not None: + n_fevals = run.result.metadata.get("n_fevals") + if n_fevals is not None: + details += f", n_fevals={n_fevals}" + if run.error: + details += f", cause={run.error!r}" + return details + + +def _raise_named_failure(run: _StageRun, tol: float) -> None: + """Raise the legacy-shaped diagnostic for a named method.""" + raise RuntimeError( + "cech did not converge: " + f"method={run.stage.method!r}, status={run.stage.status!r}, " + f"iterations={run.stage.n_iter}, duality_gap={run.stage.gap:.17g}, " + f"tol={tol:g}, duality_gap_scale={run.gap_scale:.17g}; " + f"{_stage_diagnostics(run)}; " + "a larger max_iter or a different method may be chosen" + ) + + +def _raise_auto_failure(runs: tuple[_StageRun, ...]) -> None: + """Raise a diagnostic retaining both failed auto-route stages.""" + details = "; ".join( + f"stage {index}: {_stage_diagnostics(run)}" + for index, run in enumerate(runs, start=1) + ) + raise RuntimeError(f"cech auto route failed: {details}") + + +def _assemble_result( + run: _StageRun, + *, + requested_method: str, + stages: tuple[CechStage, ...], + row_roundoff_estimates: np.ndarray, + weight_tol: float, + active_tol: float, +) -> CechResult: + """Build the public result from an accepted stage.""" + assert run.result is not None + assert run.values is not None + result = run.result + values = run.values + alpha = float(result.alpha) + support_indices = np.flatnonzero(result.weights > weight_tol) + support = tuple(int(i) for i in support_indices) + support_scale = float(np.max(np.abs(values[support_indices]))) + scale = max(abs(alpha), support_scale) + positive_indices = np.flatnonzero(result.weights > 0.0) + # The accepted allowance is weighted over every row with a positive multiplier; + # the public support remains thresholded by weight_tol. + roundoff_estimate = float( + np.dot( + result.weights[positive_indices], + row_roundoff_estimates[positive_indices], + ) + ) + negative_tolerance = max( + _NEGATIVE_ALPHA_ROUNDING_FACTOR * _MACHINE_EPSILON * max(1.0, scale), + roundoff_estimate, + ) + if alpha < -negative_tolerance: + raise ValueError("packed quadrics have no common non-negative filtration scale") + t_squared = float(max(alpha, 0.0)) + t = float(np.sqrt(max(0.0, t_squared))) + clipping_shift = t_squared - alpha + active_threshold = active_tol * max(1.0, t_squared) + clipping_shift + active_set = tuple( + int(i) for i in np.flatnonzero(t_squared - values <= active_threshold) + ) + info = CechInfo( + requested_method=requested_method, + method_used=run.stage.method, + converged=run.stage.converged, + gap=run.stage.gap, + n_iter=run.stage.n_iter, + stages=stages, + ) + return CechResult( + t=t, + point=result.circumcenter, + mu=result.weights, + support=support, + active_set=active_set, + info=info, + ) + + +def cech( + coefs: np.ndarray, + *, + method: MethodName | str = "auto", + tol: float = 1e-9, + max_iter: int = 2000, + weight_tol: float = 1e-10, + active_tol: float = 1e-9, + newton_tol: float = 1e-14, + newton_max_iter: int = 20, +) -> CechResult: + """Compute the Čech filtration time for a simplex of ellipsoids. + + The returned ``t`` satisfies ``t**2 = alpha(sigma)``, where + ``alpha(sigma) = min_x max_i f_i(x)`` is the least squared scale at which + all growing ellipsoids share a common point. For ``k = 2`` the Čech time + equals the tangency time computed by :func:`ellphi.tangency`, and ``cech`` + agrees with it. + + Args: + coefs: Packed conic coefficient vectors, shape ``(k, m)``. + method: Internal many-body solver method. The default ``"auto"`` + first runs ``"fw+brentq+newton"`` with the caller's ``tol``, + ``max_iter``, and ``newton_tol``. If that stage does not converge, + it runs gap-enforced ``"scipy-slsqp"`` from uniform dual weights + with the same ``tol``. A named method never falls back. + tol: Shared accepted-result duality-gap tolerance for every method, + relative to ``max(1, abs(dual value))``. It is not Newton's + residual tolerance. + max_iter: Maximum Frank-Wolfe iterations. + weight_tol: Threshold defining the public ``support``; it does not + remove positive-weight rows from the normalization estimate. + active_tol: Relative tolerance for the public ``active_set`` tightness + test, with only the actual zero-clipping shift added when clipping + occurs. + newton_tol: Residual tolerance for Newton's own polishing iterations; + it does not alter the shared accepted-result gap test. + newton_max_iter: Maximum Newton-polishing iterations. + + Stabilized solves are available only through the private minimax engine; + they are research controls and carry no public gradient guarantee. + + Returns: + A result containing the Čech filtration time rather than the squared + filtration value. The trailing ``info`` field records the requested + method, selected method, final gap, count, and every attempted stage. + + Raises: + ValueError: If the coefficient array is not two-dimensional, is empty, + a tolerance is not finite and non-negative, or a coefficient row + has a non-symmetric or non-positive-definite quadratic matrix, or + the packed quadrics have no common non-negative filtration scale. + Packed input requires + ``d >= 2``, as for :func:`ellphi.tangency`. + RuntimeError: If a named method does not converge or produces a + non-finite output, or if both stages of the ``"auto"`` route fail. + The error reports stage statuses, final gaps, and iteration or + evaluation counts. + """ + coefs = np.asarray(coefs, dtype=float) + if coefs.ndim != 2: + raise ValueError("Expected coefficient array with shape (k, m)") + if coefs.shape[0] == 0: + raise ValueError("simplex must contain at least one vertex (k=0 given)") + if coefs.shape[1] == 3: + raise ValueError( + "packed input requires d >= 2, as for ellphi.tangency; " + "one-dimensional packed conics are not supported" + ) + _validate_tolerance("active_tol", active_tol) + _validate_solver_parameters( + coefs.shape[0], + tol=tol, + max_iter=max_iter, + weight_tol=weight_tol, + newton_tol=newton_tol, + newton_max_iter=newton_max_iter, + ) + matrices, centers, offsets, row_roundoff_estimates = _prepare_coefs(coefs) + + if method != "auto" and method not in _NAMED_METHODS: + raise ValueError(f"Unknown method {method!r}") + requested_method = str(method) + stage_methods = ( + ("fw+brentq+newton", "scipy-slsqp") if method == "auto" else (requested_method,) + ) + runs: list[_StageRun] = [] + for stage_method in stage_methods: + run = _run_stage( + matrices, + centers, + offsets, + method=stage_method, + tol=tol, + max_iter=max_iter, + weight_tol=weight_tol, + newton_tol=newton_tol, + newton_max_iter=newton_max_iter, + ) + runs.append(run) + if run.stage.converged: + return _assemble_result( + run, + requested_method=requested_method, + stages=tuple(item.stage for item in runs), + row_roundoff_estimates=row_roundoff_estimates, + weight_tol=weight_tol, + active_tol=active_tol, + ) + if method != "auto": + _raise_named_failure(run, tol) + _raise_auto_failure(tuple(runs)) + raise AssertionError("unreachable") + + +def cech_grad( + coefs: np.ndarray, + *, + method: MethodName | str = "auto", + tol: float = 1e-9, + max_iter: int = 2000, + weight_tol: float = 1e-10, + active_tol: float = 1e-9, + newton_tol: float = 1e-14, + newton_max_iter: int = 20, +) -> CechGrad: + """Return the Čech filtration time and its coefficient-space gradient. + + For EllPHi's packed conic ``f_i(x) = coef_i @ basis(x)``, the basis is + ``[x_j**2 if j == l else 2*x_j*x_l for j <= l, 2*x_0, ..., 2*x_(d-1), 1]`` + in NumPy upper-triangle order. The envelope theorem gives + ``d(t**2)/d coef_i = mu_i * basis(point)`` and therefore + ``dt/d coef_i = mu_i * basis(point) / (2*t)``. + + The engine passes the completed-square offset + ``delta_i = c_i - b_i^T A_i^-1 b_i`` through the minimax problem. Thus + ``dt_dcoef`` is the true gradient for independent packed-coordinate + perturbations, including the constant component ``mu_i / (2*t)``. + + A packed row encodes ``c_i = xbar_i^T A_i xbar_i + delta_i``. For + ``|xbar_i|**2 >> 1`` or ill-conditioned ``A_i``, the representable + ``delta_i`` carries roundoff from the computed centered quadratic term and + its subtraction from ``c_i``. Callers needing exact normalization at large + translations should center their data first. + + This is a gradient of ``t``, not ``t**2``. Its hypotheses are a + non-degenerate input, at least two indices in ``active_set``, multipliers + supported on ``active_set``, and an exact weighted linear solve. Solver + convergence or a small residual alone is not a derivative certificate. + For far-translated inputs, evaluating this coefficient-space derivative in + the uncentred packed basis can itself be poorly conditioned. + + Args: + coefs: Packed conic coefficient vectors, shape ``(k, m)``. + method: Internal many-body solver method. The default ``"auto"`` + first runs ``"fw+brentq+newton"`` with the caller's ``tol``, + ``max_iter``, and ``newton_tol``. If that stage does not converge, + it runs gap-enforced ``"scipy-slsqp"`` from uniform dual weights + with the same ``tol``. A named method never falls back. + tol: Shared accepted-result duality-gap tolerance for every method, + relative to ``max(1, abs(dual value))``. It is not Newton's + residual tolerance. + max_iter: Maximum Frank-Wolfe iterations. + weight_tol: Threshold defining the public ``support``; it does not + remove positive-weight rows from the normalization estimate. + active_tol: Relative tolerance for the public ``active_set`` tightness + test, with only the actual zero-clipping shift added when clipping + occurs. + newton_tol: Residual tolerance for Newton's own polishing iterations; + it does not alter the shared accepted-result gap test. + newton_max_iter: Maximum Newton-polishing iterations. + + Returns: + Čech filtration data and ``dt_dcoef`` with shape ``(k, m)``. The + trailing ``info`` field mirrors the forward solve's diagnostics. + + Raises: + ValueError: If a quadratic block is not symmetric positive definite. + ZeroDivisionError: If the Čech filtration time is zero. + RuntimeError: If the forward solve fails. + """ + coefs = np.asarray(coefs, dtype=float) + result = cech( + coefs, + method=method, + tol=tol, + max_iter=max_iter, + weight_tol=weight_tol, + active_tol=active_tol, + newton_tol=newton_tol, + newton_max_iter=newton_max_iter, + ) + if result.t == 0.0: + raise ZeroDivisionError("Čech gradient is undefined when t == 0") + + basis = _coefficient_basis(result.point) + dt_dcoef = result.mu[:, np.newaxis] * basis[np.newaxis, :] / (2.0 * result.t) + return CechGrad( + t=result.t, + point=result.point, + mu=result.mu, + dt_dcoef=dt_dcoef, + support=result.support, + active_set=result.active_set, + info=result.info, + ) diff --git a/src/ellphi/cech.pyi b/src/ellphi/cech.pyi new file mode 100644 index 0000000..d808fdf --- /dev/null +++ b/src/ellphi/cech.pyi @@ -0,0 +1,84 @@ +from typing import Literal, NamedTuple, TypeAlias + +import numpy as np + +__all__ = [ + "MethodName", + "CechStage", + "CechInfo", + "CechResult", + "CechGrad", + "cech", + "cech_grad", +] + +MethodName: TypeAlias = Literal[ + "fw+bisect", + "fw+brentq", + "fw+bisect+newton", + "fw+brentq+newton", + "fw+bisect+damped-newton", + "scipy-slsqp", + "newton-cold", + "auto", +] + +class CechStage(NamedTuple): + method: str + converged: bool + status: str + gap: float + n_iter: int + +class CechInfo(NamedTuple): + requested_method: str + method_used: str + converged: bool + gap: float + n_iter: int + stages: tuple[CechStage, ...] + +class CechResult(NamedTuple): + t: float + point: np.ndarray + mu: np.ndarray + support: tuple[int, ...] + active_set: tuple[int, ...] + info: CechInfo + +class CechGrad(NamedTuple): + t: float + point: np.ndarray + mu: np.ndarray + dt_dcoef: np.ndarray + support: tuple[int, ...] + active_set: tuple[int, ...] + info: CechInfo + +def cech( + coefs: np.ndarray, + *, + method: MethodName | str = "auto", + tol: float = 1e-9, + max_iter: int = 2000, + weight_tol: float = 1e-10, + active_tol: float = 1e-9, + newton_tol: float = 1e-14, + newton_max_iter: int = 20, +) -> CechResult: + """Compute Čech time using the selected solver method.""" + ... + +def cech_grad( + coefs: np.ndarray, + *, + method: MethodName | str = "auto", + tol: float = 1e-9, + max_iter: int = 2000, + weight_tol: float = 1e-10, + active_tol: float = 1e-9, + newton_tol: float = 1e-14, + newton_max_iter: int = 20, +) -> CechGrad: + """Compute Čech time and its gradient using the selected solver method.""" + ... diff --git a/stubtest-allowlist.txt b/stubtest-allowlist.txt index 58954b0..5fb7cfe 100644 --- a/stubtest-allowlist.txt +++ b/stubtest-allowlist.txt @@ -1 +1,3 @@ ellphi._solver_python.MethodName +ellphi._minimax_python.MethodName +ellphi.cech.MethodName diff --git a/tests/factories.py b/tests/factories.py index 462759a..ac4d9ff 100644 --- a/tests/factories.py +++ b/tests/factories.py @@ -47,3 +47,87 @@ def random_cloud( coefs = coef_from_cov(means, covs) dummy_nbd = np.empty((n_ellipses, 0), dtype=int) return EllipseCloud(coef=coefs, mean=means, cov=covs, k=0, nbd=dummy_nbd) + + +def random_simplex( + k: int, + d: int, + *, + rng: np.random.Generator, + cond_bound: float = 10.0, +) -> tuple[NDArray[np.float64], NDArray[np.float64]]: + """Random centers and SPD matrices with ``cond(A_i) <= cond_bound``. + + Same sampling as ellcech's ``benchmarks.make_random_simplex``. + """ + centers = rng.standard_normal((k, d)) + matrices = np.empty((k, d, d)) + for i in range(k): + U = np.linalg.qr(rng.standard_normal((d, d)))[0] + eigvals = rng.uniform(1.0, cond_bound, size=d) + matrices[i] = U @ np.diag(eigvals) @ U.T + return matrices, centers + + +def minimax_public_surrogate() -> tuple[np.ndarray, np.ndarray]: + """Return the public-safe deterministic minimax regression instance. + + The literals are reproduced from this complete recipe. Create + ``rng = np.random.default_rng(10602)`` and traverse zero-based instances + ``0`` through ``12`` in order. For each instance, first traverse matrix + rows ``i = 0`` through ``5`` and, for each row, draw + ``q = np.linalg.qr(rng.standard_normal((2, 2)))[0]`` followed by + ``eigenvalues = np.exp(rng.uniform(-1.0, 1.0, size=2))`` and construct + ``(q * eigenvalues) @ q.T``. After all six matrices, draw + ``centers = rng.standard_normal((6, 2))``. The returned literals are the + matrices and centers from instance 12; this explicit matrix-first, + row-major traversal is part of the fixture definition. + + At the optimum, constraint ``f_4`` is nearly tight even though ``mu_4`` is + zero, placing the instance near a failure of strict complementarity. The + all-index Frank-Wolfe gap must become much smaller than that slack, so the + near-tight zero-weight vertex 4 keeps re-entering as the pairwise swap + target and the all-index gap stalls. Newton polishing on the resulting + face is rank-deficient, whereas the dual-weight SLSQP solver is unaffected. + """ + matrices = np.array( + [ + [ + [0.6508813596346038, 0.2489660729149641], + [0.2489660729149641, 0.7881614960406568], + ], + [ + [0.7004122505872777, 0.14197229432398561], + [0.14197229432398561, 1.1445419752575752], + ], + [ + [0.6661363125679673, -0.7228217513541597], + [-0.7228217513541597, 2.3994411001217943], + ], + [ + [0.555802471026214, 0.26197345221646695], + [0.26197345221646695, 0.9029253520499695], + ], + [ + [2.304214987069779, 0.6449129856946328], + [0.6449129856946328, 0.737837149118542], + ], + [ + [0.7373374529979612, 0.10773179234057872], + [0.10773179234057872, 0.8413822561278285], + ], + ], + dtype=float, + ) + centers = np.array( + [ + [0.3708705067926382, 0.48366396611109785], + [1.16738707134914, -0.9051617225542165], + [-0.6339208703399198, -0.47057481821754643], + [0.1207125164253454, -1.267286300475832], + [0.877114694674914, -0.46523959355638184], + [-0.4268970087617493, 0.6371793650807637], + ], + dtype=float, + ) + return matrices, centers diff --git a/tests/test_cech.py b/tests/test_cech.py new file mode 100644 index 0000000..3e5f41e --- /dev/null +++ b/tests/test_cech.py @@ -0,0 +1,675 @@ +"""Tests for the anisotropic Čech filtration API.""" + +from __future__ import annotations + +import importlib +import time +from typing import get_args + +import numpy as np +import pytest + +import ellphi +import ellphi._minimax_python as minimax_module +from ellphi._minimax_python import MethodName, solve_minimax +from ellphi.geometry import coef_from_cov, pack_conic, unpack_conic + +from .factories import minimax_public_surrogate, random_coef_pair, random_covariance + + +cech_module = importlib.import_module("ellphi.cech") + + +def _random_coefs(k: int, d: int, seed: int) -> np.ndarray: + rng = np.random.default_rng(seed) + centers = rng.uniform(-2.0, 2.0, size=(k, d)) + covariances = np.stack([random_covariance(rng, dim=d) for _ in range(k)]) + return coef_from_cov(centers, covariances) + + +def _relative_error(actual: np.ndarray | float, expected: np.ndarray | float) -> float: + numerator = float(np.linalg.norm(actual - expected)) + denominator = max( + float(np.linalg.norm(actual)), float(np.linalg.norm(expected)), 1e-15 + ) + return numerator / denominator + + +def test_pairwise_cech_agrees_with_tangency(solver_backend, rng): + p, q = random_coef_pair(rng, dim=3) + result = ellphi.cech(np.stack((p, q))) + pairwise = ellphi.tangency(p, q, backend=solver_backend) + + assert result.t == pytest.approx(pairwise.t, rel=1e-9) + np.testing.assert_allclose(result.point, pairwise.point, rtol=1e-9, atol=1e-10) + np.testing.assert_allclose( + result.mu, (1.0 - pairwise.mu, pairwise.mu), rtol=1e-8, atol=1e-10 + ) + + +@pytest.mark.parametrize("dimension", [2, 3]) +def test_unnormalized_pairwise_cech_agrees_with_tangency( + solver_backend, rng, dimension +): + for _ in range(5): + centers = rng.uniform(-2.0, 2.0, size=(2, dimension)) + while np.linalg.norm(centers[0] - centers[1]) < 1.5: + centers = rng.uniform(-2.0, 2.0, size=(2, dimension)) + covariances = np.stack( + [random_covariance(rng, dim=dimension) for _ in range(2)] + ) + coefs = coef_from_cov(centers, covariances) + coefs[:, -1] += rng.uniform(0.1, 1.0) + rng.uniform(-1e-4, 1e-4, size=2) + + result = ellphi.cech(coefs) + pairwise = ellphi.tangency(coefs[0], coefs[1], backend=solver_backend) + + assert result.t == pytest.approx(pairwise.t, rel=1e-9) + np.testing.assert_allclose(result.point, pairwise.point, rtol=1e-9, atol=1e-10) + + +def test_translated_pair_uses_engine_value_and_centred_constraints(solver_backend): + centers = np.array([[1e6, 0.0], [1e6 + 2.0, 0.0]]) + coefs = coef_from_cov(centers, np.repeat(np.eye(2)[None], 2, axis=0)) + pairwise_coefs = coef_from_cov( + np.array([[0.0, 0.0], [2.0, 0.0]]), np.repeat(np.eye(2)[None], 2, axis=0) + ) + + # Packed inputs carry translation-sensitive roundoff; this is a + # representation limit, not a solver property. + _, _, _, translation_estimates = cech_module._prepare_coefs(coefs) + translation_estimate = float(np.max(translation_estimates)) + result = ellphi.cech(coefs) + pairwise = ellphi.tangency( + pairwise_coefs[0], pairwise_coefs[1], backend=solver_backend + ) + gradient = ellphi.cech_grad(coefs) + tri_i, tri_j = np.triu_indices(2) + basis = np.concatenate( + ( + np.where( + tri_i == tri_j, + result.point[tri_i] ** 2, + 2.0 * result.point[tri_i] * result.point[tri_j], + ), + 2.0 * result.point, + np.array([1.0]), + ) + ) + pairwise_identity = 0.5 * basis / (2.0 * pairwise.t) + + assert abs(result.t**2 - pairwise.t**2) <= translation_estimate + assert result.active_set == (0, 1) + assert np.all(np.isfinite(gradient.dt_dcoef)) + assert _relative_error(gradient.dt_dcoef[0], pairwise_identity) < 1e-6 + assert _relative_error(gradient.dt_dcoef[1], pairwise_identity) < 1e-6 + + +def test_translated_singleton_roundoff_negative_scale_is_clipped(): + center = np.array([[1e6, -1e6]]) + covariance = np.array([[1.0, 0.2], [0.2, 1.7]]) + coefs = coef_from_cov(center, covariance[None]) + result = ellphi.cech(coefs) + + assert result.t == 0.0 + + +def test_clipped_negative_scale_preserves_singleton_active_set(): + coefs = coef_from_cov(np.array([[1e6, 0.0]]), np.eye(2)[None]) + coefs[0, -1] = np.nextafter(1e12, -np.inf) + + result = ellphi.cech(coefs) + + assert result.t == 0.0 + assert result.support == (0,) + assert result.active_set == (0,) + + +def test_translated_coincident_roundoff_negative_scale_is_clipped(): + center = np.array([[1e6, -1e6]]) + covariance = np.array([[1.0, 0.2], [0.2, 1.7]]) + coefs = np.repeat(coef_from_cov(center, covariance[None]), 2, axis=0) + result = ellphi.cech(coefs) + + assert result.t == 0.0 + + +@pytest.mark.parametrize("dimension", [2, 3, 4]) +@pytest.mark.parametrize("condition_number", [1e2, 1e6, 1e10, 1e12]) +def test_coef_from_cov_accepts_rotated_spd_conditioning(dimension, condition_number): + rng = np.random.default_rng(2700 + dimension) + rotation, _ = np.linalg.qr(rng.standard_normal((dimension, dimension))) + eigenvalues = np.geomspace(1.0, condition_number, dimension) + covariance = rotation @ np.diag(eigenvalues) @ rotation.T + center = rng.uniform(-2.0, 2.0, size=(1, dimension)) + coefs = coef_from_cov(center, covariance[None]) + + matrices, linear, constants = unpack_conic(coefs) + + if condition_number <= 1e6: + result = ellphi.cech(coefs) + assert np.isfinite(result.t) + point = result.point + else: + inverse_times_linear = np.linalg.solve(matrices[0], linear[0]) + centers = np.stack([-inverse_times_linear]) + centered_constant = float(linear[0] @ inverse_times_linear) + result = solve_minimax( + matrices, centers, offsets=np.array([constants[0] - centered_constant]) + ) + assert np.isfinite(result.alpha) + point = result.circumcenter + assert np.all(np.isfinite(point)) + + +def test_scaled_centres_from_cov_are_accepted(): + centers = np.array([[1e6, -1e6], [1e6 + 2.0, -1e6 + 1.0]]) + covariances = np.array( + [ + [[2.0, 0.3], [0.3, 1.5]], + [[1.2, -0.2], [-0.2, 0.8]], + ] + ) + + result = ellphi.cech(coef_from_cov(centers, covariances), method="fw+brentq") + + assert np.isfinite(result.t) + assert result.active_set == (0, 1) + + +def test_subthreshold_packed_constant_is_honoured_by_gradient(): + matrices = np.repeat(np.eye(2)[np.newaxis], 2, axis=0) + centers = np.array([[0.0, 0.0], [2.0, 0.0]]) + linear = -np.einsum("kij,kj->ki", matrices, centers) + baseline_coefs = pack_conic(matrices, linear, np.array([0.0, 4.0])) + perturbed_coefs = baseline_coefs.copy() + perturbed_coefs[0, -1] += 5e-10 + solver_kwargs = {"method": "fw+brentq+newton", "tol": 1e-12} + + baseline = ellphi.cech(baseline_coefs, **solver_kwargs) + perturbed = ellphi.cech(perturbed_coefs, **solver_kwargs) + gradient = ellphi.cech_grad(perturbed_coefs, **solver_kwargs) + + expected_change = baseline.mu[0] * 5e-10 + assert perturbed.t**2 - baseline.t**2 == pytest.approx( + expected_change, rel=1e-6, abs=1e-15 + ) + + h = 1e-7 + plus = perturbed_coefs.copy() + minus = perturbed_coefs.copy() + plus[0, -1] += h + minus[0, -1] -= h + finite_difference = ( + ellphi.cech(plus, **solver_kwargs).t - ellphi.cech(minus, **solver_kwargs).t + ) / (2.0 * h) + assert gradient.dt_dcoef[0, -1] == pytest.approx( + finite_difference, rel=1e-6, abs=1e-12 + ) + + +def test_negative_packed_scale_raises(): + coefs = coef_from_cov( + np.array([[0.0, 0.0], [2.0, 0.0]]), np.repeat(np.eye(2)[None], 2, axis=0) + ) + coefs[:, -1] -= 2.0 + + with pytest.raises( + ValueError, match="packed quadrics have no common non-negative filtration scale" + ): + ellphi.cech(coefs) + + +def test_negative_scale_rounding_estimate_does_not_grow_with_simplex_size(): + k = 100 + centers = np.repeat(np.array([[1e6, 0.0]]), k, axis=0) + coefs = coef_from_cov(centers, np.repeat(np.eye(2)[None], k, axis=0)) + coefs[:, -1] = 1e12 - 0.5 + + with pytest.raises( + ValueError, match="packed quadrics have no common non-negative filtration scale" + ): + ellphi.cech(coefs) + + +def test_subthreshold_positive_weight_contributes_rounding_estimate(): + matrices = np.repeat(np.eye(2)[None], 2, axis=0) + centers = np.array([[2**40, 0.0], [0.0, 0.0]]) + linear = -np.einsum("kij,kj->ki", matrices, centers) + constants = np.array([2047 * 2**60 - 3 * 2**26, -(2**60)], dtype=float) + coefs = pack_conic(matrices, linear, constants) + + result = ellphi.cech( + coefs, + method="scipy-slsqp", + tol=1.0, + weight_tol=0.01, + ) + + assert result.support == (1,) + assert np.isfinite(result.t) + + +def test_active_set_uses_only_actual_clipping_shift(): + matrices = np.repeat(np.eye(2)[None], 2, axis=0) + centers = np.repeat(np.array([[1e7, 0.0]]), 2, axis=0) + linear = -np.einsum("kij,kj->ki", matrices, centers) + constants = np.einsum("ki,kij,kj->k", centers, matrices, centers) + coefs = pack_conic(matrices, linear, constants + np.array([1.0, 0.0])) + + result = ellphi.cech(coefs, method="scipy-slsqp", active_tol=0.0) + + assert result.active_set == (0,) + + +def test_rounding_estimate_includes_solve_residual(monkeypatch): + matrices = np.array([[[2.0, 0.0], [0.0, 1.0]]]) + linear = np.array([[2.0, 1.0]]) + constants = np.array([7.0]) + original_solve = np.linalg.solve + perturbation = np.array([0.125, -0.25]) + + def perturbed_solve(matrix, vector): + return original_solve(matrix, vector) + perturbation + + monkeypatch.setattr(cech_module.np.linalg, "solve", perturbed_solve) + _, _, _, estimates = cech_module._prepare_coefs( + pack_conic(matrices, linear, constants) + ) + + y_hat = original_solve(matrices[0], linear[0]) + perturbation + residual = linear[0] - matrices[0] @ y_hat + solve_estimate = ( + np.linalg.norm(linear[0]) + * np.linalg.norm(residual) + / np.linalg.eigvalsh(matrices[0])[0] + ) + centered_constant = float(linear[0] @ y_hat) + rounding_estimate = ( + (matrices.shape[1] + 2) + * np.finfo(float).eps + * (abs(constants[0]) + abs(centered_constant)) + ) + assert estimates[0] == pytest.approx(solve_estimate + rounding_estimate, rel=1e-14) + + +@pytest.mark.parametrize("api", [ellphi.cech, ellphi.cech_grad]) +@pytest.mark.parametrize( + "matrix", + [ + np.diag([1.0, -1.0]), + -np.eye(2), + ], +) +def test_public_api_rejects_non_spd_quadratic_blocks(api, matrix): + coefs = pack_conic(matrix[None], np.zeros((1, 2)), np.zeros(1)) + + with pytest.raises( + ValueError, + match=r"coefficient row 0 has a non-positive definite quadratic matrix", + ): + api(coefs) + + +@pytest.mark.parametrize("api", [ellphi.cech, ellphi.cech_grad]) +def test_public_api_rejects_non_symmetric_quadratic_blocks(api, monkeypatch): + matrices = np.array([[[2.0, 0.25], [0.0, 2.0]]]) + linear = np.zeros((1, 2)) + constants = np.zeros(1) + monkeypatch.setattr( + cech_module, + "unpack_conic", + lambda _: (matrices, linear, constants), + ) + + with pytest.raises( + ValueError, + match=r"coefficient row 0 has a non-symmetric quadratic matrix", + ): + api(np.zeros((1, 6))) + + +@pytest.mark.parametrize("api", [ellphi.cech, ellphi.cech_grad]) +def test_public_api_accepts_ill_conditioned_spd_quadratic_block(api): + matrix = np.diag([1.0, 1e12]) + matrices = np.repeat(matrix[None], 2, axis=0) + centers = np.array([[0.0, 0.0], [2.0, 0.0]]) + linear = -np.einsum("kij,kj->ki", matrices, centers) + constants = np.einsum("ki,kij,kj->k", centers, matrices, centers) + coefs = pack_conic(matrices, linear, constants) + + result = api(coefs) + assert result.t == pytest.approx(1.0) + + +def test_packed_one_dimensional_input_is_rejected_like_pairwise(): + coefs = np.array([[1.0, -2.0, 1.0], [1.0, -4.0, 4.0]]) + + with pytest.raises( + ValueError, match="packed input requires d >= 2, as for ellphi.tangency" + ): + ellphi.cech(coefs) + + +@pytest.mark.parametrize("method", list(get_args(MethodName))) +def test_right_triangle_distinguishes_support_and_active_set(method): + centers = np.array([[2.0, 0.0], [0.0, 2.0], [0.0, 0.0]]) + coefs = coef_from_cov(centers, np.repeat(np.eye(2)[None], 3, axis=0)) + + result = ellphi.cech(coefs, method=method, active_tol=1e-12) + + assert result.t == pytest.approx(np.sqrt(2.0), rel=1e-12) + np.testing.assert_allclose(result.point, [1.0, 1.0], atol=1e-12) + assert len(result.support) == 2 + assert len(result.active_set) == 3 + assert result.support == (0, 1) + assert set(result.support) <= set(result.active_set) + assert result.active_set == (0, 1, 2) + assert len(result.info.stages) == 1 + assert result.info.method_used == method + + +def test_singleton_and_empty_cech_input(): + coefs = coef_from_cov(np.array([[1.0, -2.0]]), np.eye(2)[None]) + result = ellphi.cech(coefs) + + assert result.t == 0.0 + np.testing.assert_allclose(result.point, [1.0, -2.0]) + np.testing.assert_array_equal(result.mu, [1.0]) + assert result.support == (0,) + assert result.active_set == (0,) + + with pytest.raises(ValueError, match="at least one vertex"): + ellphi.cech(np.empty((0, coefs.shape[1]))) + + +def test_non_convergence_raises_with_gap_and_retry_guidance(): + coefs = _random_coefs(5, 3, seed=101) + with pytest.raises(RuntimeError) as exc_info: + ellphi.cech(coefs, method="fw+bisect", max_iter=1, tol=1e-15) + + message = str(exc_info.value) + assert "method='fw+bisect'" in message + assert "iterations=1" in message + assert "duality_gap=" in message + assert "tol=1e-15" in message + assert "larger max_iter or a different method" in message + + +@pytest.mark.parametrize("method", ["fw+bisect", "auto"]) +def test_factorization_failure_status_is_preserved_in_public_diagnostics( + monkeypatch, method +): + coefs = _random_coefs(2, 2, seed=102) + + def fail_factorization(*args, **kwargs): + raise np.linalg.LinAlgError("test factorization failure") + + monkeypatch.setattr(minimax_module.linalg, "cho_factor", fail_factorization) + + with pytest.raises(RuntimeError) as exc_info: + ellphi.cech(coefs, method=method) + + message = str(exc_info.value) + assert "factorization_failed" in message + assert "nonfinite" not in message + if method == "auto": + assert "stage 1" in message + assert "stage 2" in message + + +@pytest.mark.parametrize("api", [ellphi.cech, ellphi.cech_grad]) +@pytest.mark.parametrize( + "name, value", + [ + ("regularization", 1e-6), + ("condition_number_limit", 1e6), + ("max_conditioning_steps", 4), + ], +) +def test_public_api_rejects_private_stabilization_controls(api, name, value): + coefs = _random_coefs(2, 2, seed=107) + + with pytest.raises(TypeError): + api(coefs, **{name: value}) + + +def test_public_newton_cold_reports_nonconvergence_without_retry(): + matrices, centers = minimax_public_surrogate() + linear = -np.einsum("kij,kj->ki", matrices, centers) + constants = np.einsum("ki,kij,kj->k", centers, matrices, centers) + coefs = pack_conic(matrices, linear, constants) + + engine_result = solve_minimax(matrices, centers, method="newton-cold") + assert not engine_result.converged + assert engine_result.metadata is not None + assert "newton_status" in engine_result.metadata + + with pytest.raises(RuntimeError, match="duality_gap="): + ellphi.cech(coefs, method="newton-cold") + + +@pytest.mark.parametrize("k,d,seed", [(3, 2, 201), (4, 2, 203)]) +def test_gradient_matches_independent_packed_central_difference(k, d, seed): + rng = np.random.default_rng(seed) + centers = rng.uniform(-2.0, 2.0, size=(k, d)) + covariances = np.stack([random_covariance(rng, dim=d) for _ in range(k)]) + coefs = coef_from_cov(centers, covariances) + coefs[:, -1] += rng.uniform(0.1, 1.0, size=k) + + solver_kwargs = { + "method": "fw+brentq+newton", + "tol": 1e-12, + "newton_tol": 1e-14, + } + gradient = ellphi.cech_grad(coefs, **solver_kwargs) + h = 1e-6 + finite_difference = np.empty_like(coefs) + for index in np.ndindex(coefs.shape): + plus = coefs.copy() + minus = coefs.copy() + plus[index] += h + minus[index] -= h + finite_difference[index] = ( + ellphi.cech(plus, **solver_kwargs).t - ellphi.cech(minus, **solver_kwargs).t + ) / (2.0 * h) + + error = _relative_error(gradient.dt_dcoef, finite_difference) + assert error < 1e-6, f"k={k}, d={d}, relative error={error:.3e}" + + +def test_pairwise_gradient_identity(solver_backend, rng): + p, q = random_coef_pair(rng, dim=3) + gradient = ellphi.cech_grad(np.stack((p, q))) + pairwise = ellphi.tangency_grad(p, q, backend=solver_backend) + + error_p = _relative_error(gradient.dt_dcoef[0], pairwise.dt_dp) + error_q = _relative_error(gradient.dt_dcoef[1], pairwise.dt_dq) + assert error_p < 1e-6, f"dt_dp relative error={error_p:.3e}" + assert error_q < 1e-6, f"dt_dq relative error={error_q:.3e}" + + +def test_zero_time_gradient_raises(): + coefs = coef_from_cov(np.array([[1.0, -2.0]]), np.eye(2)[None]) + with pytest.raises(ZeroDivisionError, match="t == 0"): + ellphi.cech_grad(coefs) + + +@pytest.mark.parametrize("api", [ellphi.cech, ellphi.cech_grad]) +def test_unnormalized_constant_agrees_with_expected(api): + coefs = coef_from_cov( + np.array([[0.0, 0.0], [2.0, 0.0]]), np.repeat(np.eye(2)[None], 2, axis=0) + ) + coefs[0, -1] += 1.0 + + result = api(coefs) + + assert result.t == pytest.approx(1.25) + np.testing.assert_allclose(result.point, [0.75, 0.0], atol=1e-12) + assert result.active_set == (0, 1) + + +def _public_surrogate_coefs(): + matrices, centers = minimax_public_surrogate() + linear = -np.einsum("kij,kj->ki", matrices, centers) + constants = np.einsum("ki,kij,kj->k", centers, matrices, centers) + return pack_conic(matrices, linear, constants) + + +def test_auto_ordinary_simplex_uses_stage_one_only(): + coefs = _random_coefs(4, 2, seed=113) + + result = ellphi.cech(coefs, method="auto") + + assert result.info.requested_method == "auto" + assert result.info.method_used == "fw+brentq+newton" + assert result.info.converged + assert len(result.info.stages) == 1 + assert result.info.stages[0].method == "fw+brentq+newton" + assert result.info.stages[0].converged + + +def test_public_surrogate_auto_falls_back_to_slsqp(): + coefs = _public_surrogate_coefs() + started = time.perf_counter() + result = ellphi.cech(coefs, method="auto") + elapsed = time.perf_counter() - started + + assert elapsed <= 5.0, f"informational auto route wall time: {elapsed:.3f}s" + assert result.t**2 == pytest.approx(0.9024444260, abs=1e-8) + assert result.info.requested_method == "auto" + assert result.info.method_used == "scipy-slsqp" + assert result.info.converged + assert len(result.info.stages) == 2 + assert not result.info.stages[0].converged + assert result.info.stages[0].status == "armijo_rejected" + assert result.info.stages[1].converged + assert result.info.stages[1].method == "scipy-slsqp" + + +def test_auto_reports_both_failed_stages(): + coefs = _public_surrogate_coefs() + + with pytest.raises(RuntimeError) as exc_info: + ellphi.cech(coefs, method="auto", tol=1e-30, max_iter=1) + + message = str(exc_info.value) + assert "stage 1" in message + assert "stage 2" in message + assert "fw+brentq+newton" in message + assert "scipy-slsqp" in message + assert "status=" in message + assert "final_gap=" in message + assert "n_iter=" in message + + +def test_public_surrogate_fw_brentq_succeeds_with_large_budget(): + coefs = _public_surrogate_coefs() + + result = ellphi.cech(coefs, method="fw+brentq", max_iter=20000) + + assert result.t**2 == pytest.approx(0.9024444260, abs=1e-8) + assert result.support == (1, 3, 5) + assert result.info.method_used == "fw+brentq" + assert len(result.info.stages) == 1 + + +def test_public_surrogate_slsqp_succeeds_at_default_budget(): + coefs = _public_surrogate_coefs() + + result = ellphi.cech(coefs, method="scipy-slsqp") + + assert result.t**2 == pytest.approx(0.9024444260, abs=1e-8) + assert result.support == (1, 3, 5) + assert result.info.method_used == "scipy-slsqp" + assert len(result.info.stages) == 1 + + +def test_named_method_does_not_fall_back_on_surrogate(): + coefs = _public_surrogate_coefs() + + with pytest.raises(RuntimeError) as exc_info: + ellphi.cech(coefs, method="fw+brentq+newton") + + message = str(exc_info.value) + assert "method='fw+brentq+newton'" in message + assert "scipy-slsqp" not in message + + +@pytest.mark.parametrize("api", [ellphi.cech, ellphi.cech_grad]) +@pytest.mark.parametrize("name", ["active_tol", "weight_tol"]) +@pytest.mark.parametrize("value", [np.nan, -1.0]) +def test_cech_tolerances_must_be_finite_and_nonnegative(api, name, value): + coefs = coef_from_cov( + np.array([[0.0, 0.0], [2.0, 0.0]]), np.repeat(np.eye(2)[None], 2, axis=0) + ) + + with pytest.raises(ValueError, match=f"{name} must be finite and non-negative"): + api(coefs, **{name: value}) + + +@pytest.mark.parametrize("value", [np.inf, np.nan, 0.0]) +def test_public_tol_must_be_finite_and_positive(value): + coefs = coef_from_cov( + np.array([[0.0, 0.0], [2.0, 0.0]]), np.repeat(np.eye(2)[None], 2, axis=0) + ) + + with pytest.raises(ValueError, match="tol must be finite and > 0"): + ellphi.cech(coefs, tol=value) + + +@pytest.mark.parametrize("value", [np.nan, 0.0]) +def test_public_newton_tol_must_be_finite_and_positive(value): + coefs = coef_from_cov( + np.array([[0.0, 0.0], [2.0, 0.0]]), np.repeat(np.eye(2)[None], 2, axis=0) + ) + + with pytest.raises(ValueError, match="newton_tol must be finite and > 0"): + ellphi.cech(coefs, newton_tol=value) + + +def test_public_namedtuple_field_order(): + assert ellphi.CechResult._fields == ( + "t", + "point", + "mu", + "support", + "active_set", + "info", + ) + assert ellphi.CechGrad._fields == ( + "t", + "point", + "mu", + "dt_dcoef", + "support", + "active_set", + "info", + ) + + +def test_public_exports_hide_internal_engine(): + expected = { + "CechResult", + "CechGrad", + "CechStage", + "CechInfo", + "cech", + "cech_grad", + } + assert expected <= set(ellphi.__all__) + assert not { + "MinimaxResult", + "GradientResult", + "solve_minimax", + "solve_minimax_from_coefs", + "compute_gradient", + } & set(ellphi.__all__) + for name in ( + "MinimaxResult", + "GradientResult", + "solve_minimax", + "solve_minimax_from_coefs", + "compute_gradient", + ): + assert not hasattr(ellphi, name) diff --git a/tests/test_minimax.py b/tests/test_minimax.py new file mode 100644 index 0000000..dc2c29f --- /dev/null +++ b/tests/test_minimax.py @@ -0,0 +1,930 @@ +"""Tests for the many-body minimax solver. + +Adapted from uda-lab/ellcech ``tests/test_minimax.py`` at commit +82d13e3e174f4903cdcd6adcee1f469348361c91. + +Key validation: + For |σ| = 2, alpha(σ) must equal ellphi.tangency(p, q).t ** 2. + This cross-validates the Frank-Wolfe solver against ellphi's 1D root-finding, + which is the established reference implementation for the pairwise case. +""" + +from typing import get_args + +import numpy as np +import pytest + +import ellphi +import ellphi._minimax_python as minimax_mod +from ellphi.geometry import unpack_conic +from ellphi._minimax_python import solve_minimax, solve_minimax_from_coefs + +from .factories import random_coef_pair + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def _make_spd(rng, d): + """Return a random d×d SPD matrix.""" + L = rng.standard_normal((d, d)) + return L @ L.T + np.eye(d) * 0.5 + + +def _pairwise_alpha_ellphi(pcoef, qcoef): + """Ground truth: α({i,j}) = t² from ellphi.""" + return ellphi.tangency(pcoef, qcoef).t ** 2 + + +# --------------------------------------------------------------------------- +# Trivial cases +# --------------------------------------------------------------------------- + + +class TestTrivialCases: + def test_empty_simplex_raises(self): + with pytest.raises(ValueError, match="k=0"): + solve_minimax( + np.zeros((0, 2, 2)), + np.zeros((0, 2)), + ) + + def test_single_point_alpha_zero(self): + A = np.eye(2) + x = np.array([1.0, 2.0]) + res = solve_minimax(A[np.newaxis], x[np.newaxis]) + assert res.alpha == pytest.approx(0.0, abs=1e-12) + np.testing.assert_allclose(res.circumcenter, x) + assert res.converged + assert res.n_iter == 0 + + def test_single_point_3d(self): + rng = np.random.default_rng(0) + A = _make_spd(rng, 3) + x = rng.standard_normal(3) + res = solve_minimax(A[np.newaxis], x[np.newaxis]) + assert res.alpha == pytest.approx(0.0, abs=1e-12) + + def test_single_point_offset(self): + res = solve_minimax( + np.eye(2)[np.newaxis], + np.array([[1.0, 2.0]]), + offsets=np.array([0.75]), + ) + assert res.alpha == 0.75 + + def test_single_point_uses_stabilized_matrix(self): + res = solve_minimax( + np.eye(2)[np.newaxis], + np.array([[1.0, 0.0]]), + regularization=1.0, + ) + + assert res.alpha == pytest.approx(0.25) + np.testing.assert_allclose(res.circumcenter, [0.5, 0.0]) + assert res.converged + assert res.n_iter == 0 + + def test_single_point_nan_offset_raises(self): + with pytest.raises(ValueError, match="offsets must be finite"): + solve_minimax( + np.eye(2)[np.newaxis], + np.zeros((1, 2)), + offsets=np.array([np.nan]), + ) + + +# --------------------------------------------------------------------------- +# Isotropic (A_i = I) special cases +# --------------------------------------------------------------------------- + + +class TestIsotropic: + def test_two_points_midpoint(self): + """With A_i = I, x* is the midpoint of two equidistant points.""" + x0 = np.array([-1.0, 0.0]) + x1 = np.array([+1.0, 0.0]) + A = np.eye(2)[np.newaxis].repeat(2, axis=0) + centers = np.stack([x0, x1]) + res = solve_minimax(A, centers) + assert res.alpha == pytest.approx(1.0, rel=1e-6) + np.testing.assert_allclose(res.circumcenter, [0.0, 0.0], atol=1e-6) + assert res.converged + + def test_two_points_alpha_is_half_sq_distance(self): + """For A=I, α({i,j}) = ||x̄_i - x̄_j||² / 4 (midpoint equidistant).""" + rng = np.random.default_rng(42) + x0, x1 = rng.standard_normal((2, 3)) + A = np.eye(3)[np.newaxis].repeat(2, axis=0) + centers = np.stack([x0, x1]) + res = solve_minimax(A, centers) + expected = np.sum((x0 - x1) ** 2) / 4.0 + assert res.alpha == pytest.approx(expected, rel=1e-6) + + def test_three_points_circumcenter_equidistant(self): + """With A_i = I, x* is the circumcenter (equidistant from all points).""" + # Equilateral triangle + c = 2.0 * np.pi / 3.0 + x0 = np.array([1.0, 0.0]) + x1 = np.array([np.cos(c), np.sin(c)]) + x2 = np.array([np.cos(2 * c), np.sin(2 * c)]) + A = np.eye(2)[np.newaxis].repeat(3, axis=0) + centers = np.stack([x0, x1, x2]) + res = solve_minimax(A, centers, tol=1e-10) + # All f_i should be equal + diff = res.circumcenter[np.newaxis] - centers + f = np.einsum("ki,kij,kj->k", diff, A, diff) + assert np.max(f) - np.min(f) == pytest.approx(0.0, abs=1e-6) + assert res.converged + + def test_two_points_with_offset(self): + matrices = np.repeat(np.eye(2)[np.newaxis], 2, axis=0) + centers = np.array([[0.0, 0.0], [2.0, 0.0]]) + + res = solve_minimax(matrices, centers, offsets=np.array([1.0, 0.0])) + + assert res.converged + assert res.alpha == pytest.approx(1.5625, abs=1e-12) + np.testing.assert_allclose(res.circumcenter, [0.75, 0.0], atol=1e-12) + + +# --------------------------------------------------------------------------- +# Cross-validation with ellphi (pairwise case |σ| = 2) +# --------------------------------------------------------------------------- + + +class TestCrossValidationEllphi: + """For |σ| = 2, alpha must match ellphi.tangency(p, q).t**2.""" + + @pytest.mark.parametrize("seed", [0, 1, 2, 7, 42]) + def test_pairwise_random_2d(self, seed): + rng = np.random.default_rng(seed) + x0, x1 = rng.standard_normal((2, 2)) + cov0 = _make_spd(rng, 2) + cov1 = _make_spd(rng, 2) + pcoef = ellphi.coef_from_cov(x0, cov0)[0] + qcoef = ellphi.coef_from_cov(x1, cov1)[0] + coefs = np.stack([pcoef, qcoef]) + + res = solve_minimax_from_coefs(coefs, tol=1e-10) + expected = _pairwise_alpha_ellphi(pcoef, qcoef) + + assert res.alpha == pytest.approx( + expected, rel=1e-5 + ), f"seed={seed}: alpha={res.alpha:.8f}, expected={expected:.8f}" + assert res.converged, f"seed={seed}: did not converge in {res.n_iter} iters" + + @pytest.mark.parametrize("seed", [0, 1, 5]) + def test_pairwise_random_3d(self, seed): + rng = np.random.default_rng(seed + 100) + x0, x1 = rng.standard_normal((2, 3)) + cov0 = _make_spd(rng, 3) + cov1 = _make_spd(rng, 3) + pcoef = ellphi.coef_from_cov(x0, cov0)[0] + qcoef = ellphi.coef_from_cov(x1, cov1)[0] + coefs = np.stack([pcoef, qcoef]) + + res = solve_minimax_from_coefs(coefs, tol=1e-10) + expected = _pairwise_alpha_ellphi(pcoef, qcoef) + + assert res.alpha == pytest.approx( + expected, rel=1e-5 + ), f"seed={seed}: alpha={res.alpha:.8f}, expected={expected:.8f}" + + def test_pairwise_identical_ellipses(self): + """Two identical ellipses: α = 0, x* = center.""" + x = np.array([1.0, 2.0]) + cov = np.diag([0.5, 2.0]) + coef = ellphi.coef_from_cov(x, cov)[0] + coefs = np.stack([coef, coef]) + res = solve_minimax_from_coefs(coefs, tol=1e-10) + assert res.alpha == pytest.approx(0.0, abs=1e-7) + np.testing.assert_allclose(res.circumcenter, x, atol=1e-6) + + def test_pairwise_tangent_point_location(self): + """The circumcenter x* should satisfy f_i(x*) = f_j(x*) = alpha.""" + rng = np.random.default_rng(99) + x0, x1 = rng.standard_normal((2, 2)) + cov0 = _make_spd(rng, 2) + cov1 = _make_spd(rng, 2) + pcoef = ellphi.coef_from_cov(x0, cov0)[0] + qcoef = ellphi.coef_from_cov(x1, cov1)[0] + coefs = np.stack([pcoef, qcoef]) + + A_arr, b_arr, _ = unpack_conic(coefs) + centers = np.stack([-np.linalg.solve(A_arr[i], b_arr[i]) for i in range(2)]) + + res = solve_minimax(A_arr, centers, tol=1e-12) + xstar = res.circumcenter + diff = xstar[np.newaxis] - centers + f = np.einsum("ki,kij,kj->k", diff, A_arr, diff) + + assert f[0] == pytest.approx(f[1], rel=1e-5) + assert f[0] == pytest.approx(res.alpha, rel=1e-5) + + +# --------------------------------------------------------------------------- +# Higher-dimensional simplex (|σ| ≥ 3) +# --------------------------------------------------------------------------- + + +class TestHigherOrder: + def test_triangle_active_set(self): + """For a centered equilateral triangle, all 3 weights should be active.""" + c = 2.0 * np.pi / 3.0 + pts = np.array( + [ + [1.0, 0.0], + [np.cos(c), np.sin(c)], + [np.cos(2 * c), np.sin(2 * c)], + ] + ) + A = np.eye(2)[np.newaxis].repeat(3, axis=0) + res = solve_minimax(A, pts, tol=1e-10) + assert len(res.active_set) == 3 + assert res.converged + + def test_degenerate_triangle_reduces_to_edge(self): + """If one point is much closer, its weight should be near zero (inactive).""" + # x2 is very close to x* of {x0, x1}, so active set should be {0, 1} + pts = np.array([[0.0, 0.0], [2.0, 0.0], [1.0, 1e-6]]) + A = np.eye(2)[np.newaxis].repeat(3, axis=0) + res = solve_minimax(A, pts, tol=1e-8) + # alpha should be close to alpha({0, 1}) = 1.0 + assert res.alpha == pytest.approx(1.0, rel=1e-3) + + @pytest.mark.parametrize("k,d", [(3, 2), (4, 3), (5, 4)]) + def test_random_simplex_convergence(self, k, d): + """Frank-Wolfe should converge for random k-simplex in R^d.""" + rng = np.random.default_rng(k * 100 + d) + centers = rng.standard_normal((k, d)) + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + res = solve_minimax(matrices, centers, tol=1e-8, max_iter=5000) + assert res.converged, f"k={k}, d={d}: did not converge in {res.n_iter} iters" + assert res.alpha >= 0.0 + assert len(res.active_set) >= 1 + + +# --------------------------------------------------------------------------- +# Numerical stability controls (Gap F) +# --------------------------------------------------------------------------- + + +class TestNumericalStabilityControls: + @pytest.mark.parametrize("method", list(get_args(minimax_mod.MethodName))) + def test_zero_offsets_are_bitwise_identical(self, method): + matrices = np.repeat(np.eye(2)[np.newaxis], 3, axis=0) + centers = np.array([[0.0, 0.0], [2.0, 0.0], [0.5, 1.0]]) + + baseline = solve_minimax(matrices, centers, method=method) + explicit_zeros = solve_minimax( + matrices, centers, offsets=np.zeros(3), method=method + ) + + assert explicit_zeros.alpha == baseline.alpha + np.testing.assert_array_equal( + explicit_zeros.circumcenter, baseline.circumcenter + ) + np.testing.assert_array_equal(explicit_zeros.weights, baseline.weights) + assert explicit_zeros.converged == baseline.converged + + explicit_stability_zeros = solve_minimax( + matrices, + centers, + method=method, + regularization=0.0, + condition_number_limit=None, + max_conditioning_steps=8, + ) + assert explicit_stability_zeros.alpha == baseline.alpha + np.testing.assert_array_equal( + explicit_stability_zeros.circumcenter, baseline.circumcenter + ) + np.testing.assert_array_equal( + explicit_stability_zeros.weights, baseline.weights + ) + assert explicit_stability_zeros.converged == baseline.converged + + def test_explicit_default_controls_match_previous_behaviour(self): + rng = np.random.default_rng(321) + centers = rng.standard_normal((4, 3)) + matrices = np.stack([_make_spd(rng, 3) for _ in range(4)]) + + baseline = solve_minimax(matrices, centers, tol=1e-10) + with_explicit_defaults = solve_minimax( + matrices, + centers, + tol=1e-10, + regularization=0.0, + condition_number_limit=None, + max_conditioning_steps=8, + ) + + assert with_explicit_defaults.alpha == pytest.approx( + baseline.alpha, rel=1e-12, abs=1e-12 + ) + np.testing.assert_allclose( + with_explicit_defaults.circumcenter, + baseline.circumcenter, + atol=1e-12, + rtol=1e-12, + ) + np.testing.assert_allclose( + with_explicit_defaults.weights, + baseline.weights, + atol=1e-12, + rtol=1e-12, + ) + + def test_regularization_and_conditioning_produce_finite_solution(self): + matrices = np.array( + [ + [[1e-12, 0.0], [0.0, 1.0]], + [[1e12, 0.0], [0.0, 1.0]], + [[1.0, 0.0], [0.0, 1e-12]], + ] + ) + centers = np.array( + [ + [0.0, 0.0], + [1.0, 0.0], + [0.5, 1.0], + ] + ) + + res = solve_minimax( + matrices, + centers, + regularization=1e-8, + condition_number_limit=1e8, + max_conditioning_steps=4, + tol=1e-9, + max_iter=4000, + ) + assert np.isfinite(res.alpha) + assert np.all(np.isfinite(res.circumcenter)) + assert np.all(np.isfinite(res.weights)) + assert np.sum(res.weights) == pytest.approx(1.0, abs=1e-8) + + def test_stability_control_parameter_validation(self): + A = np.eye(2)[np.newaxis].repeat(2, axis=0) + centers = np.array([[0.0, 0.0], [1.0, 0.0]]) + with pytest.raises(ValueError): + solve_minimax(A, centers, regularization=-1.0) + with pytest.raises(ValueError): + solve_minimax(A, centers, condition_number_limit=1.0) + with pytest.raises(ValueError): + solve_minimax(A, centers, max_conditioning_steps=-1) + result = solve_minimax( + A, + centers, + condition_number_limit=2.0, + max_conditioning_steps=0, + ) + assert np.isfinite(result.alpha) + with pytest.raises(ValueError, match="offsets shape"): + solve_minimax(A, centers, offsets=np.zeros(3)) + + @pytest.mark.parametrize("value", [np.inf, np.nan, 0.0]) + def test_fw_tolerance_must_be_finite_and_positive(self, value): + A = np.eye(2)[np.newaxis].repeat(2, axis=0) + centers = np.array([[0.0, 0.0], [1.0, 0.0]]) + with pytest.raises(ValueError, match="tol must be finite and > 0"): + solve_minimax(A, centers, tol=value) + + @pytest.mark.parametrize("k", [2, 3, 4]) + @pytest.mark.parametrize("weight_tol", [None, 0.5]) + def test_weight_tol_cannot_empty_the_fw_face(self, k, weight_tol): + A = np.repeat(np.eye(2)[np.newaxis], k, axis=0) + centers = np.arange(2 * k, dtype=float).reshape(k, 2) + threshold = 1.0 / k if weight_tol is None else weight_tol + with pytest.raises(ValueError, match="weight_tol must satisfy"): + solve_minimax(A, centers, weight_tol=threshold) + + def test_fw_empty_face_rechecks_gap_after_last_update(self): + A = np.repeat(np.eye(2)[np.newaxis], 2, axis=0) + centers = np.array([[0.0, 0.0], [2.0, 0.0]]) + offsets = np.array([1.0, 0.0]) + Ax = np.einsum("kij,kj->ki", A, centers) + mu, converged, n_iter = minimax_mod._run_fw_bisect( + A, + Ax, + centers, + offsets, + np.full(2, 0.5), + tol=1e-12, + max_iter=1, + weight_tol=1.0, + regularization=0.0, + condition_number_limit=None, + max_conditioning_steps=8, + ) + + assert n_iter == 1 + assert converged + assert np.all(np.isfinite(mu)) + + def test_regularized_slsqp_objective_gradient_matches(self): + matrices = np.array( + [ + [[2.0, 0.2], [0.2, 1.0]], + [[1.0, -0.1], [-0.1, 3.0]], + [[1.5, 0.3], [0.3, 2.0]], + ] + ) + centers = np.array([[0.0, 0.0], [1.0, -0.5], [-0.5, 1.0]]) + Ax = np.einsum("kij,kj->ki", matrices, centers) + mu = np.array([0.2, 0.3, 0.5]) + objective, gradient = minimax_mod._slsqp_objective_and_gradient( + mu, + matrices, + Ax, + centers, + None, + regularization=0.25, + condition_number_limit=None, + max_conditioning_steps=8, + ) + step = 1e-6 + finite_difference = np.empty_like(mu) + for i in range(mu.size): + plus = mu.copy() + minus = mu.copy() + plus[i] += step + minus[i] -= step + plus_value, _ = minimax_mod._slsqp_objective_and_gradient( + plus, + matrices, + Ax, + centers, + None, + regularization=0.25, + condition_number_limit=None, + max_conditioning_steps=8, + ) + minus_value, _ = minimax_mod._slsqp_objective_and_gradient( + minus, + matrices, + Ax, + centers, + None, + regularization=0.25, + condition_number_limit=None, + max_conditioning_steps=8, + ) + finite_difference[i] = (plus_value - minus_value) / (2.0 * step) + + np.testing.assert_allclose(finite_difference, gradient, rtol=1e-6, atol=1e-8) + assert np.isfinite(objective) + + def test_conditioning_slsqp_objective_gradient_matches(self): + matrices = np.array( + [ + [[1e-8, 0.0], [0.0, 1.0]], + [[2e-8, 0.0], [0.0, 1.0]], + [[3e-8, 0.0], [0.0, 1.0]], + ] + ) + centers = np.array([[0.0, 0.0], [1.0, -0.5], [-0.5, 1.0]]) + Ax = np.einsum("kij,kj->ki", matrices, centers) + mu = np.array([0.2, 0.3, 0.5]) + kwargs = { + "regularization": 0.0, + "condition_number_limit": 1e4, + "max_conditioning_steps": 12, + } + objective, gradient = minimax_mod._slsqp_objective_and_gradient( + mu, matrices, Ax, centers, None, **kwargs + ) + step = 1e-6 + finite_difference = np.empty_like(mu) + for i in range(mu.size): + plus = mu.copy() + minus = mu.copy() + plus[i] += step + minus[i] -= step + plus_value, _ = minimax_mod._slsqp_objective_and_gradient( + plus, matrices, Ax, centers, None, **kwargs + ) + minus_value, _ = minimax_mod._slsqp_objective_and_gradient( + minus, matrices, Ax, centers, None, **kwargs + ) + finite_difference[i] = (plus_value - minus_value) / (2.0 * step) + + np.testing.assert_allclose(finite_difference, gradient, rtol=1e-6, atol=1e-8) + assert np.isfinite(objective) + + def test_fw_gap_uses_conditioning_shift_gradient(self): + matrices = np.array([[[1.0]], [[2.0]]]) + centers = np.array([[1e8], [2e8]]) + Ax = np.einsum("kij,kj->ki", matrices, centers) + mu = np.array([0.5, 0.5]) + kwargs = { + "regularization": 0.0, + "condition_number_limit": 1e6, + "max_conditioning_steps": 0, + } + + _, unadjusted_f = minimax_mod._eval_f(mu, matrices, Ax, centers, None, **kwargs) + offsets = np.array([0.0, unadjusted_f[0] - unadjusted_f[1]]) + xstar, f = minimax_mod._eval_f(mu, matrices, Ax, centers, offsets, **kwargs) + gradient = minimax_mod._dual_gradient( + mu, matrices, Ax, centers, offsets, xstar, f, **kwargs + ) + raw_gap = float(np.max(f) - np.dot(mu, f)) + stabilized_gap = float(np.max(gradient) - np.dot(mu, gradient)) + stabilized_dual = float(np.dot(mu, gradient)) + relative_tol = stabilized_gap / (100.0 * max(1.0, abs(stabilized_dual))) + + assert raw_gap < 1.0 + assert stabilized_gap > 1.0 + for method in ("fw+bisect", "fw+brentq"): + runner = ( + minimax_mod._run_fw_bisect + if method == "fw+bisect" + else minimax_mod._run_fw_brentq + ) + runner_result = runner( + matrices, + Ax, + centers, + offsets, + mu.copy(), + tol=relative_tol, + max_iter=1, + weight_tol=1e-10, + **kwargs, + ) + _, converged, n_iter = runner_result[:3] + assert n_iter == 1 + assert not converged + + def test_regularized_fw_gap_uses_actual_dual_value_scale(self): + scale = 1e10 + matrices = np.repeat(np.eye(2)[np.newaxis], 2, axis=0) + centers = np.array([[scale, 0.0], [scale + 2.0, 0.0]]) + offsets = np.full(2, -(scale**2 + 2.0 * scale + 3.0) / 2.0) + + result = solve_minimax( + matrices, + centers, + offsets=offsets, + method="fw+brentq", + regularization=1.0, + max_iter=1, + ) + + assert result.converged + np.testing.assert_allclose(result.weights, [0.0, 1.0]) + + def test_newton_polishing_uses_stabilized_stationarity(self): + matrices = np.array([np.eye(2), 4.0 * np.eye(2)]) + centers = np.array([[0.0, 0.0], [1.0, 0.0]]) + stabilized = solve_minimax( + matrices, + centers, + method="fw+bisect+newton", + regularization=1.0, + tol=1e-12, + ) + unadjusted = solve_minimax( + matrices, centers, method="fw+bisect+newton", tol=1e-12 + ) + Ax = np.einsum("kij,kj->ki", matrices, centers) + _, values = minimax_mod._eval_f( + stabilized.weights, + matrices, + Ax, + centers, + None, + regularization=1.0, + condition_number_limit=None, + max_conditioning_steps=8, + ) + + assert stabilized.converged + assert np.max(values) - stabilized.weights @ values < 1e-12 + assert stabilized.alpha == pytest.approx(4.0 / 9.0, abs=1e-12) + assert unadjusted.alpha == pytest.approx(stabilized.alpha, abs=1e-12) + assert not np.array_equal(stabilized.weights, unadjusted.weights) + + def test_newton_polishing_uses_conditioning_shift_gradient(self): + """The quasi-Newton polish meets the exact stabilised residual test. + + The mu-dependent conditioning shift is omitted from the Hessian, but + the residual uses the exact stabilised gradient. + """ + matrices = np.array([[[1.0]], [[2.0]]]) + centers = np.array([[1e8], [2e8]]) + Ax = np.einsum("kij,kj->ki", matrices, centers) + newton_tol = 1.0 + stability = { + "regularization": 0.0, + "condition_number_limit": 1e6, + "max_conditioning_steps": 0, + } + _, unadjusted_values = minimax_mod._eval_f( + np.array([0.5, 0.5]), + matrices, + Ax, + centers, + None, + **stability, + ) + offsets = np.array([0.0, unadjusted_values[0] - unadjusted_values[1]]) + mu, _, metadata = minimax_mod._damped_newton_polish( + np.array([0.5, 0.5]), + [0, 1], + matrices, + Ax, + centers, + offsets, + max_iter=2, + tol=newton_tol, + **stability, + ) + xstar, values = minimax_mod._eval_f( + mu, matrices, Ax, centers, offsets, **stability + ) + gradient = minimax_mod._dual_gradient( + mu, matrices, Ax, centers, offsets, xstar, values, **stability + ) + + assert metadata["newton_status"] == "converged" + assert abs(float(gradient[0] - gradient[1])) < newton_tol + + def test_fw_gap_includes_subthreshold_positive_weights(self): + matrices = np.repeat(np.eye(2)[np.newaxis], 3, axis=0) + centers = np.array([[2.0, 0.0], [0.0, 2.0], [0.0, 0.0]]) + Ax = np.einsum("kij,kj->ki", matrices, centers) + mu = np.array([0.405, 0.405, 0.19]) + + _, converged, n_iter = minimax_mod._run_fw_bisect( + matrices, + Ax, + centers, + None, + mu, + tol=1e-12, + max_iter=1, + weight_tol=0.2, + regularization=0.0, + condition_number_limit=None, + max_conditioning_steps=8, + ) + + assert n_iter == 1 + assert not converged + + def test_newton_rechecks_global_gap_beyond_weight_face(self): + matrices = np.repeat(np.eye(2)[np.newaxis], 3, axis=0) + centers = np.array([[-1.0, 0.0], [1.0, 0.0], [0.0, 3.0]]) + weights = np.array([0.5, 0.5, 0.0]) + Ax = np.einsum("kij,kj->ki", matrices, centers) + + converged = minimax_mod._check_newton_convergence( + weights, + matrices, + Ax, + centers, + None, + "converged", + tol=1e-3, + regularization=0.0, + condition_number_limit=None, + max_conditioning_steps=8, + ) + + assert converged is False + + def test_newton_acceptance_uses_relative_gap_without_extra_residual_factor(self): + matrices = np.repeat(np.eye(1)[np.newaxis], 2, axis=0) + centers = np.array([[0.0], [1.0]]) + Ax = np.einsum("kij,kj->ki", matrices, centers) + weights = np.array([0.999, 0.001]) + + converged = minimax_mod._check_newton_convergence( + weights, + matrices, + Ax, + centers, + np.array([1.0, 0.0]), + "converged", + tol=0.01, + regularization=0.0, + condition_number_limit=None, + max_conditioning_steps=8, + ) + + assert converged is True + + def test_factorization_failure_is_reported_as_nonconvergence(self, monkeypatch): + matrices = np.repeat(np.eye(2)[np.newaxis], 2, axis=0) + centers = np.array([[0.0, 0.0], [1.0, 0.0]]) + + def fail_factorization(*args, **kwargs): + raise np.linalg.LinAlgError("test factorization failure") + + monkeypatch.setattr(minimax_mod.linalg, "cho_factor", fail_factorization) + result = solve_minimax(matrices, centers, method="fw+bisect") + + assert not result.converged + assert result.metadata == {"solver_status": "factorization_failed"} + + def test_hybrid_rechecks_fw_optimum_hit_on_last_iteration(self): + matrices = np.array([np.eye(2), 4.0 * np.eye(2)]) + centers = np.array([[0.0, 0.0], [1.0, 0.0]]) + reference = solve_minimax( + matrices, + centers, + method="fw+bisect", + tol=1e-12, + max_iter=2, + ) + assert reference.converged + last_update = reference.n_iter - 1 + + result = solve_minimax( + matrices, + centers, + method="fw+bisect+newton", + tol=1e-12, + max_iter=last_update, + ) + + assert result.converged + assert result.alpha == reference.alpha + np.testing.assert_allclose(result.weights, reference.weights, atol=1e-15) + + def test_slsqp_converges_with_regularization(self): + matrices = np.array( + [ + [[2.0, 0.2], [0.2, 1.0]], + [[1.0, -0.1], [-0.1, 3.0]], + [[1.5, 0.3], [0.3, 2.0]], + ] + ) + centers = np.array([[0.0, 0.0], [1.0, -0.5], [-0.5, 1.0]]) + result = solve_minimax( + matrices, + centers, + method="scipy-slsqp", + regularization=0.25, + ) + + assert result.converged + assert np.isfinite(result.alpha) + assert np.all(np.isfinite(result.weights)) + assert result.weights.sum() == pytest.approx(1.0) + + def test_slsqp_zero_regularization_objective_path_is_unchanged(self): + matrices = np.repeat(np.eye(2)[np.newaxis], 2, axis=0) + centers = np.array([[0.0, 0.0], [2.0, 0.0]]) + Ax = np.einsum("kij,kj->ki", matrices, centers) + mu = np.array([0.25, 0.75]) + objective, gradient = minimax_mod._slsqp_objective_and_gradient( + mu, + matrices, + Ax, + centers, + None, + regularization=0.0, + condition_number_limit=None, + max_conditioning_steps=8, + ) + _, f = minimax_mod._eval_f( + mu, + matrices, + Ax, + centers, + None, + regularization=0.0, + condition_number_limit=None, + max_conditioning_steps=8, + ) + + assert objective == -float(np.dot(mu, f)) + np.testing.assert_array_equal(gradient, -f) + + def test_conditioning_steps_cap_is_strict(self): + A = np.diag([1e-12, 1.0]) + eye = np.eye(2) + limited0 = minimax_mod._condition_matrix( + A, + regularization=1e-6, + condition_number_limit=2.0, + max_conditioning_steps=0, + ) + limited1 = minimax_mod._condition_matrix( + A, + regularization=1e-6, + condition_number_limit=2.0, + max_conditioning_steps=1, + ) + + np.testing.assert_allclose(limited0, A + 1e-6 * eye) + np.testing.assert_allclose(limited1, A + 1e-5 * eye) + + def test_from_coefs_matches_direct_minimax_with_non_default_stability(self): + rng = np.random.default_rng(77) + x0, x1 = rng.standard_normal((2, 2)) + cov0 = _make_spd(rng, 2) + cov1 = _make_spd(rng, 2) + pcoef = ellphi.coef_from_cov(x0, cov0)[0] + qcoef = ellphi.coef_from_cov(x1, cov1)[0] + coefs = np.stack([pcoef, qcoef]) + + A_arr, b_arr, _ = unpack_conic(coefs) + centers = np.stack([-np.linalg.solve(A_arr[i], b_arr[i]) for i in range(2)]) + + kwargs = { + "regularization": 1e-6, + "condition_number_limit": 1e6, + "max_conditioning_steps": 2, + "tol": 1e-10, + } + direct = solve_minimax(A_arr, centers, **kwargs) + from_coefs = solve_minimax_from_coefs(coefs, **kwargs) + + assert from_coefs.alpha == pytest.approx(direct.alpha, rel=1e-10, abs=1e-12) + np.testing.assert_allclose( + from_coefs.circumcenter, + direct.circumcenter, + rtol=1e-10, + atol=1e-12, + ) + np.testing.assert_allclose( + from_coefs.weights, direct.weights, rtol=1e-10, atol=1e-12 + ) + + +# --------------------------------------------------------------------------- +# Pairwise compatibility with ellphi.tangency (both backends) +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize("dim", [2, 3]) +def test_pairwise_alpha_is_squared_tangency_time(solver_backend, rng, dim): + """For k = 2, ``solve_minimax(...).alpha == tangency(p, q).t ** 2``.""" + for _ in range(5): + pcoef, qcoef = random_coef_pair(rng, dim=dim) + t = ellphi.tangency(pcoef, qcoef, backend=solver_backend).t + + A_arr, b_arr, _ = unpack_conic(np.stack([pcoef, qcoef])) + centers = np.stack([-np.linalg.solve(A_arr[i], b_arr[i]) for i in range(2)]) + res = solve_minimax(A_arr, centers) + + assert res.converged + assert res.alpha == pytest.approx(t**2, rel=1e-9) + assert res.weights.sum() == pytest.approx(1.0, abs=1e-12) + + +@pytest.mark.parametrize("dim", [2, 3]) +def test_unnormalized_pairwise_coefs_match_tangency(solver_backend, rng, dim): + """General packed constants agree with pairwise tangency.""" + for _ in range(5): + pcoef, qcoef = random_coef_pair(rng, dim=dim) + coefs = np.stack([pcoef, qcoef]) + coefs[:, -1] += rng.uniform(0.1, 1.0, size=2) + + pairwise = ellphi.tangency(coefs[0], coefs[1], backend=solver_backend) + res = solve_minimax_from_coefs(coefs, method="scipy-slsqp", tol=1e-11) + + assert res.converged + assert res.alpha == pytest.approx(pairwise.t**2, rel=1e-9) + np.testing.assert_allclose( + res.circumcenter, pairwise.point, rtol=1e-9, atol=1e-10 + ) + + +# --------------------------------------------------------------------------- +# Regression fixture: weight support versus tight constraint set +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize("method", list(get_args(minimax_mod.MethodName))) +def test_right_triangle_support_differs_from_tight_constraint_set(method): + """Support-vs-tight-set fixture: three unit balls on a right triangle. + + The centers ``(2, 0)``, ``(0, 2)`` and ``(0, 0)`` give ``alpha = 2`` at the + circumcenter ``(1, 1)``. All three constraints are tight there, but the + unique optimal weights are ``(1/2, 1/2, 0)``, so ``active_set`` (the + weight support) has two elements while the tight set has three. + """ + centers = np.array([[2.0, 0.0], [0.0, 2.0], [0.0, 0.0]]) + matrices = np.repeat(np.eye(2)[np.newaxis], 3, axis=0) + + res = solve_minimax(matrices, centers, method=method) + + assert res.converged + assert res.alpha == pytest.approx(2.0, rel=1e-12) + np.testing.assert_allclose(res.circumcenter, [1.0, 1.0], atol=1e-12) + np.testing.assert_allclose(res.weights, [0.5, 0.5, 0.0], atol=1e-10) + assert res.active_set == [0, 1] + + diff = res.circumcenter[np.newaxis] - centers + f = np.einsum("ki,kij,kj->k", diff, matrices, diff) + tight = [i for i in range(3) if abs(f[i] - res.alpha) <= 1e-12 * res.alpha] + assert tight == [0, 1, 2] diff --git a/tests/test_minimax_grad.py b/tests/test_minimax_grad.py new file mode 100644 index 0000000..06c10ec --- /dev/null +++ b/tests/test_minimax_grad.py @@ -0,0 +1,343 @@ +"""Tests for the minimax gradient with respect to centers and metric matrices. + +Adapted from uda-lab/ellcech ``tests/test_minimax_grad.py`` at commit +82d13e3e174f4903cdcd6adcee1f469348361c91. + +Validation strategy: finite-difference check with relative error < 1e-5. + +The gradient formulas: + d alpha / d xbar_k = 2 mu_k* A_k (xbar_k - x*) + d alpha / d A_k = mu_k* outer(xbar_k - x*, xbar_k - x*) +""" + +from __future__ import annotations + +import numpy as np +import pytest + +from ellphi._minimax_grad_python import GradientResult, compute_gradient +from ellphi._minimax_python import solve_minimax + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def _make_spd(rng: np.random.Generator, d: int, scale: float = 1.0) -> np.ndarray: + """Return a random d×d SPD matrix.""" + L = rng.standard_normal((d, d)) + return scale * (L @ L.T) + np.eye(d) * 0.5 + + +def _solve_alpha(matrices: np.ndarray, centers: np.ndarray) -> float: + """Compute alpha via fw+brentq+newton (recommended method).""" + res = solve_minimax(matrices, centers, method="fw+brentq+newton") + return res.alpha + + +def _fd_grad_xbar(matrices, centers, k, h=1e-5): + """Finite-difference gradient of alpha w.r.t. centers[k], shape (d,).""" + d = centers.shape[1] + grad = np.zeros(d) + for j in range(d): + c_plus = centers.copy() + c_plus[k, j] += h + c_minus = centers.copy() + c_minus[k, j] -= h + alpha_plus = _solve_alpha(matrices, c_plus) + alpha_minus = _solve_alpha(matrices, c_minus) + grad[j] = (alpha_plus - alpha_minus) / (2 * h) + return grad + + +def _fd_grad_A(matrices, centers, k, h=1e-6): + """Finite-difference gradient of alpha w.r.t. matrices[k], shape (d,d).""" + d = centers.shape[1] + grad = np.zeros((d, d)) + for i in range(d): + for j in range(d): + m_plus = matrices.copy() + m_minus = matrices.copy() + # Symmetric perturbation + m_plus[k, i, j] += h + m_plus[k, j, i] += h + m_minus[k, i, j] -= h + m_minus[k, j, i] -= h + # Ensure still SPD (perturbation is small) + alpha_plus = _solve_alpha(m_plus, centers) + alpha_minus = _solve_alpha(m_minus, centers) + grad[i, j] = (alpha_plus - alpha_minus) / (4 * h) # factor 2 from symmetry + return grad + + +def _rel_err(analytical, fd): + """Relative error between analytical and finite-difference gradients.""" + denom = max(np.linalg.norm(fd), 1e-10) + return float(np.linalg.norm(analytical - fd) / denom) + + +# --------------------------------------------------------------------------- +# Basic functionality tests +# --------------------------------------------------------------------------- + + +class TestGradientResult: + def test_returns_gradient_result(self): + rng = np.random.default_rng(0) + d, k = 2, 3 + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + centers = rng.standard_normal((k, d)) + result = solve_minimax(matrices, centers, method="fw+brentq+newton") + grad = compute_gradient(result, centers, matrices) + assert isinstance(grad, GradientResult) + + def test_output_shapes(self): + rng = np.random.default_rng(1) + d, k = 3, 4 + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + centers = rng.standard_normal((k, d)) + result = solve_minimax(matrices, centers, method="fw+brentq+newton") + grad = compute_gradient(result, centers, matrices) + assert len(grad.d_xbar) == k + assert len(grad.d_A) == k + assert grad.d_offsets.shape == (k,) + for i in range(k): + assert grad.d_xbar[i].shape == (d,) + assert grad.d_A[i].shape == (d, d) + + def test_passive_vertices_zero(self): + """Inactive vertices (mu_k=0) should have zero gradient.""" + rng = np.random.default_rng(2) + d = 2 + # k=2 pairwise: both vertices are active. Test k=4 where some may be inactive. + k = 4 + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + centers = rng.standard_normal((k, d)) + result = solve_minimax(matrices, centers, method="fw+brentq+newton") + grad = compute_gradient(result, centers, matrices) + for i in range(k): + if result.weights[i] < 1e-10: + np.testing.assert_allclose(grad.d_xbar[i], 0.0, atol=1e-14) + np.testing.assert_allclose(grad.d_A[i], 0.0, atol=1e-14) + assert grad.d_offsets[i] == pytest.approx(0.0, abs=1e-14) + + def test_d_A_symmetric(self): + """d alpha / d A_k = mu_k outer(diff, diff) is symmetric.""" + rng = np.random.default_rng(3) + d, k = 3, 3 + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + centers = rng.standard_normal((k, d)) + result = solve_minimax(matrices, centers, method="fw+brentq+newton") + grad = compute_gradient(result, centers, matrices) + for i in range(k): + np.testing.assert_allclose(grad.d_A[i], grad.d_A[i].T, atol=1e-15) + + def test_alpha_matches_result(self): + rng = np.random.default_rng(4) + d, k = 2, 2 + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + centers = rng.standard_normal((k, d)) + result = solve_minimax(matrices, centers, method="fw+brentq+newton") + grad = compute_gradient(result, centers, matrices) + assert grad.alpha == pytest.approx(result.alpha, rel=1e-12) + + def test_weights_match_result(self): + rng = np.random.default_rng(5) + d, k = 2, 2 + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + centers = rng.standard_normal((k, d)) + result = solve_minimax(matrices, centers, method="fw+brentq+newton") + grad = compute_gradient(result, centers, matrices) + np.testing.assert_allclose(grad.weights, result.weights, atol=1e-15) + np.testing.assert_array_equal(grad.d_offsets, result.weights) + + def test_zero_offsets_leave_existing_gradients_unchanged(self): + rng = np.random.default_rng(6) + d, k = 2, 3 + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + centers = rng.standard_normal((k, d)) + + baseline = solve_minimax(matrices, centers, method="scipy-slsqp") + with_zeros = solve_minimax( + matrices, + centers, + offsets=np.zeros(k), + method="scipy-slsqp", + ) + baseline_grad = compute_gradient(baseline, centers, matrices) + zero_grad = compute_gradient(with_zeros, centers, matrices) + + for actual, expected in zip(zero_grad.d_xbar, baseline_grad.d_xbar): + np.testing.assert_array_equal(actual, expected) + for actual, expected in zip(zero_grad.d_A, baseline_grad.d_A): + np.testing.assert_array_equal(actual, expected) + np.testing.assert_array_equal(zero_grad.d_offsets, baseline_grad.d_offsets) + + +# --------------------------------------------------------------------------- +# Finite-difference validation tests +# --------------------------------------------------------------------------- + + +class TestFiniteDifferenceValidation: + """Validate analytical gradients against finite differences. + + Target: relative error < 1e-5 for all cases. + """ + + @pytest.mark.parametrize( + "k,d,seed", + [ + (2, 2, 10), + (2, 3, 11), + (3, 2, 12), + (3, 3, 13), + (4, 2, 14), + ], + ) + def test_grad_xbar_fd(self, k, d, seed): + """d alpha / d xbar_k matches finite differences.""" + rng = np.random.default_rng(seed) + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + centers = rng.standard_normal((k, d)) + + result = solve_minimax(matrices, centers, method="fw+brentq+newton") + grad = compute_gradient(result, centers, matrices) + + for i in range(k): + if result.weights[i] < 1e-8: + continue # skip genuinely inactive + fd = _fd_grad_xbar(matrices, centers, i) + err = _rel_err(grad.d_xbar[i], fd) + assert err < 1e-5, ( + f"k={k},d={d},i={i}: d_xbar rel_err={err:.2e} " + f"analytical={grad.d_xbar[i]}, fd={fd}" + ) + + @pytest.mark.parametrize( + "k,d,seed", + [ + (2, 2, 20), + (2, 3, 21), + (3, 2, 22), + ], + ) + def test_grad_A_fd(self, k, d, seed): + """d alpha / d A_k matches finite differences.""" + rng = np.random.default_rng(seed) + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + centers = rng.standard_normal((k, d)) + + result = solve_minimax(matrices, centers, method="fw+brentq+newton") + grad = compute_gradient(result, centers, matrices) + + for i in range(k): + if result.weights[i] < 1e-8: + continue # skip inactive + fd = _fd_grad_A(matrices, centers, i) + err = _rel_err(grad.d_A[i], fd) + assert err < 1e-5, f"k={k},d={d},i={i}: d_A rel_err={err:.2e}" + + def test_pairwise_matches_ellphi_direction(self): + """For k=2, gradient direction should be consistent with ellphi's formula.""" + rng = np.random.default_rng(30) + d = 3 + matrices = np.stack([_make_spd(rng, d) for _ in range(2)]) + centers = rng.standard_normal((2, d)) + + result = solve_minimax(matrices, centers, method="fw+brentq+newton") + grad = compute_gradient(result, centers, matrices) + + # Verify against FD + for i in range(2): + fd = _fd_grad_xbar(matrices, centers, i) + err = _rel_err(grad.d_xbar[i], fd) + assert err < 1e-5, f"pairwise xbar[{i}] rel_err={err:.2e}" + + def test_offset_gradient_matches_finite_differences(self): + rng = np.random.default_rng(31) + d, k = 2, 3 + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + centers = rng.standard_normal((k, d)) + offsets = np.array([0.2, 0.5, 0.1]) + result = solve_minimax(matrices, centers, offsets=offsets, method="scipy-slsqp") + grad = compute_gradient(result, centers, matrices) + h = 1e-6 + + finite_difference = np.empty(k) + for i in range(k): + plus = offsets.copy() + minus = offsets.copy() + plus[i] += h + minus[i] -= h + alpha_plus = solve_minimax( + matrices, centers, offsets=plus, method="scipy-slsqp" + ).alpha + alpha_minus = solve_minimax( + matrices, centers, offsets=minus, method="scipy-slsqp" + ).alpha + finite_difference[i] = (alpha_plus - alpha_minus) / (2.0 * h) + + np.testing.assert_allclose( + grad.d_offsets, finite_difference, rtol=1e-5, atol=1e-7 + ) + + +# --------------------------------------------------------------------------- +# Input validation tests +# --------------------------------------------------------------------------- + + +class TestInputValidation: + def test_shape_mismatch_raises(self): + rng = np.random.default_rng(40) + d, k = 2, 3 + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + centers = rng.standard_normal((k, d)) + result = solve_minimax(matrices, centers, method="fw+brentq+newton") + # Wrong: pass centers with k+1 rows but same matrices (k rows) → shape mismatch + bad_centers = rng.standard_normal((k + 1, d)) + bad_matrices = np.stack([_make_spd(rng, d) for _ in range(k + 1)]) + with pytest.raises(ValueError, match="weights length"): + compute_gradient(result, bad_centers, bad_matrices) + + def test_single_point_zero_gradient(self): + """Single-vertex simplex: circumcenter = xbar, weights=[1], grad=0.""" + rng = np.random.default_rng(41) + d = 2 + A = _make_spd(rng, d)[np.newaxis] + center = rng.standard_normal((1, d)) + result = solve_minimax(A, center, method="fw+brentq+newton") + grad = compute_gradient(result, center, A) + # xstar = xbar, so diff = 0, grad = 0 + np.testing.assert_allclose(grad.d_xbar[0], 0.0, atol=1e-14) + np.testing.assert_allclose(grad.d_A[0], 0.0, atol=1e-14) + + def test_all_methods_give_same_gradient(self): + """Gradient should be consistent across solver methods.""" + rng = np.random.default_rng(42) + d, k = 2, 3 + matrices = np.stack([_make_spd(rng, d) for _ in range(k)]) + centers = rng.standard_normal((k, d)) + + methods = ["fw+bisect", "fw+brentq", "fw+bisect+newton", "fw+brentq+newton"] + grads = [] + for method in methods: + result = solve_minimax(matrices, centers, method=method) + grad = compute_gradient(result, centers, matrices) + grads.append(grad) + + # All methods should agree to ~1e-6 + ref = grads[0] + for g in grads[1:]: + for i in range(k): + np.testing.assert_allclose( + g.d_xbar[i], + ref.d_xbar[i], + atol=1e-6, + err_msg=f"d_xbar[{i}] mismatch", + ) + np.testing.assert_allclose( + g.d_A[i], ref.d_A[i], atol=1e-6, err_msg=f"d_A[{i}] mismatch" + ) diff --git a/tests/test_minimax_methods.py b/tests/test_minimax_methods.py new file mode 100644 index 0000000..88761c3 --- /dev/null +++ b/tests/test_minimax_methods.py @@ -0,0 +1,556 @@ +"""Tests for multi-method dispatch in the minimax solver. + +Adapted from uda-lab/ellcech ``tests/test_minimax_methods.py`` at commit +82d13e3e174f4903cdcd6adcee1f469348361c91. The benchmark smoke tests are not +ported because the benchmark module is not part of ellphi. + +Methods that converge must agree on alpha to tight tolerances. +fw+bisect is the reference (validated against ellphi in test_minimax.py). +""" + +from __future__ import annotations + +import warnings +from typing import get_args + +import numpy as np +import pytest +import scipy.optimize + +from ellphi._minimax_python import MethodName, MinimaxResult, solve_minimax + +from .factories import minimax_public_surrogate, random_simplex + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def _random_simplex(k: int, d: int, seed: int = 0): + return random_simplex(k, d, rng=np.random.default_rng(seed)) + + +ALL_METHODS: list[str] = list(get_args(MethodName)) + +CANONICAL_METHODS = ( + "fw+bisect", + "fw+brentq", + "fw+bisect+newton", + "fw+brentq+newton", + "fw+bisect+damped-newton", + "scipy-slsqp", + "newton-cold", +) + +SURROGATE_ALPHA = 0.9024444260258915 + + +def _surrogate_diagnostics(result, matrices, centers): + diff = result.circumcenter[np.newaxis, :] - centers + values = np.einsum("ki,kij,kj->k", diff, matrices, diff) + gap = float(np.max(values) - result.weights @ values) + stationarity = np.einsum("k,kij,kj->i", result.weights, matrices, diff) + residual = float(np.linalg.norm(stationarity, ord=np.inf)) + return gap, residual + + +def _constraint_gap(result, matrices, centers): + diff = result.circumcenter[np.newaxis, :] - centers + values = np.einsum("ki,kij,kj->k", diff, matrices, diff) + return float(np.max(values) - result.weights @ values) + + +@pytest.fixture(scope="module") +def public_surrogate_results(): + matrices, centers = minimax_public_surrogate() + results = { + method: solve_minimax(matrices, centers, method=method) + for method in ("fw+bisect", "fw+brentq", "scipy-slsqp") + } + return matrices, centers, results + + +def test_public_surrogate_recipe_matches_literals(): + rng = np.random.default_rng(10602) + expected_matrices = np.empty((6, 2, 2)) + expected_centers = np.empty((6, 2)) + for instance in range(13): + matrices = np.empty((6, 2, 2)) + for row in range(6): + q = np.linalg.qr(rng.standard_normal((2, 2)))[0] + eigenvalues = np.exp(rng.uniform(-1.0, 1.0, size=2)) + matrices[row] = (q * eigenvalues) @ q.T + centers = rng.standard_normal((6, 2)) + + if instance == 12: + expected_matrices = matrices + expected_centers = centers + + matrices, centers = minimax_public_surrogate() + np.testing.assert_allclose(matrices, expected_matrices, rtol=1e-15, atol=1e-15) + np.testing.assert_array_equal(centers, expected_centers) + + +class TestPublicSurrogate: + def test_slsqp_reference_value_and_gap(self, public_surrogate_results): + matrices, centers, results = public_surrogate_results + result = results["scipy-slsqp"] + gap, residual = _surrogate_diagnostics(result, matrices, centers) + + assert result.converged + assert abs(result.alpha - SURROGATE_ALPHA) <= 1e-8 + assert gap <= 1e-8 + assert residual <= 1e-8 + + def test_pairwise_fw_converges_with_large_budget(self): + matrices, centers = minimax_public_surrogate() + result = solve_minimax( + matrices, + centers, + method="fw+brentq", + max_iter=20000, + ) + + assert result.converged + assert result.n_iter > 2000 + assert result.metadata is not None + assert abs(result.alpha - SURROGATE_ALPHA) <= 1e-8 + + def test_default_budget_reports_pairwise_fw_newton_nonconvergence(self): + matrices, centers = minimax_public_surrogate() + result = solve_minimax(matrices, centers) + + assert not result.converged + assert result.method == "fw+brentq+newton" + assert result.metadata is not None + + def test_pairwise_fw_swaps_only_best_and_worst_weights(self): + """One pairwise FW step changes two coordinates and preserves their mass.""" + matrices = np.repeat(np.eye(1)[np.newaxis, :, :], 3, axis=0) + centers = np.array([[0.0], [1.0], [3.0]]) + initial = np.full(3, 1.0 / 3.0) + + result = solve_minimax( + matrices, + centers, + method="fw+bisect", + max_iter=1, + tol=1e-30, + ) + + changed = np.flatnonzero(result.weights != initial) + assert changed.size == 2 + assert result.weights[changed].sum() == pytest.approx(initial[changed].sum()) + + +# --------------------------------------------------------------------------- +# Basic dispatch / API +# --------------------------------------------------------------------------- + + +class TestMethodDispatch: + def test_exactly_seven_canonical_methods(self): + assert tuple(ALL_METHODS) == CANONICAL_METHODS + assert "fw+newton" not in ALL_METHODS + + def test_invalid_method_raises(self): + A = np.eye(2)[np.newaxis] + x = np.zeros((1, 2)) + with pytest.raises(ValueError, match="Unknown method"): + solve_minimax(A, x, method="not-a-method") # type: ignore[arg-type] + + @pytest.mark.parametrize("method", ALL_METHODS) + def test_result_has_method_field(self, method): + matrices, centers = _random_simplex(3, 2, seed=7) + res = solve_minimax(matrices, centers, method=method) + assert res.method == method + + @pytest.mark.parametrize("method", ALL_METHODS) + def test_single_point_trivial(self, method): + A = np.eye(3)[np.newaxis] + x = np.array([[1.0, 2.0, 3.0]]) + res = solve_minimax(A, x, method=method) + assert res.alpha == pytest.approx(0.0, abs=1e-12) + assert res.converged + + @pytest.mark.parametrize("method", ALL_METHODS) + def test_k0_raises(self, method): + with pytest.raises(ValueError, match="k=0"): + solve_minimax(np.zeros((0, 2, 2)), np.zeros((0, 2)), method=method) + + def test_default_method_is_fw_brentq_newton(self): + matrices, centers = _random_simplex(2, 2) + res = solve_minimax(matrices, centers) + assert res.method == "fw+brentq+newton" + + @pytest.mark.parametrize("newton_tol", [np.nan, 0.0]) + def test_non_positive_or_non_finite_newton_tol_raises(self, newton_tol): + matrices, centers = _random_simplex(2, 2) + with pytest.raises(ValueError, match="newton_tol must be finite and > 0"): + solve_minimax(matrices, centers, newton_tol=newton_tol) + + +# --------------------------------------------------------------------------- +# Numerical agreement across methods +# --------------------------------------------------------------------------- + + +class TestMethodAgreement: + """All methods must agree on alpha to within reasonable tolerances.""" + + @pytest.mark.parametrize("seed", [0, 1, 2, 3, 4]) + def test_pairwise_k2_d2(self, seed): + """k=2: all methods should agree to 1e-7 with fw+bisect reference.""" + matrices, centers = _random_simplex(2, 2, seed=seed) + ref = solve_minimax(matrices, centers, method="fw+bisect", tol=1e-12) + for m in ALL_METHODS: + if m == "fw+bisect": + continue + res = solve_minimax(matrices, centers, method=m) + assert res.alpha == pytest.approx( + ref.alpha, rel=1e-7 + ), f"method={m}, seed={seed}: alpha={res.alpha}, ref={ref.alpha}" + + @pytest.mark.parametrize( + "k,d,seed", + [ + (3, 2, 10), + (4, 3, 11), + (5, 3, 12), + (3, 5, 13), + ], + ) + def test_general_k_d(self, k, d, seed): + """General k: all methods (except newton-cold) agree to 1e-5.""" + matrices, centers = _random_simplex(k, d, seed=seed) + ref = solve_minimax( + matrices, centers, method="fw+bisect", tol=1e-11, max_iter=5000 + ) + for m in ALL_METHODS: + if m in ("fw+bisect", "newton-cold"): + continue + res = solve_minimax(matrices, centers, method=m) + assert res.alpha == pytest.approx(ref.alpha, rel=1e-5), ( + f"method={m}, k={k}, d={d}, seed={seed}: " + f"alpha={res.alpha}, ref={ref.alpha}" + ) + + +# --------------------------------------------------------------------------- +# fw+brentq specific +# --------------------------------------------------------------------------- + + +class TestFwBrentq: + @pytest.mark.parametrize("seed", range(5)) + def test_alpha_agrees_with_fw_bisect(self, seed): + """fw+brentq alpha should match fw+bisect to 1e-7.""" + matrices, centers = _random_simplex(4, 3, seed=seed) + ref = solve_minimax(matrices, centers, method="fw+bisect", tol=1e-11) + res = solve_minimax(matrices, centers, method="fw+brentq", tol=1e-11) + assert res.alpha == pytest.approx( + ref.alpha, rel=1e-7 + ), f"seed={seed}: brentq={res.alpha}, bisect={ref.alpha}" + + def test_metadata_has_line_search_evals(self): + matrices, centers = _random_simplex(4, 3, seed=42) + res = solve_minimax(matrices, centers, method="fw+brentq") + assert res.metadata is not None + assert "line_search_evals" in res.metadata + assert res.metadata["line_search_evals"] > 0 + + def test_brentq_bisection_safeguard_is_recorded(self, monkeypatch): + matrices = np.repeat(np.eye(2)[np.newaxis], 3, axis=0) + centers = np.array([[2.0, 0.0], [0.0, 2.0], [0.0, 0.0]]) + + def reject_brentq(*args, **kwargs): + raise ValueError("test bracket rejection") + + monkeypatch.setattr(scipy.optimize, "brentq", reject_brentq) + result = solve_minimax( + matrices, + centers, + method="fw+brentq", + max_iter=2, + tol=1e-30, + ) + + assert result.metadata is not None + assert result.metadata["line_search_bisection_safeguards"] > 0 + + def test_relative_gap_is_scale_invariant(self): + matrices, centers = _random_simplex(4, 2, seed=0) + scaled_matrices = matrices * 1e-8 + scaled_centers = centers * 1e8 + + result = solve_minimax(matrices, centers, method="fw+brentq") + scaled = solve_minimax(scaled_matrices, scaled_centers, method="fw+brentq") + + assert result.converged + assert scaled.converged + assert result.active_set == scaled.active_set + assert scaled.alpha == pytest.approx(result.alpha * 1e8, rel=1e-9) + + def test_large_alpha_uses_relative_gap(self): + matrices, centers = _random_simplex(4, 2, seed=0) + scaled_matrices = matrices * 1e-8 + scaled_centers = centers * 1e8 + scaled = solve_minimax(scaled_matrices, scaled_centers, method="fw+brentq") + + gap = _constraint_gap(scaled, scaled_matrices, scaled_centers) + assert scaled.converged + assert gap <= 1e-9 * max(1.0, abs(scaled.alpha)) + + old_absolute_accuracy = solve_minimax( + scaled_matrices, + scaled_centers, + method="fw+brentq", + tol=1e-9 / scaled.alpha, + ) + assert not old_absolute_accuracy.converged + + +# --------------------------------------------------------------------------- +# fw+brentq+newton specific +# --------------------------------------------------------------------------- + + +class TestFwBrentqNewton: + @pytest.mark.parametrize("seed", range(5)) + def test_alpha_precision(self, seed): + """fw+brentq+newton should achieve high precision.""" + matrices, centers = _random_simplex(4, 3, seed=seed) + ref = solve_minimax( + matrices, + centers, + method="fw+bisect", + tol=1e-14, + max_iter=10000, + ) + res = solve_minimax(matrices, centers, method="fw+brentq+newton") + assert res.alpha == pytest.approx(ref.alpha, rel=1e-7) + + def test_metadata_has_newton_info(self): + matrices, centers = _random_simplex(4, 3, seed=42) + res = solve_minimax(matrices, centers, method="fw+brentq+newton") + assert res.metadata is not None + assert "newton_iters" in res.metadata + assert "hessian_cond" in res.metadata + + +# --------------------------------------------------------------------------- +# newton-cold specific +# --------------------------------------------------------------------------- + + +class TestNewtonCold: + @pytest.mark.parametrize("seed", range(5)) + def test_pairwise_converges(self, seed): + """newton-cold should converge for k=2 (pairwise case).""" + matrices, centers = _random_simplex(2, 2, seed=seed) + res = solve_minimax(matrices, centers, method="newton-cold") + ref = solve_minimax(matrices, centers, method="fw+bisect", tol=1e-12) + assert res.converged + assert res.alpha == pytest.approx( + ref.alpha, rel=1e-4 + ), f"seed={seed}: cold={res.alpha}, ref={ref.alpha}" + + @pytest.mark.parametrize("seed", range(5)) + def test_result_finite(self, seed): + """newton-cold should produce finite results even if not converged.""" + matrices, centers = _random_simplex(5, 3, seed=seed) + res = solve_minimax(matrices, centers, method="newton-cold") + assert np.isfinite(res.alpha) + assert np.all(np.isfinite(res.circumcenter)) + + def test_failed_newton_reports_its_status_without_retry(self): + matrices, centers = _random_simplex(5, 3, seed=0) + result = solve_minimax( + matrices, + centers, + method="newton-cold", + newton_max_iter=1, + tol=1e-10, + ) + + assert not result.converged + assert result.metadata is not None + assert "newton_status" in result.metadata + + +# --------------------------------------------------------------------------- +# fw+bisect+damped-newton specific +# --------------------------------------------------------------------------- + + +class TestDampedNewton: + @pytest.mark.parametrize("seed", range(5)) + def test_converges(self, seed): + matrices, centers = _random_simplex(4, 3, seed=seed) + res = solve_minimax(matrices, centers, method="fw+bisect+damped-newton") + ref = solve_minimax(matrices, centers, method="fw+bisect", tol=1e-12) + assert res.converged + assert res.alpha == pytest.approx(ref.alpha, rel=1e-7) + + def test_metadata_has_hessian_cond(self): + matrices, centers = _random_simplex(4, 3, seed=42) + res = solve_minimax(matrices, centers, method="fw+bisect+damped-newton") + assert res.metadata is not None + assert "hessian_cond" in res.metadata + + def test_empty_thresholded_face_is_nonconverged(self): + matrices, centers = _random_simplex(3, 2, seed=42) + with pytest.raises(ValueError, match="weight_tol must satisfy"): + solve_minimax( + matrices, + centers, + method="fw+bisect+damped-newton", + weight_tol=1.0, + ) + + +# --------------------------------------------------------------------------- +# fw+bisect+newton specific +# --------------------------------------------------------------------------- + + +class TestFwBisectNewton: + def test_pairwise_higher_precision_than_fw_bisect(self): + """fw+bisect+newton should achieve tighter residual than fw+bisect alone.""" + matrices, centers = _random_simplex(3, 3, seed=99) + + ref_tight = solve_minimax( + matrices, centers, method="fw+bisect", tol=1e-14, max_iter=10000 + ) + res_newton = solve_minimax(matrices, centers, method="fw+bisect+newton") + + assert ( + abs(res_newton.alpha - ref_tight.alpha) + <= abs( + solve_minimax(matrices, centers, method="fw+bisect").alpha + - ref_tight.alpha + ) + + 1e-12 + ) + + @pytest.mark.parametrize("seed", range(5)) + def test_converged(self, seed): + matrices, centers = _random_simplex(4, 3, seed=seed) + res = solve_minimax(matrices, centers, method="fw+bisect+newton") + assert res.converged, f"fw+bisect+newton did not converge for seed={seed}" + + def test_active_set_all_equal_fi_at_circumcenter(self): + """At Newton-polished optimum, all active f_i(x*) should be equal.""" + matrices, centers = _random_simplex(4, 3, seed=42) + res = solve_minimax(matrices, centers, method="fw+bisect+newton") + x = res.circumcenter + active = res.active_set + f_vals = np.array( + [ + float( + np.einsum("i,ij,j->", x - centers[i], matrices[i], x - centers[i]) + ) + for i in active + ] + ) + np.testing.assert_allclose(f_vals, res.alpha, rtol=1e-7, atol=1e-10) + + +# --------------------------------------------------------------------------- +# Backward compatibility: "fw+newton" alias +# --------------------------------------------------------------------------- + + +class TestBackwardCompat: + def test_fw_newton_alias_works(self): + """'fw+newton' should work as alias for 'fw+bisect+newton'.""" + matrices, centers = _random_simplex(3, 2, seed=0) + with warnings.catch_warnings(): + warnings.simplefilter("ignore", DeprecationWarning) + res = solve_minimax(matrices, centers, method="fw+newton") + assert res.method == "fw+bisect+newton" + + def test_fw_newton_alias_warns(self): + matrices, centers = _random_simplex(3, 2, seed=0) + with pytest.warns(DeprecationWarning, match="deprecated"): + solve_minimax(matrices, centers, method="fw+newton") + + +# --------------------------------------------------------------------------- +# scipy-slsqp specific +# --------------------------------------------------------------------------- + + +class TestScipySlsqp: + def test_success_flag_does_not_override_requested_gap(self): + matrices = np.array([1e-16 * np.eye(2), 4e-16 * np.eye(2)]) + centers = np.array([[0.0, 0.0], [1.0, 0.0]]) + + result = solve_minimax( + matrices, + centers, + method="scipy-slsqp", + tol=1e-20, + newton_tol=1e-20, + ) + + assert result.converged + assert result.alpha == pytest.approx(4.4444444444444444e-17, rel=1e-6) + assert result.metadata is not None + assert result.metadata["accuracy_polish"] == "newton" + + @pytest.mark.parametrize("k", [2, 3, 5]) + def test_alpha_agrees_with_reference(self, k): + matrices, centers = _random_simplex(k, 3, seed=k * 7) + ref = solve_minimax( + matrices, centers, method="fw+bisect", tol=1e-12, max_iter=5000 + ) + res = solve_minimax(matrices, centers, method="scipy-slsqp") + assert res.alpha == pytest.approx( + ref.alpha, rel=1e-6 + ), f"scipy-slsqp alpha mismatch for k={k}: {res.alpha} vs ref {ref.alpha}" + + def test_weights_on_simplex(self): + matrices, centers = _random_simplex(5, 4, seed=55) + res = solve_minimax(matrices, centers, method="scipy-slsqp") + assert res.weights.sum() == pytest.approx(1.0, abs=1e-10) + assert np.all(res.weights >= -1e-12) + + +# --------------------------------------------------------------------------- +# MinimaxResult backward compatibility +# --------------------------------------------------------------------------- + + +class TestMinimaxResultBackwardCompat: + def test_default_method_field(self): + r = MinimaxResult( + alpha=1.0, + circumcenter=np.zeros(2), + weights=np.array([0.5, 0.5]), + active_set=[0, 1], + converged=True, + n_iter=3, + ) + assert r.method == "fw+bisect" + assert r.metadata is None + + def test_namedtuple_positional(self): + r = MinimaxResult(2.0, np.zeros(2), np.ones(2) * 0.5, [0, 1], True, 5) + assert r.alpha == 2.0 + assert r.method == "fw+bisect" + assert r.metadata is None + + def test_metadata_field(self): + r = MinimaxResult( + alpha=1.0, + circumcenter=np.zeros(2), + weights=np.array([0.5, 0.5]), + active_set=[0, 1], + converged=True, + n_iter=3, + method="fw+brentq", + metadata={"fw_iters": 10}, + ) + assert r.metadata == {"fw_iters": 10}