Tuning sweeps, band efficiency, exact scorecards, tuning guide + FAQ

CHANGELOG.md

@@ -7,7 +7,33 @@ inside each artifact (`schema_version`).
 
 ## [Unreleased]
 
+## [0.1.1] - 2026-08-16
+
+First complete release. Supersedes 0.1.0, whose wheel carried a stale
+hardcoded `compileml.__version__` of `0.1.0.dev0` (the PyPI metadata was
+correct; the module attribute and artifact `compileml_version` were not).
+The version now has a single source of truth — `compileml.__version__` —
+which pyproject reads at build time.
+
 ### Added
+- Tuning sweeps (`compileml.tune`): `sweep_whitebox` (trees × depth grid with
+  measured retention, rank agreement, artifact size, and explanation cost)
+  and `sweep_bands` (band-count grid with retention, Gini gap, and worst
+  within-band AUC).
+- `compileml.bands.band_efficiency`: the "money on the table" diagnostic —
+  continuous-vs-band Gini gap plus per-band within-band AUC with bootstrap
+  CIs and refinable / exhausted / inconclusive verdicts. Validation check 4
+  now carries these fields whenever outcomes are supplied, advisory by
+  default and gateable via `max_within_band_auc=`.
+- Exact scorecard extraction (`compileml.scorecard`, stdlib-only): at
+  whitebox depth ≤ 2 the artifact collapses into bin → points tables plus
+  explicit pairwise interaction grids whose integers re-sum to every
+  decision's `raw_micro` bit-for-bit (`score_from_scorecard` re-derives any
+  decision from the printed tables; asserted in tests). Markdown and CSV
+  renderers; `compileml scorecard` CLI subcommand; refuses above depth 2.
+- Docs: tuning guide (`howto/tuning.md`) answering the configuration
+  questions — tree count, depth, the trees-vs-depth asymmetry, band count,
+  band efficiency, scorecards — and a FAQ page.
 - Decision artifact schema v2: integer-quantized leaves, integer calibration
   tables, fixed-point band ladders, half-micro exact attribution with the
   reconciliation identity, missing-value policy, SHA-256 verify-on-load.

RELEASING.md

@@ -24,16 +24,18 @@ tokens stored anywhere.
 
 ## Releasing vX.Y.Z
 
-1. Bump `version` in `pyproject.toml` (drop the `.dev0`); move the
-   `[Unreleased]` CHANGELOG section under the new version with the date.
+1. Bump `__version__` in `src/compileml/__init__.py` — the single source of
+   truth; pyproject reads it at build time — and move the `[Unreleased]`
+   CHANGELOG section under the new version with the date.
 2. Commit: `release: vX.Y.Z`, then tag and push:
    ```bash
    git tag vX.Y.Z
    git push && git push --tags
    ```
 3. Create a GitHub Release from the tag (paste the CHANGELOG section).
    Publishing to PyPI triggers automatically from the release event.
-4. Post-release: bump `version` to the next `.dev0` and commit.
+4. Post-release: bump `__version__` in `src/compileml/__init__.py` to the
+   next `.dev0` and commit.
 
 ## Release gates (all enforced by CI before you ever tag)
 

docs/faq.md

@@ -0,0 +1,120 @@
+# FAQ
+
+### How many trees should the whitebox have? What depth?
+
+Measured answers beat rules of thumb: run
+[`sweep_whitebox`](howto/tuning.md) on a holdout and pick the elbow of the
+retention curve. The short version: trees are a cheap linear knob that never
+costs you a guarantee; depth is the knob that takes exact attribution away
+above 2. **Spend on trees, be stingy with depth.**
+
+### Why is depth 2 the default?
+
+It is the boundary of two guarantees at once. At depth ≤ 2 the pairwise
+attribution is *complete* — the residual is zero as an arithmetic identity —
+and the artifact collapses into an [exact scorecard](howto/tuning.md#producing-a-scorecard).
+At depth 3 both break, for the same mathematical reason: three-way structure
+appears. The tooling warns, records, draws, and refuses accordingly.
+
+### If I keep adding capacity, don't I just get the teacher back and lose the point of compiling?
+
+No. Fidelity converges toward the teacher; the compilation properties —
+integer determinism, one hashed artifact, SQL/COBOL export, sub-millisecond
+scoring — hold at any size. Only exact attribution is at risk, and only from
+depth. What grows with trees is artifact bytes and explanation milliseconds,
+linearly, and [`sweep_whitebox`](howto/tuning.md) prices both.
+
+### How many bands should I use?
+
+Either let the data answer — [`semantic_bands` / `governance_bands`](concepts/bands.md)
+return the band count they can statistically *defend*, and honestly return
+one band on noise — or sweep fixed K with `sweep_bands` and read the
+retention-vs-K table.
+
+### How do I know my banding isn't leaving money on the table?
+
+[`band_efficiency`](howto/tuning.md#money-on-the-table-within-band-auc). The
+`gini_gap` is the discrimination your ladder discards; per-band within-band
+AUCs (with bootstrap CIs) tell you *where* — a band whose CI sits above 0.55
+can still rank risk internally and is a refinement candidate. Validation
+check 4 carries the same numbers on every run.
+
+### Can I get a classic points scorecard out of this?
+
+Yes — exactly, not approximately, at depth ≤ 2: `build_scorecard(artifact)`
+or `compileml scorecard decision.json --format csv`. The printed tables
+re-sum to every production decision bit-for-bit; a validator can reproduce
+scores in a spreadsheet.
+
+### Why not just use SHAP?
+
+TreeSHAP is exact for trees and a fine analysis tool — the differences are
+about *deployment*, not correctness. CompileML's explanation is computed on
+the deployed object itself (not the pre-compilation model), in integer units
+that re-sum to the decision, by a runtime with no ML dependencies, and it
+travels into the SQL and COBOL exports. The explanation is part of the
+decision record, under the artifact's hash, rather than a separate analysis
+run that must be trusted to match.
+
+### Can I compile my XGBoost classifier directly?
+
+Directly compiled models must emit a latent in [0, 1] — a classifier's raw
+margin lives in log-odds space and will be clamped into nonsense. Distill it:
+`train_whitebox(X, model.predict_proba(X)[:, 1])`. Regressors on
+probability-like targets compile directly, and the build warns when sample
+latents fall outside range.
+
+### What about neural networks?
+
+Same route: any model that produces a probability-like latent can teach a
+whitebox. The artifact never contains the network — it contains the distilled
+trees, with the retention measured and recorded.
+
+### What happens when I retrain or recalibrate?
+
+Two mechanically distinct cases, distinguishable by hash. **Recalibration**
+(`recalibrate_artifact`) refits the PD table on fresh outcomes while the
+model and band edges stay byte-identical — provably zero band churn, with the
+predecessor's hash recorded as a provenance chain. **Retraining** produces a
+genuinely new artifact and a fresh governance cycle. See
+[zero-churn recalibration](howto/recalibrate.md).
+
+### How are missing values handled?
+
+By declared policy inside the artifact: `"baseline"` re-applies the
+training-time imputation at decision time; `"reject"` refuses the row.
+NaN never routes silently through a tree comparison, in any runtime.
+
+### Is the artifact hash a signature?
+
+No — it is an integrity check. Loaders verify it by default and refuse a
+tampered or corrupted document, but it does not prove *who* produced the
+artifact. Provenance of authorship is your repository's and your process's
+job.
+
+### My semantic_bands returned one band. Is that a bug?
+
+It is the honest answer: under your `eps_auc` strictness, the data cannot
+statistically support discrete classes — either outcomes are too noisy or
+the within-band separation you demanded isn't there. Loosen `eps_auc`
+deliberately, or accept that band boundaries would be arbitrary. A banding
+tool that always returns the requested K is a random number generator with
+labels.
+
+### Why is the full explanation slower than scoring?
+
+Exact pairwise attribution costs `1 + p + p(p−1)/2` ensemble traversals —
+single-digit milliseconds at typical feature counts, which is real-time for
+credit decisioning ([explain everything](concepts/attribution.md#cost-honestly)
+is the recommended default). The cost matters for full-book *batch*
+re-explanation, which the leaf-time roadmap item addresses.
+
+### Same artifact, same input — could two machines ever disagree?
+
+Not within the contract: scoring is integer addition, banding integer
+comparison, calibration integer lookup, and the single float operation
+(`x <= threshold`) is exact under IEEE 754. CI proves it continuously —
+committed reference integers replayed on three OSes, generated SQL executed
+and diffed row-for-row, generated COBOL compiled and run. What is *not*
+claimed: that your upstream feature pipeline produces identical bytes across
+systems ([precisely stated](concepts/determinism.md)).

docs/howto/tuning.md

@@ -0,0 +1,177 @@
+# Tuning the compilation
+
+Every question on this page has a measured answer — `compileml.tune` exists
+so configuration is a table you read, not a guess you defend.
+
+## The two knobs are not symmetric
+
+| Knob | Buys | Costs | Determinism / portability |
+|---|---|---|---|
+| `n_estimators` ↑ | fidelity to the teacher | artifact size and explanation time, both **linear** | **unaffected** |
+| `max_depth` ↑ | fidelity per tree | **exact attribution above 2**, per-tree explain cost, scorecard legibility | **unaffected** |
+
+Integer exactness, cross-runtime determinism, and hash governance are
+structural: a 500-tree artifact is exactly as deterministic and exactly as
+COBOL-exportable as a 30-tree one. Nothing about compilation degrades with
+size — what degrades above depth 2 is *explainability*, and only that.
+
+**Spend on trees; be stingy with depth.**
+
+## Finding the tree count
+
+```python
+from compileml.tune import sweep_whitebox
+
+rows = sweep_whitebox(
+    X_train, teacher_latent_train, y_train,
+    trees_grid=(20, 40, 80, 160, 320),
+    depth_grid=(1, 2),
+    X_val=X_val, y_val=y_val, teacher_latent_val=teacher_latent_val,
+)
+# pandas.DataFrame(rows) if you like tables
+```
+
+Each row reports holdout Gini and retention versus the teacher, Spearman rank
+agreement, `exact_attribution`, the quantized model's JSON size, and a
+*measured* per-row exact-explanation cost. Read it like a cost curve: retention
+climbs steeply, then plateaus; pick the elbow. In the repository benchmark,
+120 trees at depth 2 retained 97.9% of a 300-tree teacher's Gini — beyond the
+plateau you pay linear size and explain time for basis points of fidelity.
+
+Always sweep on a holdout (`X_val=`) — in-sample retention flatters every
+configuration, and the output flags `in_sample: true` when you didn't.
+
+## Choosing depth: one table, one story
+
+| depth | Attribution | Scorecard | Fidelity |
+|---|---|---|---|
+| 1 | exact, zero residual | **exact classic scorecard** (bin → points) | lowest |
+| **2** (default) | exact, zero residual | scorecard + explicit interaction grids | good |
+| 3+ | residual appears | none exists | highest |
+
+Depth ≤ 2 is not a style preference — it is the boundary of two guarantees.
+The pairwise decomposition is *complete* for functions with no three-way
+interactions, which is precisely what depth ≤ 2 trees are; at depth 3 the
+leftover becomes a real, reported residual
+([why](../concepts/attribution.md)), and no clean scorecard exists for the
+same mathematical reason. The tooling enforces the boundary honestly rather
+than hiding it:
+
+- `train_whitebox` **warns** at `max_depth > 2`;
+- the artifact **records** `whitebox_max_depth` and `exact_attribution`;
+- every explained decision **reports** its residual, and the waterfall draws
+  it as an explicit bar;
+- `decide()` **refuses** a nonzero residual on an artifact claiming exactness;
+- `build_scorecard` **raises** above depth 2.
+
+To *quantify* what depth 3 would cost you before committing, explain a sample
+and look at the residuals directly:
+
+```python
+residuals = [
+    abs(decide(artifact3, row, include_contributions=True)
+        ["attribution_residual_half_micro"]) / (2 * 1_000_000)
+    for row in X_sample
+]
+# share of decisions with any unexplained remainder, and how large it gets
+```
+
+## "Doesn't a bigger whitebox just become the teacher?"
+
+Only in the sense you want. More capacity converges toward the teacher's
+*predictions* while every compilation property — integer determinism, the
+hashed single artifact, SQL/COBOL export, sub-millisecond scoring — holds at
+any size. The one thing you can lose is exact attribution, and that is
+controlled solely by depth, not trees. The trade to actually manage is
+pragmatic: linear growth in artifact bytes and explanation milliseconds,
+which `sweep_whitebox` prices per configuration.
+
+## How many bands?
+
+Two philosophies, both shipped:
+
+**Discover K.** [`semantic_bands` and `governance_bands`](../concepts/bands.md)
+return the number of bands the data can statistically *defend* — Jeffreys-CI
+separation between neighbors, no residual rank power within any band,
+bootstrap-certified. Feed them noise and they honestly return one band.
+
+**Sweep fixed K.**
+
+```python
+from compileml.tune import sweep_bands
+
+rows = sweep_bands(latent_train, y_train, k_grid=(4, 6, 8, 10, 12, 16))
+```
+
+Per K: band-ordinal Gini and retention, the Gini gap, the worst within-band
+AUC with its verdict, minimum band volume, monotonicity violations, and any
+integer-edge collisions (a K too fine for the display scale to represent —
+the build would refuse it anyway).
+
+## Money on the table: within-band AUC
+
+A band ladder discards rank information by design; the governed question is
+*how much* and *where*:
+
+```python
+from compileml.bands import band_efficiency
+
+eff = band_efficiency(latent_val, y_val, artifact)
+eff["gini_gap"]        # continuous Gini − band-ordinal Gini: the headline
+eff["per_band"]        # n, bad rate, within-band AUC with bootstrap CI, verdict
+eff["worst_band"]      # strongest refinement candidate
+```
+
+Reading the per-band verdicts:
+
+- **exhausted** — CI upper bound ≤ 0.55: the score is used up inside this
+  band; splitting it further separates noise, not risk.
+- **refinable** — CI lower bound ≥ 0.55: the latent can still rank outcomes
+  inside the band; a finer cut there would separate risk your policy
+  currently treats as homogeneous. That is money on the table.
+- **inconclusive** — the interval spans both stories; more volume before
+  concluding anything.
+
+The same diagnostics attach to every validation run: check 4 of
+[`validate_artifact`](validate.md) reports `banding_gini_gap` and
+`worst_within_band_auc` whenever outcomes are supplied, advisory by default
+and gateable via `max_within_band_auc=` when your policy wants a hard limit.
+
+## Producing a scorecard
+
+At depth ≤ 2 the artifact *is* a points-based scorecard — exactly, not as an
+approximation:
+
+```python
+from compileml.scorecard import build_scorecard, scorecard_to_markdown
+
+scorecard = build_scorecard(artifact)
+print(scorecard_to_markdown(scorecard, labels=DISPLAY_NAMES))
+```
+
+```bash
+compileml scorecard decision.json --format csv --out scorecard.csv
+```
+
+Depth 1 collapses to the classic form — per feature, bin → points. Depth 2
+adds explicit pairwise interaction grids over the union of the relevant
+thresholds. Points are the artifact's own integers, and the identity
+
+```
+base_points + Σ main_effect(x) + Σ interaction(x) == raw_micro
+```
+
+holds bit-for-bit on every row (`score_from_scorecard` re-derives any
+decision from the printed tables alone — the test suite asserts it). Hand
+the CSV to a validator and they can reproduce production scores in a
+spreadsheet.
+
+Above depth 2, `build_scorecard` raises instead of approximating — the same
+boundary as exact attribution, for the same reason.
+
+## Defaults, for the impatient
+
+`train_whitebox(n_estimators=30, max_depth=2)` and `n_bands=10` are sane
+starting points, proven in the repository's own benchmark and examples. The
+sweeps are for when "sane" needs to become "measured" — which, in a model
+governance file, it eventually does.

docs/reference/api.md

@@ -18,6 +18,24 @@
 
 ::: compileml.bands.governance_bands
 
+::: compileml.bands.band_efficiency
+
+## Tuning
+
+::: compileml.tune.sweep_whitebox
+
+::: compileml.tune.sweep_bands
+
+## Scorecard
+
+::: compileml.scorecard.build_scorecard
+
+::: compileml.scorecard.score_from_scorecard
+
+::: compileml.scorecard.scorecard_to_markdown
+
+::: compileml.scorecard.scorecard_to_csv
+
 ## Artifact
 
 ::: compileml.artifact.build_artifact

mkdocs.yml

@@ -52,11 +52,13 @@ nav:
       - Exact attribution: concepts/attribution.md
       - Risk bands: concepts/bands.md
   - How-to:
+      - Tuning the compilation: howto/tuning.md
       - Reason codes: howto/reason-codes.md
       - Visualize decisions: howto/visualize.md
       - Zero-churn recalibration: howto/recalibrate.md
       - Deploy (runtime, SQL, COBOL): howto/deploy.md
       - Validate before deploying: howto/validate.md
+  - FAQ: faq.md
   - Reference:
       - Artifact specification: ARTIFACT_SPEC.md
       - Python API: reference/api.md