From 5bdda24aac69de79586099fcb15a9b7b22bed736 Mon Sep 17 00:00:00 2001 From: J Xu Date: Tue, 8 Sep 2026 08:39:14 -0700 Subject: [PATCH] Refresh README with foundation vision and verified CPU/CUDA scope --- AGENTS.md | 10 +- README.md | 294 ++++++++++-------- .../2026-09-08-v1-readme-state-vision.md | 58 ++++ v1-sprints/109-pr-integration.md | 26 +- v1-sprints/README.md | 7 +- 5 files changed, 254 insertions(+), 141 deletions(-) create mode 100644 learnings/2026-09-08-v1-readme-state-vision.md diff --git a/AGENTS.md b/AGENTS.md index 9cbac96..0fca981 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -148,11 +148,11 @@ next construction is required multiclass geometry/comparison/recipe integration under 080, then AFT and vector topology. No speed ratio or formal E4 pass follows. Preserve consumed run sources by revision when current implementation changes. -The user then requests another PR and preparation for merge. Follow the -[109 integration checkpoint](v1-sprints/109-pr-integration.md) before further -construction: repair demonstrated CI/docs blockers, publish the reviewed branch -as an English PR and inspect hosted checks. Preserve execution SHAs with a merge -commit; the current request does not execute the merge or resume hardware/models. +The [109 integration checkpoint](v1-sprints/109-pr-integration.md) is complete: +all five hosted checks pass at `bbd69ea`, and the user's subsequent merge request +lands PR 25 as merge commit `167ed53`, preserving execution SHAs. The README +follow-up documents the current foundation and vision. No hardware/model study +resumes as part of this documentation update. After this checkpoint, advance required CUDA recipes (080), compatible train-many (081) and evidence-led quality/cost work (082). Preserve all frozen diff --git a/README.md b/README.md index 73dd883..62ee40c 100644 --- a/README.md +++ b/README.md @@ -2,143 +2,177 @@ **A programmable boosting foundation for researchers and AI agents.** -OpenBoost v1 is being rebuilt around composable algorithm components, ordinary -Python recipes and explicit CPU/CUDA execution. The goal is to reduce the cost -of making a correct, reproducible algorithm change. - -## Current state - -This checkout is **under construction**. The retired implementation is not restored. -Initial public CPU components now provide owned numeric inputs, explicit problems, -run identity, immutable proposal/accept/reject state and mapped tree/constant ensemble -artifacts. See [B03 usage and boundaries](docs/v1/cpu-state.md). - -[Categorical input and equality splits](docs/v1/categorical.md) support typed -train-only dictionaries and explicit missing/unseen routing. - -Public [numeric operations](docs/v1/numeric-ops.md) now add quantile binning, -weighted row fields, histograms, split callbacks, routing and scalar/vector leaves. -Public [depthwise, best-first and symmetric growers](docs/v1/trees.md) compose -these operations and persist validated numeric/categorical trees. The first complete [squared-error recipe](docs/v1/squared.md) -and [Normal recipe](docs/v1/normal.md) support weights, offsets and -fixed/backtracking steps on CPU. Normal exposes ordinary/Fisher directions and -joint mean/log-scale updates. [Formula and sequential runs](docs/v1/formula-runs.md) -add structured full-metric updates and independent heterogeneous jobs. Experimental -resident scalar CUDA training passes all 212 bounded T4 checks, including -weighted/missing parity, transactions and saved CPU inference. The shared scoring -correction resolves the previous 14 failures without changing tolerances. See the -[recorded result and scope](benchmarks/v1/evidence/cuda-score-symmetry-089/README.md). -The first [Normal K=2 and installed D2 T4 run](benchmarks/v1/evidence/cuda-normal-090/README.md) -passes 381/383 checks, including all earlier scalar cases and nineteen saved-model -CPU replays. Two ordered acceptance decisions fail the frozen reference; full -Normal conformance remains open. The [follow-up diagnostic run](benchmarks/v1/evidence/cuda-acceptance-091/README.md) -preserves both failures and identifies rounding-induced false improvement at -near-stationary loss. The subsequent objective-owned comparison correction and -[bounded revalidation](benchmarks/v1/evidence/cuda-recipe-103/README.md) establish -514 earlier passes plus fifteen new recipe passes with identical production. -All 529 revised requirements have passing evidence across two executions; -the original failed verdicts remain preserved. This is not full Normal conformance. -[Binary classification](docs/v1/binary.md) now -persists typed class order and exposes probability/label inference. -[Multiclass and vector leaves](docs/v1/multiclass.md) add joint softmax updates -and separate split/leaf statistics with arbitrary output mappings. - -Independent references and comparator/data checks remain evaluation preparation. -F0.3 is still open; the user approved overlapping B03–B06 construction without -removing any v1 scope or acceptance requirements. No real quality, competitive GPU performance -or agent/adoption advantage has been established for the new foundation. - -The [early same-host performance checkpoint](benchmarks/v1/evidence/early-performance-104/README.md) -measures squared-error boosting at 10,000 rows in 7.04 s on CPU and 2.94 s on a -warm T4, with comparable quality. The four other timing pairs and the separate -profile are incomplete after deadlines. This synthetic internal result establishes -neither external-library speed parity nor practical performance across all recipes. - -- [Execution and reflections](v1-sprints/README.md) +Build boosting algorithms in Python by composing objectives, statistics, split +rules, weak learners and update policies. OpenBoost provides a NumPy CPU reference +and experimental CUDA execution with Python-authored kernels. + +## Vision + +An algorithm change can require more than a new loss function. It may need extra +statistics, a different split constraint, a custom leaf solver, coupled outputs, +or a different acceptance rule. OpenBoost aims to make each of those decisions +accessible through public components and ordinary Python training loops. + +The design takes inspiration from PyTorch's composable building blocks and explicit +execution. Here the building blocks are boosting operations: objective geometry, +named row fields, histograms, candidate scoring, feasibility, routing, leaf solving, +and immutable training transactions. A recipe is a working composition that a +researcher or agent can inspect, modify and reuse. + +Standard GBDT, NaturalBoost/NGBoost-style distributional methods, FormulaBoost-style +structured models and training many models guide the abstraction boundaries. +Applications matter equally: classification, regression, ranking, quantiles, +multi-output prediction, counts, positive and aggregate targets, survival, +distributional modeling and model selection all remain required v1 scope. + +Success means making a correct algorithm change easier, then demonstrating useful +quality and execution cost on real workloads. CPU is the semantic reference and +usable development path; CUDA is the route toward efficient execution. Comparative +agent-authoring studies are currently deferred while foundation construction and +validation continue. Author productivity and adoption benefits remain hypotheses. + +## What works today + +**Experimental v1, under construction.** The current foundation is merged through +[PR #25](https://github.com/jxucoder/openboost/pull/25). It is not a drop-in +replacement for XGBoost, LightGBM or CatBoost, and full v1 acceptance remains open. + +Public CPU components include typed numeric/categorical data, fitted preparation, +weights and offsets, named statistics, composable split/routing/leaf operations, +and depthwise, best-first and symmetric growers. Scalar/vector leaves and mapped +outputs share explicit proposal/accept/reject state. All twelve CPU recipes support +independent validation patience; saved models retain their required inference metadata. + +The CUDA implementation uses CuPy-owned storage and streams with Python kernels +compiled by `numba-cuda`. Public operations expose storage, fields, histograms, +scores, feasibility masks, routing, scalar leaves, depthwise trees and resident +transactions. Training uses explicit device interfaces; the CPU recipe API does +not automatically dispatch to CUDA. Current CUDA tree growth covers numeric and +missing features, with scalar trees and mapped multi-parameter updates. + +| Use case | CPU implementation | Verified CUDA scope | +| --- | --- | --- | +| Regression | [Squared error](docs/v1/squared.md) | Resident squared recipe | +| Classification | [Binary](docs/v1/binary.md), [multiclass](docs/v1/multiclass.md) | Binary recipe; multiclass pending | +| Counts and positive/aggregate targets | [Poisson with exposure](docs/v1/poisson.md), [Gamma](docs/v1/gamma.md), [fixed-power Tweedie](docs/v1/tweedie.md), [frequency–severity composition](docs/v1/frequency-severity.md) | Poisson recipe; other cells pending | +| Ranking and quantiles | [Query-local pairwise/lambda ranking](docs/v1/ranking.md), [quantile and penalized leaves](docs/v1/quantile.md) | Pending | +| Survival | [Fixed-scale log-normal AFT with events/right censoring](docs/v1/aft.md) | Pending | +| Distributional and structured models | [Normal ordinary/Fisher updates](docs/v1/normal.md), [saturation Formula/full-GGN updates](docs/v1/formula-runs.md) | Bounded Normal joint/ordered recipes; Formula pending | +| Multi-output regression | [Independent/shared trees, projected splits and target scaling](docs/v1/multioutput.md) | Vector topology pending | +| Train-many | [Shared preparation and independent sequential runs](docs/v1/preparation.md), verified at M=1/8/32 | Compatible resident execution pending | + +CUDA entries describe bounded correctness evidence, not complete feature coverage +or a speed guarantee. Categorical CUDA growth, broader vector learners and fused +train-many remain unverified. See the [CPU component guide](docs/v1/numeric-ops.md), +[tree contracts](docs/v1/trees.md), [stopping semantics](docs/v1/stopping.md) and +[explicit CUDA interfaces](docs/v1/execution.md) for supported inputs and limits. + +## Evidence and performance + +- **Latest CUDA validation:** [run 12](benchmarks/v1/evidence/cuda-glm-108/README.md) + passes 571/571 real T4 cases: 153 binary/Poisson checks and 418 regressions. All + 77 JSON artifacts are retained. The offline audit verifies 246 numerical + loss-change comparisons and replays 32 final/best models from saved input bytes. +- **Reliable Normal decisions:** [comparison and revalidation evidence](benchmarks/v1/evidence/cuda-recipe-103/README.md) + covers all 529 revised requirements across two executions. Earlier failed + verdicts remain preserved. This is bounded coverage, not full Normal conformance. +- **Measured internal improvement:** [run 11](benchmarks/v1/evidence/parallel-validation-105/README.md) + passes 474 T4 checks and three cost gates. Parallel field validation reduces + median warm fit time for synthetic squared boosting at 100,000 rows from + 13.513 to 8.947 seconds, with unchanged model/prediction bytes. That workload + uses 16 features, depth three and 20 rounds; the reduction is 33.79% against the + earlier OpenBoost implementation on the same T4. +- **CPU and packaging:** the merged checkpoint has 2,292 local CPU tests passing. + [Hosted CI](https://github.com/jxucoder/openboost/actions/runs/34241939802) passes + Linux/macOS on Python 3.10/3.12, including offline audits and package builds; + [strict documentation checks](https://github.com/jxucoder/openboost/actions/runs/34241939812) + also pass. Historical tests are explicitly separated from current conformance. + +These results do not establish competitive speed or predictive quality against +mature boosting libraries. Real application evaluations and the formal end-to-end +quality/cost gate remain open. The [earlier incomplete performance checkpoint](benchmarks/v1/evidence/early-performance-104/README.md) +is retained alongside the later complete measurements. + +## Try the CPU foundation + +Use Python 3.10+ and install from this checkout: + +```bash +uv sync --extra test +``` + +This example supplies a custom learner through the public growth and feasibility +operations, fits a squared-error recipe, and saves its best validation model. + +```python +from functools import partial + +from openboost import NumericData, Problem, RunContext +from openboost.artifacts import Model +from openboost.ops import feasible +from openboost.recipes import squared +from openboost.tree import depthwise + +train_x = NumericData([[0], [1], [2], [3]], [10, 11, 12, 13], ("x",)) +valid_x = NumericData([[0.5], [2.5]], [20, 21], ("x",)) +train = Problem(train_x, [[-3], [-1], [1], [3]], train_x.row_ids) +valid = Problem(valid_x, [[-2], [2]], valid_x.row_ids) + + +def learner(binned, fields): + return depthwise( + binned, fields, max_depth=1, + legality=partial(feasible, min_child_h=2), + ) + + +fit = squared( + train, valid, context=RunContext("example", seed=7), + learner=learner, rounds=3, learning_rate=0.5, bins=4, +) +model = fit.state.best_model +prediction = model.predict(valid_x) # Shape: (2, 1) +model.save("model.json") +restored = Model.load("model.json") +``` + +For deeper changes, compose [objective/statistics operations](docs/v1/numeric-ops.md) +and [run transactions](docs/v1/cpu-state.md) directly. CUDA users need real NVIDIA +hardware and the optional dependencies (`uv sync --extra cuda`); start with the +[separate device execution guide](docs/v1/execution.md). + +## Next milestones + +1. Complete the [required CUDA recipes](v1-sprints/080-cuda-required-recipes.md), + starting with multiclass, then AFT and vector topology, with independent + mathematics, CPU/CUDA checks and persisted inference for each declared scope. +2. Establish [compatible train-many execution](v1-sprints/081-cuda-train-many.md), + preserving independent state while reusing preparation and device resources. +3. Measure [real-workload quality and complete execution cost](v1-sprints/082-end-to-end-cost.md) + with fair baselines, then stabilize the public contracts supported by that evidence. + +All [R1–R9 / C1–C7 / A1–A13 requirements](planning/openboost-v1-evaluation.md) +remain in scope. Each [application family](planning/foundation-application-contracts.md) +needs its own implementation and evaluation. Multi-GPU, Ray and out-of-core +expansion are outside the active plan. + - [Construction design](planning/foundation-construction-design.md) - [v1 plan](planning/agent-boosting-foundation-plan.md) -- [Required tasks](planning/foundation-tasks.md) -- [Acceptance and evaluation](planning/openboost-v1-evaluation.md) - -All R1–R9 / C1–C7 / A1–A13 remain required. Classification, regression, ranking, -quantiles, multi-output, count/positive/aggregate targets, survival, distributional -and formula models, and train-many each need their own implementation and evidence. +- [Execution, evidence and reflections](v1-sprints/README.md) -## Development +## Development and history ```bash -uv sync --extra test uv run pytest tests/ -m "not gpu and not benchmark" -n 0 -q uv run ruff check src/openboost tests/v1 tests/conftest.py +uv run mkdocs build --strict uv build ``` -Python 3.10+. Current tests cover CPU implementation, independent references and -evaluation infrastructure. Experimental CUDA storage, named fields and histograms -and bounded resident squared training have real T4 evidence. The full required -device recipe, quality and cost gates remain open. Run GPU-marked tests only on -real hardware; publishing remains separate. - -## Historical implementation and evidence - -Revision `50acfc6` is the last revision containing the old production code plus -Sprint 001 references. Use that revision in a separate checkout to reproduce -old APIs, examples and experiments; no compatibility layer remains here. - -Historical tests, examples, documentation and benchmark artifacts are retained -as evidence and sources of mathematical counterexamples. Default test discovery -runs `tests/v1/` only. Old tests are not counted as v1 passes or skips. -The current documentation build uses `docs/v1/`; other documentation describes -the retired implementation. Published packages and historical results do not -establish the new architecture's quality, speed or adoption. - -[Query-local ranking](docs/v1/ranking.md) adds pairwise/lambda CPU geometry and -fixed-step recipes with validation NDCG selection. Real A4 evaluation remains open. - -[Quantile and penalized leaves](docs/v1/quantile.md) expose routed residuals/original -weights and compose all three CPU growth policies. Real A5 evaluation remains open. - -[Poisson counts and exposure](docs/v1/poisson.md) add a CPU count recipe with explicit -rate/count outputs. Real A7 evaluation remains open. - -[Gamma positive-target means](docs/v1/gamma.md) add weighted CPU mean regression. -Real A8 quality and distributional calibration remain unverified. - -[Tweedie nonnegative means](docs/v1/tweedie.md) support fixed-power CPU fitting and -explicit annualized-loss weight semantics. Real A9 evaluation remains open. - -[Frequency–severity composition](docs/v1/frequency-severity.md) binds matched paid-loss aggregates -and persists two-model inference with explicit output units. Real A9 evaluation remains open. - -[Log-normal AFT](docs/v1/aft.md) adds event/right-censored CPU training and -persisted scale-aware survival outputs. Real A10 evaluation remains open. - -[Current execution and reflections](v1-sprints/README.md) -separates implemented CPU coverage from remaining authoring, practical execution, -real selection and GPU evidence. - -[Multi-output squared regression](docs/v1/multioutput.md) supports independent/shared trees, -projected splits and persisted training-only target scaling. Real A6 evaluation remains open. - -[Shared training preparation](docs/v1/preparation.md) reuses fitted CPU binning/codes -across independent jobs, verified at M=1/8/32. -[Independent stopping](docs/v1/stopping.md) adds validation patience to every CPU -recipe while keeping model acceptance and best-model selection independent. - - -[Experimental CUDA operations](docs/v1/execution.md) provide context-owned buffers, -named fields, once-only weighting, routed histograms, candidate scores, composable -feasibility masks, routing and scalar leaves, with -[88 passing real T4 checks](benchmarks/v1/evidence/cuda-splits-078/README.md). -Independent cohort constraints change split selection through the public device -operations. Separate experimental resident squared geometry, scalar trees and -accepted/proposal training now pass the separate -[212-case T4 matrix](benchmarks/v1/evidence/cuda-score-symmetry-089/README.md). -Shared mapped transactions, Normal geometry and joint/ordered recipes now have -[bounded passing comparison and recipe evidence](benchmarks/v1/evidence/cuda-recipe-103/README.md), -with the historical numerical failures preserved in the earlier archives. -The installed D2 learner uses the same public field/feasibility/tree operations. -Binary/Poisson objectives, numerical loss-change comparisons and the shared scalar -recipe now pass [153 GLM T4 checks plus 418 regressions](benchmarks/v1/evidence/cuda-glm-108/README.md), -with retained input bytes, class-aware inference and 32 audited final/best models. -Other required CUDA recipes and full phase acceptance remain open. +Default discovery runs `tests/v1/`; see [test scope](tests/README.md). Run GPU-marked +tests only on real hardware. The current documentation lives in `docs/v1/`. + +The retired implementation remains at revision `50acfc6` for reproducing old APIs, +examples and experiments. There is no compatibility layer in v1. Historical +packages and benchmarks describe their recorded revisions; all new claims must +link to reproducible evidence for the current foundation. diff --git a/learnings/2026-09-08-v1-readme-state-vision.md b/learnings/2026-09-08-v1-readme-state-vision.md new file mode 100644 index 0000000..796a6b5 --- /dev/null +++ b/learnings/2026-09-08-v1-readme-state-vision.md @@ -0,0 +1,58 @@ +# 2026-09-08: Present the foundation's current state and vision + +## Context + +After PR 25 merges as `167ed53`, the user asks to update the README. Its appended +checkpoint narrative mixes early CPU construction, old CUDA failures and newer +results, obscuring both the programmable-foundation vision and present capability. + +## Decision or Result + +Present the public component model first, then distinguish implemented CPU scope, +bounded CUDA evidence, real-workload evaluation gaps and next construction. Keep +all required application families visible. Agent productivity is a hypothesis; +comparative authoring studies remain deferred. CPU remains the semantic reference +and usable development path, with explicit Python-authored CUDA execution. + +## Changes + +- Rewrite `README.md` with the vision, CPU/CUDA recipe matrix, linked run-11/run-12 + evidence, a public learner example, and the required recipe/train-many/cost work. +- Replace the incomplete run-10 timing headline with the complete run-11 internal + comparison, retaining the earlier failure link and precise limits on speed claims. +- Add a runnable CPU example that replaces growth/feasibility through a learner + callback and saves/loads the best model. It does not imply automatic GPU dispatch. +- Record the completed PR merge in the 109 sprint, execution index and agent guide. + The local documentation branch starts from merged main; no new push is requested. + +## Verification + +- Execute the exact README Python block in a temporary output directory: three + accepted rounds, three-node stumps, improved training loss, finite `(2, 1)` + predictions and exact saved-model prediction replay all pass. +- All 34 relative README links resolve. English-prose and whitespace checks pass. +- Ruff passes production and the extracted example. +- `uv run --no-sync mkdocs build --strict` passes. +- `uv build --offline` builds wheel/sdist; the wheel metadata contains the complete + updated README verbatim. +- Capability and cost statements are checked against public code/tests and the + committed run-11/run-12 reports. No production, frozen source or raw result changes. + +## Failed Attempts + +The first temporary validation script used `TreeTerm.tree` to inspect stump size; +the actual public field is `TreeTerm.learner`. Correcting that extra check passes. +The README example itself already executed, trained and replayed successfully. + +## Risks and Follow-ups + +The README describes the merged experimental checkpoint, not complete v1 acceptance. +No new GPU, external-baseline, quality or author study is run for this update. +Refresh the capability matrix only when the next construction has its own evidence; +keep test counts and internal timings separate from product/adoption claims. + +## Commits + +- `167ed53`: merged PR 25 and preserved execution history. +- `bbd69ea`: the verified implementation/test checkpoint described in the README. +- This entry accompanies the README documentation commit. diff --git a/v1-sprints/109-pr-integration.md b/v1-sprints/109-pr-integration.md index d240f07..fa0d7f0 100644 --- a/v1-sprints/109-pr-integration.md +++ b/v1-sprints/109-pr-integration.md @@ -1,8 +1,8 @@ # Sprint 109: PR integration checkpoint -Status: local integration checks complete; publish and inspect the PR for branch -`codex/v1-retrospective-plan`. Live hosted-check and merge status belong to that PR; -merge is not yet requested. +Status: complete. All five hosted checks pass at `bbd69ea`; the user's subsequent +merge instruction completes [PR 25](https://github.com/jxucoder/openboost/pull/25) +with merge commit `167ed53`. Original execution commits remain in main's history. This delivery checkpoint follows the completed 108 retrospective and precedes further multiclass construction. No GPU upload, model study or release is included. @@ -121,3 +121,23 @@ Final local regression passes 2,292 tests with one platform skip in 18.23 second using the configured parallel CPU suite. Full lint passes. Commit the correction and let the PR record the next complete hosted matrix; leave the actual merge to the user's next instruction. + +## README follow-up + +The user now requests the latest state and vision in the README. This is a local +documentation update from merged main, with no new construction or evaluation run. + +1. Check public recipes, component boundaries, run-11/run-12 evidence and remaining + requirements against the implementation and tests. +2. Replace the accumulated checkpoint narrative with the vision, current CPU/CUDA + coverage, scoped evidence, a runnable public composition example and next work. +3. Execute the README Python example, check relative links, build strict docs and + packages, and commit the verified update. The smallest executable check is the + exact quickstart, including the custom learner and saved-model round trip. + +The rewritten README passes that quickstart: three accepted rounds, custom +three-node learners, reduced loss, finite predictions and exact persisted replay. +All 34 relative links resolve; strict docs, wheel/sdist builds and production/ +example Ruff pass. Wheel metadata contains the exact new README. The update +clarifies the current capability/evidence boundary without changing production or +launching another evaluation. See the [documentation learning](../learnings/2026-09-08-v1-readme-state-vision.md). diff --git a/v1-sprints/README.md b/v1-sprints/README.md index f6f1ef8..915d6f1 100644 --- a/v1-sprints/README.md +++ b/v1-sprints/README.md @@ -83,9 +83,10 @@ bytes. [The 108 retrospective](108-glm-validation-result.md) is complete; all tw allowances are consumed. Stop here, then begin the required multiclass construction on the next continuation. AFT/vector topology, train-many and formal E4 remain open. -The user's next request starts the [109 PR integration checkpoint](109-pr-integration.md) -before further construction: prepare an English PR, verify hosted CI and preserve -the historical execution commits. Actual merge remains a subsequent action. +The [109 PR integration checkpoint](109-pr-integration.md) is complete: all five +hosted checks pass at `bbd69ea`, and the user's subsequent request merges PR 25 +as `167ed53`, preserving the historical execution commits. The README follow-up +updates the public vision, capability matrix and evidence without new construction. After this checkpoint, advance [required CUDA recipes](080-cuda-required-recipes.md),