Skip to content

docs: clarify train_rows vs full-shard streaming (recipe 1.2) #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
echobt merged 1 commit into from
Aug 7, 2026
+21 −3
Merged

docs: clarify train_rows vs full-shard streaming (recipe 1.2) #5

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 21 additions & 3 deletions docs/getting-started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,12 +71,30 @@ eval as `ChallengeInternal` — never a miner score.
| Hard step cap | 20 000 |
| Source size | 128 KiB per script |
| Model parameters | ≤ **350 000 000** after `build_model` |
| `train_rows` (from `GET /v1/recipe`) | **2048** — baseline / default cut in `ctx` |
| `val_rows` | **256** — frozen val scored by the harness |
Comment on lines +74 to +75

@coderabbitai coderabbitai Bot Aug 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

curl --fail --silent --show-error \
  https://chain.joinbase.ai/v1/recipe |
  jq '{version, recipe_version, train_rows, val_rows, caps, pin_hex}'

Repository: BaseIntelligence/prism

Length of output: 211

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

echo "Changed docs context:"
sed -n '60,105p' docs/getting-started.md

echo
echo "Locate recipe references:"
rg -n 'v1/recipe|GET /v1/recipe|recipe_version|pin_hex|val_rows|train_rows|Production today' docs README.md examples/baseline/training.py 2>/dev/null || true

Repository: BaseIntelligence/prism

Length of output: 3272

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

echo "$1=$1"
echo "API docs candidates:"
cat -n docs/api.md | sed -n '1,80p'

Repository: BaseIntelligence/prism

Length of output: 195

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

echo "docs/api.md route references:"
cat -n docs/api.md | sed -n '1,80p'

echo
echo "All route references in docs and README:"
rg -n 'chain\.joinbase\.ai|/v1/recipe|/recipe/baseline|v1/recipe|recipe version|val_rows|pin_hex' docs README.md 2>/dev/null || true

Repository: BaseIntelligence/prism

Length of output: 4170

Update getting-started.md to document /challenge/prism/v1/recipe.

Docs elsewhere record chain.joinbase.ai as the prod gateway plus the /challenge/prism/... prefix, but these lines and the Always trust live... guidance point to GET /v1/recipe, which is the 404 route. Keep the live URL/path accurate alongside the documented train_rows, val_rows, pin_hex, caps, and 1.2.0 values.

Also applies to: 92-97

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/getting-started.md` around lines 74 - 75, Update the recipe endpoint
references in getting-started.md, including the “Always trust live...” guidance
and the documented train_rows, val_rows, pin_hex, caps, and 1.2.0 values, to use
the live chain.joinbase.ai gateway with the /challenge/prism/v1/recipe path
instead of /v1/recipe.


`train_rows` is what the **sealed baseline** trains on (~2M GPT-2 tokens for
that slice). It is **not** a hard “you only get 2048 rows” ceiling for
competitive recipes: the harness gives you the full pinned parquet at
`ctx["dataset_path"]`, and you may stream it until the 6h / 20k-step guard
fires. Token count then depends on your loop and the GPU — a long Lium run can
reach ~O(10⁹) tokens. Marketing charts that once said “2.6B tokens · single
pass” were showing a leader’s **observed** telemetry, not a fixed recipe
Comment on lines +81 to +83

@coderabbitai coderabbitai Bot Aug 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Use precise notation for the observed token count.

~O(10⁹) mixes approximation and asymptotic notation. Use a clear expression for an observed quantity.

Proposed wording 
-a long Lium run can reach ~O(10⁹) tokens.
+a long Lium run can process on the order of 10⁹ tokens.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
fires. Token count then depends on your loop and the GPU — a long Lium run can
reach ~O(10⁹) tokens. Marketing charts that once said “2.6B tokens · single
pass” were showing a leader’s **observed** telemetry, not a fixed recipe
fires. Token count then depends on your loop and the GPU — a long Lium run can
process on the order of 10⁹ tokens. Marketing charts that once said “2.6B tokens · single
pass” were showing a leader’s **observed** telemetry, not a fixed recipe
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/getting-started.md` around lines 81 - 83, Update the token-count wording
in the getting-started documentation to use precise notation for an observed
quantity, replacing the mixed approximate/asymptotic expression “~O(10⁹)” with a
clear approximate count or range. Keep the surrounding explanation about loop-
and GPU-dependent counts unchanged.

quota. Always trust live `GET /v1/recipe` (`pin_hex`, `train_rows`, caps).

The sealed baseline is deliberately mediocre (short cut, few steps). Matching
a board BPB near ~4–5 requires a competitive trainer, not an unmodified
baseline on a 4090 for a few minutes.

## Recipe pin

`GET /v1/recipe` returns the versioned descriptor (dataset URL/hash, caps, harness
digest, recipe version). `GET /v1/recipe/baseline` returns the official baseline
scripts — the best starting point for your own architecture.
`GET /v1/recipe` returns the versioned descriptor (dataset URL/hash, caps,
`train_rows` / `val_rows`, recipe version, `pin_hex`). Production today is
recipe **1.2.0** — open docs PRs that advertise 1.3+/1.4.0/v3 scoring describe
**unreleased** control-plane work (`prism-better`), not what
`https://chain.joinbase.ai` executes. `GET /v1/recipe/baseline` returns the
Comment on lines +93 to +96

@coderabbitai coderabbitai Bot Aug 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Replace the relative production date.

Production today will become stale and can contradict the live-endpoint guidance. State the exact verification date, such as As of August 7, 2026, or identify this version as the last verified value.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/getting-started.md` around lines 93 - 96, Update the production-status
wording in the surrounding recipe-version documentation to replace “Production
today” with an explicit verification date or clearly identify recipe 1.2.0 as
the last verified production value, while preserving the existing live-endpoint
guidance.

official baseline scripts — the best starting point for your own architecture.

## Next

Expand Down
Loading