From e915832a3304ab166b9b9147f7b81099ea4f0c07 Mon Sep 17 00:00:00 2001 From: rwkeyes <86691029+rwkeyes@users.noreply.github.com> Date: Fri, 13 Nov 2026 12:16:36 -0500 Subject: [PATCH] docs: cite where the 28.8 GB lookup table figure comes from The front page quotes a 28.8 GB lookup table without saying where the number comes from. It is the model's second GGUF file: one per_layer_token_embd.weight tensor, 160 x 320,001,536 values at IQ4_NL. Verified on the pack this branch serves: the shard is 28,800,138,432 bytes, the tensor's packed payload is 160 x 320,001,536 x 18/32 = 28,800,138,240 bytes, and the 192 bytes left over are the GGUF header. Upstream documents the same shape in docs/UNSLOTH_Q6.md and the 28.8 GB figure in HOW_IT_WORKS.md, DETAILS.md, MAC.md and DGX_SPARK_LINUX.md. The glossary entry now says what the number is, that it is decimal GB (26.8 GiB in df), and that a pack storing the table in another type is larger - Q8_0 makes the same table 54.4 GB. --- README.md | 3 ++- docs/GLOSSARY.md | 6 ++++++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 7eb8f408a77..8a89d4a45fd 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,8 @@ engine's performance is not ours. It comes from four decisions: - **Long prompts are read in big pieces** (up to 8,192 tokens), which is why a long document is read at over 1,000 tokens a second, and a follow-up reuses the context already read instead of starting over. - **A 28.8 GB lookup table lives on the SSD** and is read a few rows per token, so it costs disk bandwidth - instead of memory. + instead of memory. It is the model's second file: one `per_layer_token_embd.weight` tensor of 160 × 320,001,536 + values at 4 bits — 28,800,138,240 bytes, decimal GB, 26.8 GiB. - **Guess, then check.** The model's built-in draft layer guesses up to three tokens and the model verifies all of them in one pass (2.4–3.2 tokens per pass), so an answer arrives 1.6–1.8× sooner with the same text. diff --git a/docs/GLOSSARY.md b/docs/GLOSSARY.md index c05e23c2a0a..bfcf292eff5 100644 --- a/docs/GLOSSARY.md +++ b/docs/GLOSSARY.md @@ -57,6 +57,12 @@ guesses; the big model still decides every word, so answers do not change. See a The large lookup table (28.8 GB for Qwen3.8-Flash-Next) that Strata reads a few rows of per token from the SSD instead of holding in memory. `--ple-gguf` points at it; `--ple-io` chooses how it is read. +The size is not a round number chosen by a doc: it is the model's **second file**, one tensor +`per_layer_token_embd.weight` of 160 × 320,001,536 values stored as IQ4_NL (18 bytes per 32 values), which is +160 × 320,001,536 × 18/32 = **28,800,138,240 bytes**. "28.8 GB" throughout these guides is decimal GB — the same +file reads 26.8 GiB in `df`. A pack that stores the table in another type is bigger: Unsloth's UD-Q6_K_XL keeps it +as Q8_0 (170 bytes per 160-value row) and the same table becomes 54.4 GB. + ## The engine **Engine**