From 2a1428454c488a868024b5dafe29e060635e5b3d Mon Sep 17 00:00:00 2001 From: rwkeyes <86691029+rwkeyes@users.noreply.github.com> Date: Fri, 13 Nov 2026 11:44:58 -0500 Subject: [PATCH] docs: a front page and a path for someone who has never used Strata The repository's front page described a 0.2.11 release and opened with the fork's GuideLLM table, so a reader who had never run Strata learned what the fork measured before learning what Strata is, what it needs, or how to start it. The guides themselves were accurate but scattered across 81 pages with no glossary and no entry point. Front page (README.md), rewritten: - what Strata is in plain words, and what this repository is (a fork of Niko1221/Strata, source only, default branch = the v0.2.12 verified snapshot, engine 0.1.58) - a start-here table with the five real paths and their cost in time - hardware and memory stated before any download, not after - a quick start that builds this snapshot, prepares a pack, serves it and proves the binary with --version; the flags are checked against the option parsers in this tree - the terms a newcomer meets on the first day, a first-run failure table, a repository map, and a pointer to DIRIGO.md for the numbers New pages: - docs/GETTING-STARTED.md: two routes (released install, this snapshot), each step with a check that proves it worked, plus model choice, connecting a coding tool and the "is it working" checklist - docs/GLOSSARY.md: every term the guides use, grouped by model / engine / server / measurement, in plain language Quality control: - docs/README.md: start-here block, this-fork section, glossary, and a rule not to hard-code a version number in an instruction - the five pages that claimed "version 0.2.5" now name 0.2.12; the eight pages that named Strata-0.2.9-*.exe installers (which the release page has not carried since 0.2.9) now name the file by its release - docs/UPSTREAM-README.md: relative links were written for the repository root and were dead at docs/; all 60 local references now resolve - DIRIGO.md: the suite-log link pointed at a directory that is not in this repository; the row now points at the record that carries the result and says the raw log is not shipped --- AGENTS.md | 3 +- DIRIGO.md | 18 +- README.de.md | 2 +- README.es.md | 2 +- README.fr.md | 2 +- README.ja.md | 2 +- README.md | 237 +++++++++++++++----- README.pl.md | 2 +- README.pt-BR.md | 2 +- README.zh-CN.md | 2 +- dirigo-results/community-report/README.md | 2 +- docs/AI_REFERENCE.md | 4 +- docs/AI_SETUP.md | 6 +- docs/AMD_HIP.md | 4 +- docs/GETTING-STARTED.md | 256 ++++++++++++++++++++++ docs/GLOSSARY.md | 175 +++++++++++++++ docs/INSTALL.md | 9 +- docs/README.md | 31 ++- docs/SOURCE_INSTALL.md | 2 +- docs/STRIX_HALO.md | 4 +- docs/TROUBLESHOOTING.md | 2 +- docs/UPSTREAM-README.md | 62 +++--- 22 files changed, 716 insertions(+), 113 deletions(-) create mode 100644 docs/GETTING-STARTED.md create mode 100644 docs/GLOSSARY.md diff --git a/AGENTS.md b/AGENTS.md index 165e716b5dd..3867f463181 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,7 +15,8 @@ Strata has three parts: - **The app** (`apps/desktop`, the Tauri shell; `apps/host`, the Node host and the web UI). It is for people who do not want a terminal. One installer per system ([latest release](https://github.com/Niko1221/Strata/releases/latest)): - `Strata-0.2.9-windows-x64-setup.exe`, `Strata-0.2.9-linux-x86_64.AppImage`, `Strata-0.2.9-macos-arm64.dmg`. + `Strata--windows-x64-setup.exe`, `Strata--linux-x86_64.AppImage`, `Strata--macos-arm64.dmg` + (the file names carry the release's version; take them from the release page, not from this list). The Windows and Linux installers carry no engine: the app downloads the engine for the graphics card on first start, or installs a file the user already has. The Mac dmg keeps its small Metal engine inside. - **The Terminal** (`strata.py`, `strata_cli/`, `strata_control/`): the `strata` command. It sets up a model, serves diff --git a/DIRIGO.md b/DIRIGO.md index 538e738c060..6fa56f6d162 100644 --- a/DIRIGO.md +++ b/DIRIGO.md @@ -1,12 +1,20 @@ # Dirigo Strata 0.2.12 — the verified tree, and what was measured -This branch is the **one place** for the Strata tree Dirigo Agents runs and the evidence behind it. It is a single -snapshot commit: upstream Strata **v0.2.12** (`146df3c46`, engine **0.1.58**) plus this fork's carry, in the state that -served `:8095` on vega on 2026-11-08, with every measurement kept under [`dirigo-results/`](#what-is-in-dirigo-results). +This branch is the **one place** for the Strata tree Dirigo Agents runs and the evidence behind it. It is one +snapshot commit — upstream Strata **v0.2.12** (`146df3c46`, engine **0.1.58**) plus this fork's carry, in the state +that served `:8095` on vega on 2026-11-08 — with every measurement kept under +[`dirigo-results/`](#what-is-in-dirigo-results). + +The snapshot is `9fd4e15d` (5,837 tracked files); the tree hash below belongs to it. Commits after it on this branch +change documentation only and leave `src/`, `include/`, `sycl/`, `metal/`, `serve/`, `setup.py` and `CMakeLists.txt` +byte-identical, so a re-clone of the tag plus the carry still reproduces the deployed tree. Nothing here is aspirational: each number below was produced on 2026-11-07/08 on one Intel Arc Pro B70 (32 GB), with the model and configuration named in the same document that carries the result. +> **New to Strata?** This page is the evidence, not the tour. Read [README.md](README.md) for what this fork is and +> [docs/GETTING-STARTED.md](docs/GETTING-STARTED.md) to get running; come back here when you want the numbers. + ## Provenance — what this tree is | | | @@ -29,7 +37,7 @@ the measurement tooling, and a test fix that makes `test_hip_ordinal` independen | --- | --- | --- | | Clean-room reproduction | fresh `git clone --branch v0.2.12` + carry = deployed tree, **same tree hash** `b5335e96a518a5c90e87aaaf92f52bfdf0f6ef04` (5,837 tracked files both) | `dirigo-results/notes/deployment-and-audit-2026-11-08.md` | | Build from scratch | 0 errors, engine **112,593,920 bytes** (same size as deployed), `--version` → `strata 0.1.58`, AOT `bmg-g31` | same | -| Unit / integration suite | **1,535 tests, 611 s, 17 skipped, 0 failures** | [`dirigo-results/unit-tests/suite-0.2.12-deployed-tree.log`](dirigo-results/unit-tests/suite-0.2.12-deployed-tree.log) | +| Unit / integration suite | **1,535 tests, 611 s, 17 skipped, 0 failures** | [deployment record](dirigo-results/notes/deployment-and-audit-2026-11-08.md) — the raw log is not kept in this repository | | Card gate (live service) | minefield **5/5**, one-token 200, extraction at a 24-token budget answered in 0.42 s with **0 reasoning chars**, live `POST /v1/tune` applied | deployment record | | Standard benchmark protocol (`strata-qwen38-thinking-v1-draft.1`) | **status complete, 21/21 requests, 0 failed, 12/12 sanity**; prompt 1,404 tok/s at 4K, 1,773 at 16K; decode 80.8/68.1 tok/s; TTFT 2.94/9.27 s | [`dirigo-results/standard-benchmark/`](dirigo-results/standard-benchmark/) | | Community-format comparison (0.2.11 / 0.2.12 / stock 0.2.12, interleaved) | 0.2.12 decode +0.4…+1.2 % over 0.2.11, within ±0.3 % of stock; needle 6/6 | [`dirigo-results/community-report/`](dirigo-results/community-report/) | @@ -92,7 +100,7 @@ Full write-up: [`dirigo-results/notes/improvements-2026-11-08.md`](dirigo-result | `notes/deployment-and-audit-2026-11-08.md` | the 0.2.11 → 0.2.12 promotion: what changed, the evidence it took, the rollback command, the lever audit, the clean-room reproduction | | `notes/improvements-2026-11-08.md` | every experiment of 2026-11-08 with its numbers, including the ones that failed | | `notes/carry-0212.patch` | the complete delta against tag v0.2.12 — 35 files, +5,024/−264 | -| `unit-tests/` | full suite logs: the deployed tree (1,535, OK) and the clean-room tree before the topology fix | +| `unit-tests/` | not shipped in this repository: the suite result (1,535 tests, OK) is recorded in `notes/deployment-and-audit-2026-11-08.md` | | `standard-benchmark/` | the standard-protocol result folder: `README.md`, `results.json`, `outputs.jsonl`, `config.json`, template and corpus provenance, plus the operator's hardware correction | | `community-report/` | the three-build comparison (0.2.11 / 0.2.12 / stock 0.2.12): README, per-request JSON, the two probe scripts | | `guidellm/` | GuideLLM 0.8.1 results: 0.2.11, 0.2.12, 0.2.12-with-shadow, and the summaries | diff --git a/README.de.md b/README.de.md index c848366828e..dde8c95b5da 100644 --- a/README.de.md +++ b/README.de.md @@ -96,7 +96,7 @@ Unsicher, was passt? [Wähle ein Modell für deinen Computer](docs/MODELS.md#pic Installiere die App, wähle ein Modell und leg mit dem Chat los. Es gibt einen Installer pro System. Beim ersten Start lädt die App die Engine für deine Grafikkarte herunter. [Hilfe zum Installer](docs/INSTALL.md#the-app). -Dateien: `Strata-0.2.9-windows-x64-setup.exe` · `Strata-0.2.9-linux-x86_64.AppImage` · `Strata-0.2.9-macos-arm64.dmg` +Dateien: `Strata--windows-x64-setup.exe` · `Strata--linux-x86_64.AppImage` · `Strata--macos-arm64.dmg`

Strata-Desktop-App mit Chat und Live-Codevorschau nebeneinander diff --git a/README.es.md b/README.es.md index 64f41e5b645..9490e721ecd 100644 --- a/README.es.md +++ b/README.es.md @@ -96,7 +96,7 @@ Medido con el conjunto de 2.00 bpw en Strata 0.2.5. «Salida» es la velocidad d Instala la aplicación, elige un modelo y empieza a chatear. Hay un instalador por sistema. En el primer arranque, la aplicación descarga el motor para tu tarjeta gráfica. [Ayuda con el instalador](docs/INSTALL.md#the-app). -Archivos: `Strata-0.2.9-windows-x64-setup.exe` · `Strata-0.2.9-linux-x86_64.AppImage` · `Strata-0.2.9-macos-arm64.dmg` +Archivos: `Strata--windows-x64-setup.exe` · `Strata--linux-x86_64.AppImage` · `Strata--macos-arm64.dmg`

Aplicación de escritorio de Strata con un chat y una vista previa del código en vivo, una junto a otra diff --git a/README.fr.md b/README.fr.md index 41fd2c453f1..429b92066d9 100644 --- a/README.fr.md +++ b/README.fr.md @@ -96,7 +96,7 @@ Vous ne savez pas ce qui tient ? [Choisir un modèle pour votre ordinateur](docs Installez l'application, choisissez un modèle et commencez à discuter. Il y a un seul installateur par système. Au premier lancement, l'application télécharge le moteur de votre carte graphique. [Aide pour l'installateur](docs/INSTALL.md#the-app). -Fichiers: `Strata-0.2.9-windows-x64-setup.exe` · `Strata-0.2.9-linux-x86_64.AppImage` · `Strata-0.2.9-macos-arm64.dmg` +Fichiers: `Strata--windows-x64-setup.exe` · `Strata--linux-x86_64.AppImage` · `Strata--macos-arm64.dmg`

Application de bureau Strata avec une discussion et un aperçu du code en direct côte à côte diff --git a/README.ja.md b/README.ja.md index fac624042c8..6f8c6419f4b 100644 --- a/README.ja.md +++ b/README.ja.md @@ -96,7 +96,7 @@ Strata 0.2.5 で 2.00 bpw のセットを使って測定しました。「出力 アプリをインストールし、モデルを選んで、チャットを始めましょう。インストーラーはシステムごとに1つです。初回起動時に、アプリがお使いのグラフィックスカード用のエンジンをダウンロードします。[インストーラーのヘルプ](docs/INSTALL.md#the-app)。 -ファイル: `Strata-0.2.9-windows-x64-setup.exe` · `Strata-0.2.9-linux-x86_64.AppImage` · `Strata-0.2.9-macos-arm64.dmg` +ファイル: `Strata--windows-x64-setup.exe` · `Strata--linux-x86_64.AppImage` · `Strata--macos-arm64.dmg`

チャットとライブのコードプレビューを並べて表示した Strata デスクトップアプリ diff --git a/README.md b/README.md index 4baceb9f255..c9cb1273299 100644 --- a/README.md +++ b/README.md @@ -1,60 +1,195 @@ -# Strata_Dirigo — release 0.2.11 +# Strata — Dirigo Agents' verified build -Dirigo Agents' fork of [Strata](https://github.com/Niko1221/Strata) (MIT), based on upstream tag **`v0.2.11`** -(engine **0.1.57**). Source only — no engine binaries, no model weights. Upstream's own README (build, hardware, -setup) is preserved at [`docs/UPSTREAM-README.md`](docs/UPSTREAM-README.md). +**Strata is a local AI engine.** It runs a very large language model on one ordinary PC by sharing the work +between the graphics card, the CPU, the RAM and the SSD — no account, no cloud. It speaks the OpenAI and +Anthropic APIs on `localhost`, so the coding tools you already use can talk to it. -## How it differs from stock v0.2.11 +This repository is **Dirigo Agents' fork** of [Niko1221/Strata](https://github.com/Niko1221/Strata) (MIT). +It is **source only**: no installers, no engine binaries, no model files. Upstream's own README is preserved at +[`docs/UPSTREAM-README.md`](docs/UPSTREAM-README.md). -The engine, kernels and model format are upstream's. The fork adds server-level behaviour and one opt-in engine -knob; with none of them enabled, the server behaves as stock. +

+ The Strata desktop app: a chat, the files it is editing, and a live preview in one window +
Strata running a 24,576-expert model locally, on one graphics card. +

-| difference | what it does | control | -| --- | --- | --- | -| conversation cache routed by traffic shape | a run of single-shot prompts switches the cache **off** (faster prefill); a conversation switches it back **on** (prefix reuse) | `serve/conv_router.py`, `--conv-route off\|shadow\|on` (default `shadow` = classify and log only) | -| thinking policy | suppresses the scratchpad for short/factual asks, keeps it for anything that computes or orders | `serve/prompt_kind.py` + the compute/ordering guard | -| retune between requests | change engine settings on a live server — no restart, no reload | `POST /v1/tune`, `GET /v1/tuning`, `--auto-tune off\|shadow\|on` | -| single-load calibration | one engine load instead of six for FAST / FINE / STOCK | `tools/calibrate.py` | -| per-request draft depth | a client can set its own speculation depth for one request | `spec=N`, inert unless sent | -| build fixes carried | HIP `alignas` fix (upstream #778); oneMKL link narrowed to the BLAS domain | in the tree | - -## GuideLLM — each test, both arms - -Same engine version built from the same tag in both arms. **GuideLLM 0.8.1**, Intel Arc Pro B70 32 GB, model -`qwen3.8-flash-next-coder-iq1_m`, synthetic 512-in / 128-out chat requests, streaming (TTFT measured, not -inferred), `temperature 0`, 45 s per concurrency level, both arms in one session on the same card. Both arms -fully resident — 12,288 expert slots, **0 experts over PCIe**. **0 errors** at every level in both arms. - -| concurrency | TTFT median | ITL median | requests/s | output tok/s | -| --- | --- | --- | --- | --- | -| 1 — stock | 827 ms | 10.4 ms | 0.4 | 56.6 | -| 1 — this fork | **729 ms** (−11.8 %) | **8.9 ms** | **0.5** | **62.6** (+11 %) | -| 2 — stock | 2,831 ms | 9.4 ms | 0.4 | 56.0 | -| 2 — this fork | **2,570 ms** (−9.2 %) | **9.2 ms** | **0.5** | **65.4** (+17 %) | -| 4 — stock | 7,279 ms | 10.5 ms | 0.4 | 54.0 | -| 4 — this fork | **6,044 ms** (−17.0 %) | **8.8 ms** | **0.5** | **65.6** (+21 %) | -| 8 — stock | 9,872 ms | 9.9 ms | 0.4 | 54.4 | -| 8 — this fork | **8,932 ms** (−9.5 %) | 10.8 ms | **0.5** | **59.7** (+10 %) | - -GuideLLM's workload is all unique prompts, so the router settles into cache-off for it. The fork wins every column -except ITL at c=8. Neither arm scales with concurrency — the engine serves one request at a time by design, so c=8 -queues. The pp512 figure behind the c=1 gain is 880.9 vs stock 710.9 tok/s **settled**; measured cold, the first -four requests of a burst read 463.8 while the cache is still on. - -### What GuideLLM cannot show - -Its workload never repeats a context, so the reuse the router buys is invisible to it. Measured separately, same -card, own harness: - -| chat-shaped traffic | stock | this fork | +The default branch, `dirigo/0.2.12-verified`, is one snapshot: upstream tag **v0.2.12** (commit `146df3c4`, +engine **0.1.58**) plus this fork's changes, in the state that served a live machine on 2026-11-08 — with the +measurements behind it kept in [`dirigo-results/`](dirigo-results/). What was verified, and how: +[`DIRIGO.md`](DIRIGO.md). + +> **Never used Strata before?** Start with [Getting started](docs/GETTING-STARTED.md) — it walks from "what is +> this" to your first answer, and explains every word Strata uses in [plain language](docs/GLOSSARY.md). +> +> **Just want to run Strata, not build it?** Use the stock release instead — one line in the terminal and no compiler: +> [Download the release](https://github.com/Niko1221/Strata/releases/latest) · [install guide](docs/INSTALL.md). + +## Start here + +| I want to… | Go to | Time | | --- | --- | --- | -| second request on the same context | 5.29 s | **0.29 s** (17×) | -| first request on a context | 827 ms | 943 ms (+117 ms) | -| 12 short extractions, 8-token budget | 0/12 correct | **9/12 correct** | -| 26-prompt accuracy battery | 88.5 % | **92.3 %** | +| Run Strata on my own PC | [Install (stock release)](docs/INSTALL.md) — installer, or one line in the terminal | 10 min + model download | +| Reproduce the build this fork verified | [Quick start](#quick-start-build-this-snapshot) below · [evidence](DIRIGO.md) | 30–60 min of compiling | +| Connect Claude Code, Codex, OpenCode, Hermes… | `strata connect`, or [tool connections](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect) | 5 min | +| Check my hardware before downloading anything | [What you need](#what-you-need) · [which model fits](docs/MODELS.md#pick-by-memory) | 5 min | +| Understand the engine | [How it works](docs/HOW_IT_WORKS.md) · [technical reference](docs/DETAILS.md) | — | +| Read the evidence for the numbers | [`DIRIGO.md`](DIRIGO.md) · [`dirigo-results/`](dirigo-results/) | — | +| All guides, page by page | [Documentation index](docs/README.md) | — | + +## What you need + +Strata is not a small-model tool. The smallest recommended Qwen3.8-Flash-Next pack needs about **37.6 GB of +memory (RAM plus VRAM)**; the coding-only pack runs in about 32 GB. Unified memory (Apple Silicon, Strix Halo) +counts once. Check [which model fits](docs/MODELS.md#pick-by-memory) before you download tens of gigabytes. + +| | Requirement | +| --- | --- | +| Graphics card | NVIDIA RTX 20/30/40/50 · AMD Radeon RX 7000/9000 and AI PRO R9700 (RX 6000 experimental) · Intel Arc on Linux · Apple Silicon | +| Memory | ~32–55 GB for Qwen3.8-Flash-Next; 46–64 GB RAM plus a 12 GB card for DeepSeek V4.1-Flash | +| Disk | SSD with room for the model — roughly 40 GB for Qwen3.8-Flash-Next, 239–274 GB for DeepSeek V4.1-Flash | +| To build from source | Python 3, CMake 3.24+, Ninja, a C++20 compiler, and your GPU's toolkit (CUDA, ROCm, oneAPI or Xcode) | + +The card-specific details are in [NVIDIA](docs/INSTALL.md#what-you-need), [AMD](docs/AMD_HIP.md), +[Intel Arc](docs/INTEL_ARC.md), [Mac](docs/MAC.md), [multiple GPUs](docs/MULTI_GPU.md). + +## Quick start: build this snapshot + +This is the build the branch was verified with: Intel Arc, SYCL AOT `bmg-g31`, Intel oneAPI 2026.2. AMD and +NVIDIA build from the same sources with their own guides below. + +```sh +# 1. source (the default branch is the verified snapshot) +git clone --branch dirigo/0.2.12-verified https://github.com/rwkeyes/Strata.git +cd Strata + +# 2. Python side +python3 -m venv .venv && . .venv/bin/activate && pip install -r requirements.txt + +# 3. build — needs oneAPI's icpx and oneMKL on PATH: source /opt/intel/oneapi/setvars.sh +cmake -S sycl -B build-sycl-aot -DSTRATA_SYCL_AOT=bmg-g31 +cmake --build build-sycl-aot --target strata -j"$(nproc)" + +# 4. prove what you built before going further +./build-sycl-aot/strata --version # must print: strata 0.1.58 +``` + +The engine reads a **pack** (a prepared model folder) plus the model's GGUF shards. Build the pack from the +GGUF files you downloaded, once per model: + +```sh +python tools/iq_pack.py \ + --gguf /path/to/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf \ + --out packs/iq3xxs +``` + +Then serve. These are the settings the measured service ran with — replace the paths, and keep +`data/expert-profile*.bin` where the binary can find it: + +```sh +./build-sycl-aot/strata --serve \ + --pack packs/iq3xxs --native --ple-gguf \ + --expert-profile data/expert-profile.bin --expert-cache auto --stream-experts --prefill auto \ + --prompt-cache 6 --prompt-cache-every 4096 --batch 0 --spec 6 --spec-min-p 0.5 \ + --mtp --mtp-draft-vocab data/draft_vocab_en.bin --mtp-window 8192 \ + --max-context 131072 --kv int8 --kv-resident 32768 --vram-reserve-mib 700 +``` + +`--serve` is the engine's own token protocol. For a browser or an OpenAI-compatible API, run the bundled +server with those arguments in its config (`args`), and check it answers: -The 117 ms is paid once per fresh context and returned on the first repeat. +```sh +.venv/bin/python -m serve.server --engine strata --config strata.json --port 8080 +curl -s http://127.0.0.1:8080/v1/models +``` + +Other builds: [AMD / HIP](docs/AMD_HIP.md#build) · [NVIDIA / CUDA](docs/SOURCE_INSTALL.md#build-from-git-on-linux-to-try-a-pull-request-or-change-the-code) · +[Mac / Metal](docs/MAC.md) · [the guided `./setup.sh` route](docs/SOURCE_INSTALL.md). + +## Quick start: stock release + +If you want Strata running rather than built, the released app and terminal do the engine choice, download +and model preparation for you: + +```sh +# Linux / Mac +curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh +strata launch +``` + +```powershell +# Windows PowerShell, then open a new terminal +irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex +``` + +`strata launch --plan` prints what would be downloaded and how much memory it needs, and changes nothing. +[Full install guide](docs/INSTALL.md) · [Terminal launch](docs/TERMINAL_LAUNCH.md). + +## Words you will meet + +| Word | Plain meaning | +| --- | --- | +| pack | a prepared model folder the engine reads | +| shard | one part of a model file split across several downloads | +| quantization (Q2_0, IQ3_XXS, IQ1_M) | how much the model was compressed — smaller is faster and lighter, larger is better | +| expert | one small specialist network inside the model; only a few are needed per word | +| PLE table | the big lookup table Strata reads from your SSD instead of holding in memory | +| KV cache | what the model remembers about the conversation; costs memory per token of context | +| slot | one conversation being served at a time (the engine serves one at a time by design) | +| prefill / decode | reading your prompt / writing the answer | +| TTFT | time to first token — how long before the answer starts | +| spec / MTP | the helper that guesses the next words so the big model can check several at once | +| TUNE | the fork's channel for changing engine settings between requests, without a restart | + +Full list, including the fork-specific terms: [Glossary](docs/GLOSSARY.md). + +## If your first run fails + +| Symptom | What to do | +| --- | --- | +| `strata` is not found | Open a new terminal; on Linux/Mac add `~/.local/bin` to `PATH` | +| "Couldn't find an engine for this computer" | `strata update`, then `strata doctor --live` ([Troubleshooting](docs/TROUBLESHOOTING.md)) | +| Build stops at `icpx: not found` | `source /opt/intel/oneapi/setvars.sh` before `cmake` | +| Engine exits before it is ready, after a source build | The expert profile is missing — `data/expert-profile.bin` must be beside the binary's working directory | +| It used the wrong graphics card | `strata-device --list-devices`; pick one with `--device` or `ROCR_VISIBLE_DEVICES` / `CUDA_VISIBLE_DEVICES` | +| Loads, then runs out of memory | Smaller pack, lower `--max-context`, or lower `--kv-resident` | +| Very slow first answer | First start warms the file cache; measure the second request | + +More: [Troubleshooting](docs/TROUBLESHOOTING.md) · [engine errors](docs/DETAILS.md#troubleshooting). + +## What this fork changes + +The engine, kernels and model format are upstream's. This fork adds server-level behaviour and one engine +change; with none of them enabled the server behaves as stock. + +| Change | What it does | +| --- | --- | +| conversation-cache routing | turns the prompt cache off for runs of single-shot prompts, back on for a conversation | +| thinking policy | skips the scratchpad for short, factual asks; keeps it when the answer needs work | +| retune between requests | change engine settings on a live server — no restart, no reload | +| per-request draft depth | a client can set its own speculation depth for one request | + +Measured effect, limits and the full list: [`DIRIGO.md`](DIRIGO.md). The parts that belong upstream are open +as separate branches listed there. + +## Repository map + +| Path | What lives there | +| --- | --- | +| `sycl/` | the Intel Arc build of the engine | +| `src/`, `include/` | the engine core (CUDA, and the shared C++ sources) | +| `metal/` | the Apple Silicon backend | +| `serve/` | the OpenAI / Anthropic compatible server, model preparation, monitoring | +| `strata_cli/`, `strata_control/` | the `strata` terminal command | +| `apps/` | the desktop app | +| `tools/` | pack building, calibration and measurement tools | +| `bench/` | benchmark protocol and community results | +| `docs/` | all guides — [index](docs/README.md) | +| `dirigo-results/` | this fork's measurements, benchmark results and the carry patch | ## Licence -MIT — upstream Strata plus the Dirigo changes; see [`LICENSE`](LICENSE). +MIT — upstream Strata plus the Dirigo changes; see [`LICENSE`](LICENSE). Model files are not part of this +repository and carry their own licences. + +The translated READMEs (`README.de.md`, `README.es.md`, `README.fr.md`, `README.ja.md`, `README.pl.md`, +`README.pt-BR.md`, `README.zh-CN.md`) are upstream's and describe **stock** Strata, not this fork. diff --git a/README.pl.md b/README.pl.md index 6b0dafc3c15..039a8bfb673 100644 --- a/README.pl.md +++ b/README.pl.md @@ -100,7 +100,7 @@ Nie wiesz, co się zmieści? [Wybierz model do swojego komputera](docs/MODELS.md Zainstaluj aplikację, wybierz model i zacznij rozmawiać. Dla każdego systemu jest jeden instalator. Przy pierwszym uruchomieniu aplikacja pobiera silnik dla twojej karty graficznej. [Pomoc z instalatorem](docs/INSTALL.md#the-app). -Pliki: `Strata-0.2.9-windows-x64-setup.exe` · `Strata-0.2.9-linux-x86_64.AppImage` · `Strata-0.2.9-macos-arm64.dmg` +Pliki: `Strata--windows-x64-setup.exe` · `Strata--linux-x86_64.AppImage` · `Strata--macos-arm64.dmg`

Aplikacja Strata na komputer: czat i podgląd kodu na żywo obok siebie diff --git a/README.pt-BR.md b/README.pt-BR.md index a7337c4e05e..1bd703dc576 100644 --- a/README.pt-BR.md +++ b/README.pt-BR.md @@ -96,7 +96,7 @@ Não sabe o que cabe? [Escolha um modelo para o seu computador](docs/MODELS.md#p Instale o aplicativo, escolha um modelo e comece a conversar. Há um instalador por sistema. Na primeira vez que abre, o aplicativo baixa o motor da sua placa de vídeo. [Ajuda com o instalador](docs/INSTALL.md#the-app). -Arquivos: `Strata-0.2.9-windows-x64-setup.exe` · `Strata-0.2.9-linux-x86_64.AppImage` · `Strata-0.2.9-macos-arm64.dmg` +Arquivos: `Strata--windows-x64-setup.exe` · `Strata--linux-x86_64.AppImage` · `Strata--macos-arm64.dmg`

Aplicativo de desktop do Strata com um chat e uma prévia do código ao vivo lado a lado diff --git a/README.zh-CN.md b/README.zh-CN.md index ef4ed349f33..dc8eaa527b5 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -96,7 +96,7 @@ strata launch --family deepseek41 --model ist-das-2.00bpw --plan 安装应用,选择模型,就可以开始聊天。每个系统只有一个安装程序。首次启动时,应用会下载适合你显卡的引擎。[安装程序帮助](docs/INSTALL.md#the-app)。 -文件: `Strata-0.2.9-windows-x64-setup.exe` · `Strata-0.2.9-linux-x86_64.AppImage` · `Strata-0.2.9-macos-arm64.dmg` +文件: `Strata--windows-x64-setup.exe` · `Strata--linux-x86_64.AppImage` · `Strata--macos-arm64.dmg`

Strata 桌面应用,聊天和实时代码预览并排显示 diff --git a/dirigo-results/community-report/README.md b/dirigo-results/community-report/README.md index ba489fc7233..6c631d77efe 100644 --- a/dirigo-results/community-report/README.md +++ b/dirigo-results/community-report/README.md @@ -4,7 +4,7 @@ Measured on 2026-11-07 by [@rwkeyes](https://github.com/rwkeyes). One Intel Arc configuration, three builds, run in six interleaved engine windows so machine drift lands on all three arms instead of one. -This is a **community report in the format of [docs/COMMUNITY_BENCHMARKS.md](../../../docs/COMMUNITY_BENCHMARKS.md)**, +This is a **community report in the format of [docs/COMMUNITY_BENCHMARKS.md](../../docs/COMMUNITY_BENCHMARKS.md)**, not a standard-protocol run: it carries no protocol ID, the sampling is greedy (`temperature 0`) rather than the model card's, and there is no twelve-case sanity suite. The main limitation is at the bottom, together with the two tuning observations that come with controls of their own. diff --git a/docs/AI_REFERENCE.md b/docs/AI_REFERENCE.md index 4f8cd29d0ba..7c9d3c030d8 100644 --- a/docs/AI_REFERENCE.md +++ b/docs/AI_REFERENCE.md @@ -10,8 +10,8 @@ holds the technical context behind the short [README](../README.md). Use [AI_SET - Strata runs supported language models locally, with a desktop app, a terminal app, and an inference engine. - The engine uses the GPU, system RAM and, for applicable models and memory modes, SSD storage. See [how it works](HOW_IT_WORKS.md) and [engine details](DETAILS.md). -- There is one app installer per system (`Strata-0.2.9-windows-x64-setup.exe`, `Strata-0.2.9-linux-x86_64.AppImage`, - `Strata-0.2.9-macos-arm64.dmg`). The Windows and Linux installers carry no engine: the app downloads the engine for the +- There is one app installer per system (`Strata--windows-x64-setup.exe`, `Strata--linux-x86_64.AppImage`, + `Strata--macos-arm64.dmg`). The Windows and Linux installers carry no engine: the app downloads the engine for the graphics card on first start, or installs a file the user already has. The Mac dmg keeps its Metal engine inside. The Terminal's `strata launch` downloads the appropriate engine and model. See [installation](INSTALL.md) and [Terminal launch](TERMINAL_LAUNCH.md). diff --git a/docs/AI_SETUP.md b/docs/AI_SETUP.md index 5886a6d8329..41c75a0ea4b 100644 --- a/docs/AI_SETUP.md +++ b/docs/AI_SETUP.md @@ -1,11 +1,11 @@ # Set up Strata for a user -Use this guide when a user asks an AI agent to install Strata (version 0.2.5). There are two release routes. Pick +Use this guide when a user asks an AI agent to install Strata (version 0.2.12). There are two release routes. Pick one; do not start from a source checkout. - **The Terminal and `strata launch`** (below). This is the route for an agent that has a shell. -- **The desktop app.** One installer per system: `Strata-0.2.9-windows-x64-setup.exe`, - `Strata-0.2.9-linux-x86_64.AppImage`, `Strata-0.2.9-macos-arm64.dmg`, from the +- **The desktop app.** One installer per system: `Strata--windows-x64-setup.exe`, + `Strata--linux-x86_64.AppImage`, `Strata--macos-arm64.dmg`, from the [latest release](https://github.com/Niko1221/Strata/releases/latest). The Windows and Linux installers carry no engine: on first start the app shows the engine for the user's graphics card and downloads it, or installs a file the user already has ("I already have the file"). Details: [INSTALL.md](INSTALL.md). diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.md index 2a8cb78a5c6..d6cb66918a8 100644 --- a/docs/AMD_HIP.md +++ b/docs/AMD_HIP.md @@ -49,7 +49,7 @@ carries it but it is untested. Details: [MODELS.md](MODELS.md#deepseek-v41-flash ## Ready-made packages The Terminal one-liner and `strata update` pick the package whose GPU code covers the card. The app installer -(`Strata-0.2.9-windows-x64-setup.exe`, `Strata-0.2.9-linux-x86_64.AppImage`) has no engine inside: on its first start the app +(`Strata--windows-x64-setup.exe`, `Strata--linux-x86_64.AppImage`) has no engine inside: on its first start the app downloads the matching package for the card. You can also press **Update engine** in Settings (or run `strata update`). | Package (variant) | OS | GPU code inside | ROCm | @@ -131,7 +131,7 @@ reused. The Monitor shows the card's load, VRAM, temperature and power from Linu ## Windows -Install the [app](INSTALL.md) (`Strata-0.2.9-windows-x64-setup.exe` from the +Install the [app](INSTALL.md) (`Strata--windows-x64-setup.exe` from the [release page](https://github.com/Niko1221/Strata/releases/latest); it downloads the engine for the card), or install the [Terminal](INSTALL.md#the-terminal) and run `strata launch`. You need Windows 10 or 11 (64-bit), a supported card and a current AMD driver. No ROCm or HIP SDK, no compiler, no admin rights. `START-HERE.bat --backend hip` is a diff --git a/docs/GETTING-STARTED.md b/docs/GETTING-STARTED.md new file mode 100644 index 00000000000..0c3a86d516a --- /dev/null +++ b/docs/GETTING-STARTED.md @@ -0,0 +1,256 @@ +# Getting started with Strata + +This page assumes you have never run Strata and takes you to your first answer. It covers both routes: +the **released app / terminal** (recommended for a first install) and **this fork's source snapshot** (if you +want the verified tree itself). Every word Strata uses is explained in the [glossary](GLOSSARY.md). + +Budget about an hour, most of it downloading a model over a normal connection. You will need roughly 40 GB of +free SSD space and 40 GB of memory for the smallest recommended model. + +## 0. The thirty-second reality check + +Strata runs a very large model by keeping the parts that do not fit in VRAM in RAM and on the SSD. That is why +it needs more memory than a normal local-AI setup: + +| Model pack | Memory (RAM + VRAM) | Speed | Quality | +| --- | ---: | --- | --- | +| Qwen3.8-Flash-Next Q2_0 | 37.6 GB | fastest | good | +| Qwen3.8-Flash-Next IQ2_XS | 39.2 GB | fast | better | +| Coder IQ1_M (coding only) | ~32 GB | fast | good at code | +| Qwen3.8-Flash-Next IQ3_S | 54.8 GB | slowest | best | + +> **Measured on:** the sizes above are upstream's estimates for model weights in RAM + VRAM, per +> [Models](MODELS.md#pick-by-memory); leave headroom for the OS and your other programs. Unified memory +> (Apple Silicon, Strix Halo) counts once. + +If you have less memory than the first row, Strata can still stream the model from an SSD — it works and is much +slower. If you want a smaller model instead, see [Qwen3.6-35B-A3B](QWEN36.md) (32–52 GB) or the +[comparison page](MODELS.md). + +Check your card is supported before downloading anything: [what you need](INSTALL.md#what-you-need) → +[AMD](AMD_HIP.md) · [Intel Arc](INTEL_ARC.md) · [Mac](MAC.md) · [older GPUs](OLDER_GPUS.md). + +## 1. Pick a route + +| | Released app / terminal | This fork's source snapshot | +| --- | --- | --- | +| Best for | running Strata, using it daily | reproducing Dirigo's verified build, testing a change | +| Needs a compiler | no | yes (oneAPI, CUDA, ROCm or Xcode) | +| Install time | ~10 min plus model download | 30–60 min of compiling plus model download | +| Gets you | `strata launch`, the desktop app, `strata connect` | `build-sycl-aot/strata` and the bundled server | +| Follow | [Route A](#route-a-released-strata-recommended) | [Route B](#route-b-this-snapshot-from-source) | + +Either way the model files are the same, and both listen on `localhost` with an OpenAI-compatible API. + +## Route A: released Strata (recommended) + +### A1. Install the terminal app + +Linux or Mac: + +```sh +curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh +``` + +Windows PowerShell (then open a **new** terminal): + +```powershell +irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex +``` + +The installer prints where `strata` went (normally `~/.local/bin/strata`). + +**Check:** `strata --version` prints a version. If the command is not found, use the full path the installer +printed and add that folder to your `PATH`. + +If you would rather have a window than a terminal, install the desktop app instead — the +[install guide](INSTALL.md#the-app) has the one file per system. The app downloads the engine for your card on +first start. + +### A2. Look before you download + +```sh +strata launch --plan +``` + +`--plan` prints what would be downloaded, where, and how much memory the chosen model needs. It changes nothing. +To choose a model and provider explicitly: + +```sh +strata launch --plan --family qwen --model iq2_xs +``` + +The recommended provider order is ISTA-DASLab, then Unsloth, then Gyro. The packs in the table above are the +`--model` values: `q2_0`, `iq2_xs`, `iq3_xxs`, `iq3_s` for Qwen3.8-Flash-Next, and the coding pack is its own +family — `strata launch --family coder --model IQ1_M`. + +### A3. Start it + +```sh +strata launch +``` + +Follow the screens: model, context length, storage folder, access settings. The monitor opens when the engine +is up. A model can take several minutes to load the first time; later loads reuse what the system has cached. + +**Check:** the monitor shows a model name, a memory figure and a request counter that moves when you send a +prompt. `strata launch --status` in another terminal should say the model is running. + +Prefer commands? `strata launch --yes --no-tui` accepts the shown choices and stays in the background. + +### A4. Use it + +- Chat in the terminal's own interface, or open the address it printed (default `http://127.0.0.1:8080`). +- Point a coding tool at it: `strata connect` prints the setup line for Claude Code, Codex, OpenCode, Hermes + and others. [Connection guide](TERMINAL_LAUNCH.md#connect-your-apps-strata-connect). +- Stop it with `strata launch --stop`. Update with `strata update --check` and then `strata update`. + +## Route B: this snapshot, from source + +Use this route to reproduce the build in this repository. It is the Intel Arc SYCL build (oneAPI 2026.2, AOT +for `bmg-g31`, engine 0.1.58); for AMD, NVIDIA or Mac, use that backend's build commands and the same source. + +### B1. Get the source and the Python side + +```sh +git clone --branch dirigo/0.2.12-verified https://github.com/rwkeyes/Strata.git +cd Strata +python3 -m venv .venv && . .venv/bin/activate && pip install -r requirements.txt +``` + +### B2. Build + +You need CMake 3.24+, Ninja, a C++20 compiler and Intel oneAPI (DPC++ `icpx` and oneMKL) on `PATH`: + +```sh +source /opt/intel/oneapi/setvars.sh # or wherever oneAPI is installed +cmake -S sycl -B build-sycl-aot -DSTRATA_SYCL_AOT=bmg-g31 +cmake --build build-sycl-aot --target strata -j"$(nproc)" +``` + +`bmg-g31` is the Arc Pro B70. For a B580, B570 or Pro B60 use `bmg-g21`; on an A-series (Alchemist) card leave +`-DSTRATA_SYCL_AOT` out and the GPU code is compiled at first start instead. A build for the wrong card is a +build that stops at startup, not a slow run. + +**Check:** the last line says it linked `strata`, and + +```sh +./build-sycl-aot/strata --version # strata 0.1.58 +``` + +The version string is the point of this step. A successful build of some other tree is not evidence that you +have this one; the number is. Budget ~17 minutes of compiling on a 10-core machine for the SYCL AOT build. + +Other backends: [AMD / HIP](AMD_HIP.md#build) · [NVIDIA / CUDA](SOURCE_INSTALL.md#build-from-git-on-linux-to-try-a-pull-request-or-change-the-code) · +[Mac / Metal](MAC.md). The guided `./setup.sh` route builds, prepares the model and writes the run settings for +you — [source install](SOURCE_INSTALL.md). + +### B3. Prepare the model + +The engine reads a **pack**: a prepared folder with an index, the dense weights and the expert layout. Build it +once per model from the GGUF files from Hugging Face: + +```sh +python tools/iq_pack.py \ + --gguf /path/to/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf \ + --out packs/iq3xxs +``` + +The pack is finished only when it contains `native_experts.txt`; a pack without it will not load. Keep the pack, +the tokenizer folder and the GGUF shards together. + +### B4. Serve the engine + +```sh +./build-sycl-aot/strata --serve \ + --pack packs/iq3xxs --native --ple-gguf \ + --expert-profile data/expert-profile.bin --expert-cache auto --stream-experts --prefill auto \ + --prompt-cache 6 --prompt-cache-every 4096 --batch 0 --spec 6 --spec-min-p 0.5 \ + --mtp --mtp-draft-vocab data/draft_vocab_en.bin --mtp-window 8192 \ + --max-context 131072 --kv int8 --kv-resident 32768 --vram-reserve-mib 700 +``` + +These are the settings the verified service ran with; adapt the paths and the context to your card. +[DIRIGO.md](../DIRIGO.md) records what they produced. Start smaller if unsure — a 4K–8K `--max-context` with a +matching `--kv-resident` is a smoke test, not a model limit. + +**Check:** the engine prints its load progress and then waits. If it exits immediately, read the message: a +missing `data/expert-profile.bin`, a pack without `native_experts.txt` and a card the binary was not compiled +for are the three usual causes. + +### B5. Put the HTTP API in front of it + +`--serve` is the engine's own token protocol. For a browser, a coding tool or `curl`, run the bundled server +with those arguments in its config (`args`) — a working example is in +[the local server example](ORCA.md#local-server): + +```sh +.venv/bin/python -m serve.server --engine strata --config strata.json --port 8080 +``` + +**Check:** + +```sh +curl -s http://127.0.0.1:8080/v1/models +``` + +It returns the model name, and then a first answer: + +```sh +curl -s http://127.0.0.1:8080/v1/chat/completions \ + -H 'Content-Type: application/json' \ + -d '{"model":"local","messages":[{"role":"user","content":"Say hello in five words."}]}' +``` + +## Choose a model + +- **Download sizes and what fits:** [Models](MODELS.md#pick-by-memory). +- **Providers:** ISTA-DASLab (original packs), Unsloth (4–8 bit), Gyro (least memory, slower), Swift 1.5, + NVFP4, OrcaRouter. +- **Quantization names** (`Q2_0`, `IQ2_XS`, `IQ3_XXS`, `IQ1_M`): [explained in the glossary](GLOSSARY.md#the-model). + Roughly: Q2_0 is the fastest, IQ3_S the most faithful, and `IQ1_M` is the coding-only pack for a small machine. +- **Adding or switching models:** [Models § adding](MODELS.md#adding-or-switching-models). + +## Connect a coding tool + +The server speaks OpenAI (`/v1/chat/completions`) and Anthropic (`/v1/messages`). With the released terminal, +`strata connect` prints ready-made settings for Claude Code, Codex CLI, OpenCode, pi.dev, Hermes Agent, +OpenClaw and others — [connection guide](TERMINAL_LAUNCH.md#connect-your-apps-strata-connect). Otherwise point +the tool at `http://127.0.0.1:8080/v1` as an OpenAI-compatible base URL. + +## If it is slow + +| What you see | Where to look | +| --- | --- | +| First answer takes minutes, later ones are fast | Expected: the first start warms the file cache | +| Answers are slow from the first token | Smaller pack, lower `--max-context`, fewer resident KV tokens | +| Very fast card, still slow | RAM is the bottleneck — check `strata doctor --live`, and that the experts fit | +| Everything slow and the disk is busy | The model is streaming from the SSD; free RAM or use a smaller pack | +| Fine in a terminal, slow in an app | The app and the terminal may be using different engines: check `strata launch --status` | + +Measurement guidance — what a number means, and how to compare two configurations honestly — is in +[details](DETAILS.md) and [performance notes](AMD_HIP_PERFORMANCE.md) for AMD. + +## What "working" looks like + +- [ ] `strata --version` (or `./build-sycl-aot/strata --version`) prints the version you meant to install. +- [ ] The model is running: `strata launch --status` says so, or the engine process is alive. +- [ ] `/v1/models` answers on the port you expect. +- [ ] One real request returns an answer, not an error. +- [ ] You can stop it cleanly: `strata launch --stop` ends the model, and the port stops answering. + +If a box is ticked falsely, that is a bug report worth filing: use +[upstream's issue templates](https://github.com/Niko1221/Strata/issues/new/choose) for engine or install bugs. +This fork's issue tracker is off — changes and corrections come in as pull requests. + +## Where to read next + +| Topic | Page | +| --- | --- | +| Every Strata word, in plain language | [Glossary](GLOSSARY.md) | +| Install, update, where files go | [Install](INSTALL.md) | +| Build from a source checkout | [Source install](SOURCE_INSTALL.md) | +| Terminal commands and options | [Terminal launch](TERMINAL_LAUNCH.md) · [Terminal reference](TERMINAL.md) | +| How the engine works | [How it works](HOW_IT_WORKS.md) · [Details](DETAILS.md) | +| This fork's verified build and evidence | [DIRIGO.md](../DIRIGO.md) | +| All guides | [Documentation index](README.md) | diff --git a/docs/GLOSSARY.md b/docs/GLOSSARY.md new file mode 100644 index 00000000000..c05e23c2a0a --- /dev/null +++ b/docs/GLOSSARY.md @@ -0,0 +1,175 @@ +# Glossary + +Every word Strata's guides use, in plain language. Sorted into groups rather than one alphabet, because the +words come in families: the **model**, the **engine**, the **server**, and **measuring** it. + +If a term you met is not here, it is a bug — say so on the [documentation index](README.md). + +## The model + +**Model / model family** +A set of weights. This repository's guides cover Qwen3.8-Flash-Next (with the Coder variant), Qwen3.6-35B-A3B +and DeepSeek V4.1-Flash. Each family has its own packs, memory needs and support. + +**GGUF** +The file format the weights arrive in: a set of tensors plus a header describing them, split into parts when +large. The engine reads them directly. + +**Shard** +One part of a split GGUF. A model can be one file or several (`…-00001-of-00002.gguf`). All parts are needed; +a missing shard stops the pack build. + +**Pack** +A folder the engine loads instead of the GGUF: an `index.txt` describing every tensor, the dense weights +(`dense.bin`), optionally the expert blobs (`experts.bin`), the expert layout (`native_experts.txt`), and the +`tokenizer/` folder. Packs are built once per model with `tools/iq_pack.py`. A pack without +`native_experts.txt` is unfinished and will not load. + +**Quantization** +How much the weights were compressed. Smaller numbers are smaller and faster and lose a little quality: + +| Name | Roughly | Typical use | +| --- | --- | --- | +| `Q2_0` | 2 bits, uniform | fastest, least memory, "good" quality | +| `IQ2_XS` | ~2 bits, importance-aware | a little better at a little more size | +| `IQ3_XXS` | ~3 bits | the usual balance | +| `IQ3_S` | ~3 bits, larger | the most faithful of the small packs | +| `IQ1_M` | ~1 bit, importance-aware, coding-only | the Coder pack for a small machine | + +The i-quants (`IQ…`) keep the experts in their GGUF form; `Q2_0` cannot express them and has its own pack +form. Sizes and what fits: [Models](MODELS.md#pick-by-memory). + +**Expert / mixture of experts (MoE)** +The model is a team of small specialist networks — 24,576 of them in Qwen3.8-Flash-Next — and each word needs +only about ten. Strata can therefore keep most of them out of VRAM without losing quality. + +**Router** +The small network that decides which experts a given word needs. It runs on the card for every token. + +**Shared expert** +An expert used for every token, so it always lives on the card. + +**Draft layer (MTP)** +A small built-in model that guesses the next few words so the big model can check several at once. It only +guesses; the big model still decides every word, so answers do not change. See also **spec**. + +**PLE table** +The large lookup table (28.8 GB for Qwen3.8-Flash-Next) that Strata reads a few rows of per token from the SSD +instead of holding in memory. `--ple-gguf` points at it; `--ple-io` chooses how it is read. + +## The engine + +**Engine** +The program that runs the model (`strata`). Separate from the app and from the terminal command, and versioned +separately — an app version and an engine version are different numbers. + +**Backend** +Which graphics API the engine was compiled for: **CUDA** (NVIDIA), **HIP** (AMD), **SYCL** (Intel Arc), +**Metal** (Apple Silicon). A binary built for one backend does not run on another card. + +**VRAM / RAM / unified memory** +Memory on the graphics card / the system's memory / one pool shared between them (Apple Silicon, Strix Halo). +Strata needs both VRAM and RAM; unified memory counts once. + +**Expert cache** +The part of VRAM that holds the most-used experts. It fills the card and adapts while you chat. Each extra GB +of VRAM holds roughly 700 more experts. + +**Expert profile** +A small file (`data/expert-profile.bin`) telling the engine which experts to start with. Without it the engine +exits on start — a source build needs it beside the working directory. + +**`--stream-experts`** +Stream the next layer's experts to the card while the current layer computes, instead of waiting for a whole +layer to arrive. + +**KV cache** +What the model remembers about the conversation. It costs memory per token of context, which is why context +length and `--kv` precision show up in every memory table. `--kv int8` stores it at 8 bits; `--kv-resident` +keeps N tokens of it on the card. + +**Context / `--max-context`** +How many tokens the model can see at once, prompt plus answer. Larger is more useful and needs more memory. + +**Slot** +One request being served. The engine serves one at a time by design, so extra concurrent requests queue. + +**Prefill / decode** +Reading your prompt (fast, batched, thousands of tokens per second) / writing the answer (one token at a +time). Different speeds; a single "tokens per second" number usually means decode. + +**Spec / speculation / `--spec`** +How many tokens the draft layer guesses ahead. Guessing more can help, guessing badly costs — `--spec-min-p` +is the confidence floor for accepting a guess. + +**Prompt cache** +Keeping the processed prompt so a follow-up on the same conversation does not re-read it. `--prompt-cache` +controls how many, `--prompt-cache-every` how often it is refreshed. + +**Thinking / reasoning effort** +Some models write a scratchpad before answering. It can help on hard problems and costs time and tokens; the +app, the terminal and the server can all turn it down or off. + +**AOT** +Ahead-of-time compilation of the GPU code. For Intel Arc, `-DSTRATA_SYCL_AOT=bmg-g31` compiles for the Arc +Pro B70, `bmg-g21` for B580/B570/Pro B60; leave it empty on A-series cards and the code is compiled at first +start instead. + +**ISA floor (AVX, AVX2)** +The CPU instruction set the engine requires for the experts the card does not hold. Ready-made x86 engines +need AVX2; source builds can be made for older CPUs, which is experimental. + +## The server and this fork + +**Server / `serve/`** +The HTTP layer in front of the engine: an OpenAI-compatible API (`/v1/chat/completions`, `/v1/models`), an +Anthropic-compatible one (`/v1/messages`), and the app-facing endpoints. `--serve` alone is the engine's own +internal protocol, not HTTP. + +**Terminal / `strata`** +The installed command-line product: `strata launch`, `strata connect`, `strata doctor`, `strata update`, +`strata models`. It picks the right engine, prepares models, and manages the server. + +**Run config** +The JSON file the terminal or setup writes (`strata-.json`). It holds the engine `args`, the model paths +and the server's own settings. Setup can rewrite the fields it owns — back it up before editing. + +**TUNE / retune (this fork)** +A channel for changing engine settings on a running engine, between requests, without a restart or reload. +`GET /v1/tuning` reports the current settings; `POST /v1/tune` changes them. Stock upstream Strata has no such +channel. + +**conv route / conversation-cache routing (this fork)** +A server-side rule that turns the prompt cache off during a run of one-shot prompts (faster prefill) and back +on for a conversation (prefix reuse). Modes: `off`, `shadow` (classify and log only), `on`. + +**Carry** +This fork's delta on top of an upstream tag: 35 files, +5,024/−264 for v0.2.12. Kept as one patch in +[`dirigo-results/notes/carry-0212.patch`](../dirigo-results/notes/carry-0212.patch) so the difference from the +tag is auditable in one file. + +**Verified tree (this fork)** +The branch `dirigo/0.2.12-verified`: upstream v0.2.12 plus the carry, reproduced in a clean room to the same +git tree hash, built from scratch, tested and gated. [DIRIGO.md](../DIRIGO.md) has the evidence. + +## Measuring it + +**TTFT** +Time to first token — how long from sending a prompt until the answer starts. Dominated by prefill. + +**ITL** +Inter-token latency — the gap between tokens while the answer streams. Its inverse, with overhead, is the +decode rate. + +**tok/s** +Tokens per second. Say whether it is prompt (prefill) or answer (decode): they differ by an order of magnitude. + +**Sweep / A-B** +Changing one setting and measuring both values under the same conditions. A single number from one run is not +an A-B; interleaved runs are what make a difference believable. + +**Standard benchmark protocol** +Upstream's fixed method (`strata bench run`, protocol `strata-qwen38-thinking-v1-draft.1`): frozen prompts, +model-card sampling, a validated `results.json`. Different from a **community report**, which is anyone's own +measurement in a documented format. [Benchmark protocol](BENCHMARK_PROTOCOL.md) · +[community benchmarks](COMMUNITY_BENCHMARKS.md). diff --git a/docs/INSTALL.md b/docs/INSTALL.md index f8bd61af205..0d0a4792c66 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -1,6 +1,6 @@ # Install Strata -This page is for anyone installing or updating Strata (version 0.2.5). Use the **desktop app** for a graphical +This page is for anyone installing or updating Strata (version 0.2.12). Use the **desktop app** for a graphical interface, or the **Terminal** for `strata launch` and coding tools. Both use released engine packages. A normal install does not need a source checkout or a startup `.bat` file. @@ -38,10 +38,11 @@ hardware drivers. Models and updates need an internet connection to download. 1. Open the [latest release](https://github.com/Niko1221/Strata/releases/latest). 2. Download the one installer for your system. There is one per system, not one per graphics card: - - **Windows:** run `Strata-0.2.9-windows-x64-setup.exe`. It installs for your user and needs no administrator rights. - - **Linux:** make `Strata-0.2.9-linux-x86_64.AppImage` executable, then open it. If FUSE is unavailable, run the + - **Windows:** run `Strata--windows-x64-setup.exe` (the name carries the release's version). It installs + for your user and needs no administrator rights. + - **Linux:** make `Strata--linux-x86_64.AppImage` executable, then open it. If FUSE is unavailable, run the downloaded file with `--appimage-extract-and-run`. [Linux help](TROUBLESHOOTING.md). - - **Mac:** open `Strata-0.2.9-macos-arm64.dmg`, drag Strata into Applications, then open it. [Mac help](MAC.md). + - **Mac:** open `Strata--macos-arm64.dmg`, drag Strata into Applications, then open it. [Mac help](MAC.md). 3. On Windows and Linux the installer has no engine inside. On first start the app shows the engine for your graphics card with its size and one **Download** button, with progress and resume. If you already downloaded the engine file, choose **I already have the file** and it installs without internet. The Mac file keeps its small diff --git a/docs/README.md b/docs/README.md index 945cfb21466..cb5aea4f39f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,6 +1,23 @@ # Strata guides -**New here? Start with [Install Strata](INSTALL.md).** Download the one installer for your system (Windows, Linux or Mac), or install the Terminal with one line. Both use released engine packages. This documentation is for version 0.2.5. +**New here? Read [Getting started](GETTING-STARTED.md) first.** It takes you from "what is this" to a first +answer, on either the released install or this repository's source snapshot. Every word Strata uses is in the +[glossary](GLOSSARY.md). + +These guides describe the tree in this repository: upstream Strata **v0.2.12** (engine **0.1.58**) plus this +fork's changes. For the fork's verified build, what was measured, and the evidence, read +[DIRIGO.md](../DIRIGO.md). + +## Start here + +| You are… | Start with | +| --- | --- | +| Completely new to Strata | [Getting started](GETTING-STARTED.md) | +| Installing, updating, or wondering where files went | [Install](INSTALL.md) | +| Building from a source checkout | [Source install](SOURCE_INSTALL.md) | +| Wondering whether your PC can run it | [Which model fits](MODELS.md#pick-by-memory) | +| Stuck on an error | [Troubleshooting](TROUBLESHOOTING.md) | +| Reading this fork's numbers | [DIRIGO.md](../DIRIGO.md) · [`dirigo-results/`](../dirigo-results/) | ## Use Strata @@ -9,6 +26,7 @@ | Install or update | [Desktop app and Terminal](INSTALL.md) | | Choose a model and check memory | [Models and quantizations](MODELS.md) | | Run DeepSeek V4.1-Flash (app and Terminal) | [DeepSeek V4.1-Flash](MODELS.md#deepseek-v41-flash) | +| Run Qwen3.6-35B-A3B | [Qwen3.6-35B-A3B](QWEN36.md) | | Start from the command line | [Terminal launch](TERMINAL_LAUNCH.md) | | Connect a coding tool | [Tool connections](TERMINAL_LAUNCH.md#connect-your-apps-strata-connect) | | Fix an install or startup problem | [Troubleshooting](TROUBLESHOOTING.md) | @@ -39,6 +57,13 @@ Use the guide for your GPU and operating system. Model and feature support can d - [KVA: approximate prompt processing for the late layers (experimental, opt-in)](KVA.md) - [App development](../apps/host/README.md) and [packaging](../packaging/README.md) +## This fork + +- [Verified build, measurements and evidence](../DIRIGO.md) +- [Measurements, suite logs and the carry patch](../dirigo-results/) +- [Glossary of every term](GLOSSARY.md) +- The parts that belong upstream are open as separate branches, listed in [DIRIGO.md](../DIRIGO.md#upstream) + ## AI tools and media - [Install for a user](AI_SETUP.md) @@ -59,7 +84,9 @@ does not establish support or speed on every GPU. ## Writing a guide page - Installation steps live in [INSTALL.md](INSTALL.md), source steps in [SOURCE_INSTALL.md](SOURCE_INSTALL.md). The public READMEs stay short. -- Check command examples against the parser and filenames against the release assets. +- Check command examples against the parser and filenames against the release assets. Do not hard-code a version + number in an instruction: the release page names the file, and this repository is a snapshot that will age. +- Write for a reader who has never seen the term before; link the [glossary](GLOSSARY.md) instead of re-explaining. - A guide describes the current release. Version history, fixed bugs and superseded settings go in a closing **History / older measurements** section, in `bench/results`, or in the release notes. Do not mix them into instructions. diff --git a/docs/SOURCE_INSTALL.md b/docs/SOURCE_INSTALL.md index e7b1270e25c..1f22d06a5ce 100644 --- a/docs/SOURCE_INSTALL.md +++ b/docs/SOURCE_INSTALL.md @@ -1,7 +1,7 @@ # Install and build from a source checkout Use this guide to work on Strata's code or maintain a source-based installation. It is for developers. -To just use Strata (version 0.2.5), install the [desktop app or the Terminal](INSTALL.md). Installed apps do not use `START-HERE.bat`. +To just use Strata (version 0.2.12), install the [desktop app or the Terminal](INSTALL.md). Installed apps do not use `START-HERE.bat`. The repository includes `START-HERE.bat`, `SETUP.bat`, `UPDATE.bat`, `setup.sh` and `update.sh`. Run them from a cloned or extracted **source folder**. They may install dependencies, prepare model files and diff --git a/docs/STRIX_HALO.md b/docs/STRIX_HALO.md index d8c58c55e0d..3b61365bde9 100644 --- a/docs/STRIX_HALO.md +++ b/docs/STRIX_HALO.md @@ -23,8 +23,8 @@ Use the [current release](https://github.com/Niko1221/Strata/releases/latest). Y | System | What to install | Engine variant | | --- | --- | --- | -| Linux | `Strata-0.2.9-linux-x86_64.AppImage` (it downloads the engine on first start), or `strata launch` | `linux-x86_64-hip-halo` | -| Windows | `Strata-0.2.9-windows-x64-setup.exe` (it downloads the engine on first start), or `strata launch` | `windows-x86_64-hip-rdna3` | +| Linux | `Strata--linux-x86_64.AppImage` (it downloads the engine on first start), or `strata launch` | `linux-x86_64-hip-halo` | +| Windows | `Strata--windows-x64-setup.exe` (it downloads the engine on first start), or `strata launch` | `windows-x86_64-hip-rdna3` | Packaged engines carry the matching hipBLASLt tuning table for gfx1151 and select it on their own. Setup recognizes `gfx1151` and PCI ID `1002:1586`; other integrated Radeon chips have separate support limits. Unified memory is counted once. Setup does not diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md index e1a93f7f587..cf68501c7fa 100644 --- a/docs/TROUBLESHOOTING.md +++ b/docs/TROUBLESHOOTING.md @@ -18,7 +18,7 @@ strata launch --status **I cannot find `START-HERE.bat` or `SETUP.bat`.** Open the installed Strata app, or run `strata launch`. Those wrappers are in the source repository and are not the startup path for the app. For a fresh desktop install, download the one installer for your system -(`Strata-0.2.9-windows-x64-setup.exe`, `Strata-0.2.9-linux-x86_64.AppImage` or `Strata-0.2.9-macos-arm64.dmg`), not a +(`Strata--windows-x64-setup.exe`, `Strata--linux-x86_64.AppImage` or `Strata--macos-arm64.dmg`), not a `strata-engine-…` archive. **`strata` is not found.** diff --git a/docs/UPSTREAM-README.md b/docs/UPSTREAM-README.md index f46c4e87416..30d2bfb54f1 100644 --- a/docs/UPSTREAM-README.md +++ b/docs/UPSTREAM-README.md @@ -1,7 +1,7 @@

- - Strata + + Strata

@@ -16,11 +16,11 @@

Download Strata - MIT license + MIT license Support Strata on Buy Me a Coffee

-

English · 简体中文 · 日本語 · Deutsch · Français · Español · Português · Polski

+

English · 简体中文 · 日本語 · Deutsch · Français · Español · Português · Polski

**Free, open source, and yours to run.** Use Strata on your desktop or with your favorite coding tools. No account needed. @@ -29,30 +29,30 @@ We recommend **ISTA-DASLab first**, then **Unsloth** and other providers, where a compatible model fits.
-  Qwen3.8-Flash-Next +  Qwen3.8-Flash-Next | Provider | Choices | | --- | --- | -| **ISTA-DASLab · recommended** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder: IQ1\_M](docs/MODELS.md#coder) for coding. | +| **ISTA-DASLab · recommended** | Q2\_0, IQ2\_XS, IQ3\_XXS, IQ3\_S. [Coder: IQ1\_M](MODELS.md#coder) for coding. | | **Unsloth** | UD-IQ4\_XS, UD-Q4\_K\_XL, UD-Q5\_K\_XL, UD-Q6\_K\_XL, Q8\_0 | | **Gyro · agentionai** | [Gyro-S, Gyro-M](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF). Lower memory use but slower. | -| **Other options** | [Swift 1.5](docs/MODELS.md#swift-15), [NVFP4](docs/NVFP4.md), [OrcaRouter](docs/ORCA.md) | +| **Other options** | [Swift 1.5](MODELS.md#swift-15), [NVFP4](NVFP4.md), [OrcaRouter](ORCA.md) | -[Compare sizes, memory needs and compatibility](docs/MODELS.md). +[Compare sizes, memory needs and compatibility](MODELS.md).
-  Qwen3.6-35B-A3B +  Qwen3.6-35B-A3B **Unsloth:** UD-Q4_K_XL, UD-Q6_K_XL, Q8_0. These are the currently documented packs for this model. -[Model guide and compatibility](docs/QWEN36.md). +[Model guide and compatibility](QWEN36.md).
-  DeepSeekV4.1-Flash-1017 +  DeepSeekV4.1-Flash-1017 In the **desktop app** and the **Terminal**. What does not fit in memory is read from your SSD, so it needs a lot of free disk space. @@ -65,7 +65,7 @@ disk space. | ISTA-DAS Coder (half of the experts removed) | 191 GB | about 197 GB | Runs on NVIDIA RTX 30, 40 and 50 series (Windows and Linux), AMD Radeon AI PRO R9700 (Linux), AMD Ryzen AI Max — Strix Halo -(Windows), and Apple Silicon Macs with 64 GB or more, text only ([Mac](docs/DEEPSEEK_MAC.md)). Setup downloads the files and the DeepSeek engine, then starts the model. +(Windows), and Apple Silicon Macs with 64 GB or more, text only ([Mac](DEEPSEEK_MAC.md)). Setup downloads the files and the DeepSeek engine, then starts the model. ```sh strata launch --family deepseek41 --model ist-das-2.00bpw --plan @@ -81,13 +81,13 @@ Measured with the 2.00 bpw set on Strata 0.2.5 (Strix Halo: 0.2.7). Output is th | 4x Radeon AI PRO R9700 | 74 to 77 tokens/s | 400 to 450 tokens/s (8K) | | Ryzen AI Max+ 395 (Strix Halo), 128 GB unified | 19 to 21 tokens/s (code), 11 (chat) | 87 tokens/s (4K), 125 tokens/s (8K) | -[Memory, disk and details](docs/DEEPSEEK41.md) · [Release notes](https://github.com/Niko1221/Strata/releases) +[Memory, disk and details](DEEPSEEK41.md) · [Release notes](https://github.com/Niko1221/Strata/releases)
-Not sure what fits? [Choose a model for your computer](docs/MODELS.md#pick-by-memory). +Not sure what fits? [Choose a model for your computer](MODELS.md#pick-by-memory). ## Install @@ -98,12 +98,12 @@ Not sure what fits? [Choose a model for your computer](docs/MODELS.md#pick-by-me **[Download Strata for Windows, Linux or Mac →](https://github.com/Niko1221/Strata/releases/latest)** Install the app, choose a model, and start chatting. There is one installer for each system. On first start the app -downloads the engine for your graphics card. [Help with the installer](docs/INSTALL.md#the-app). +downloads the engine for your graphics card. [Help with the installer](INSTALL.md#the-app). Files: `Strata-0.2.9-windows-x64-setup.exe` · `Strata-0.2.9-linux-x86_64.AppImage` · `Strata-0.2.9-macos-arm64.dmg`

- Strata desktop app with a chat and live code preview side by side + Strata desktop app with a chat and live code preview side by side
Chat, files and a live preview in one window.

@@ -134,18 +134,18 @@ On Windows, open a new terminal afterwards. Then: strata launch ``` -Follow the prompts to download and start a model. [Terminal guide](docs/TERMINAL_LAUNCH.md). +Follow the prompts to download and start a model. [Terminal guide](TERMINAL_LAUNCH.md).

- Strata terminal app showing model activity, speed, memory and recent requests + Strata terminal app showing model activity, speed, memory and recent requests
Watch your model and hardware while it runs.

-[Build from source](docs/SOURCE_INSTALL.md) · [Docker](docs/DOCKER.md) · [Full install guide](docs/INSTALL.md) +[Build from source](SOURCE_INSTALL.md) · [Docker](DOCKER.md) · [Full install guide](INSTALL.md) @@ -153,13 +153,13 @@ Follow the prompts to download and start a model. [Terminal guide](docs/TERMINAL | Supported hardware | Guide | | --- | --- | -| **NVIDIA GeForce RTX 20 / 30 / 40 / 50 series** | [NVIDIA setup](docs/INSTALL.md#what-you-need) | -| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [AMD setup](docs/AMD_HIP.md) | -| **AMD Radeon RX 6800 / 6900 · experimental** | [RX 6000 setup](docs/AMD_HIP.md#rdna2-gfx1030) | -| **Intel Arc · Linux** | [Intel setup](docs/INTEL_ARC.md) | -| **Apple Silicon · M1 or newer** | [Mac setup](docs/MAC.md) | -| **AMD Ryzen AI Max · Strix Halo** | [Strix Halo setup](docs/STRIX_HALO.md) | -| **NVIDIA DGX Spark · Linux** | [DGX Spark setup](docs/DGX_SPARK_LINUX.md) | +| **NVIDIA GeForce RTX 20 / 30 / 40 / 50 series** | [NVIDIA setup](INSTALL.md#what-you-need) | +| **AMD Radeon RX 7700 / 7800 / 7900, RX 9060 / 9070, AI PRO R9700** | [AMD setup](AMD_HIP.md) | +| **AMD Radeon RX 6800 / 6900 · experimental** | [RX 6000 setup](AMD_HIP.md#rdna2-gfx1030) | +| **Intel Arc · Linux** | [Intel setup](INTEL_ARC.md) | +| **Apple Silicon · M1 or newer** | [Mac setup](MAC.md) | +| **AMD Ryzen AI Max · Strix Halo** | [Strix Halo setup](STRIX_HALO.md) | +| **NVIDIA DGX Spark · Linux** | [DGX Spark setup](DGX_SPARK_LINUX.md) | @@ -167,7 +167,7 @@ Follow the prompts to download and start a model. [Terminal guide](docs/TERMINAL **Claude Code · Codex CLI · OpenCode · oh-my-pi · pi.dev · Hermes Agent · OpenClaw**, and more through the OpenAI- and Anthropic-compatible APIs. -Run **`strata connect`** to see setup instructions for your tool. [Connection guide](docs/TERMINAL_LAUNCH.md#connect-your-apps-strata-connect). +Run **`strata connect`** to see setup instructions for your tool. [Connection guide](TERMINAL_LAUNCH.md#connect-your-apps-strata-connect). @@ -176,8 +176,8 @@ Run **`strata connect`** to see setup instructions for your tool. [Connection gu ## Help -[All guides](docs) · [Troubleshooting](docs/TROUBLESHOOTING.md) · [Report a bug](https://github.com/Niko1221/Strata/issues/new/choose) · -[Benchmarks](docs/COMMUNITY_BENCHMARKS.md) · [How it works](docs/HOW_IT_WORKS.md) · [Contribute](CONTRIBUTING.md) +[All guides](README.md) · [Troubleshooting](TROUBLESHOOTING.md) · [Report a bug](https://github.com/Niko1221/Strata/issues/new/choose) · +[Benchmarks](COMMUNITY_BENCHMARKS.md) · [How it works](HOW_IT_WORKS.md) · [Contribute](../CONTRIBUTING.md) ## Support Strata @@ -185,11 +185,11 @@ Run **`strata connect`** to see setup instructions for your tool. [Connection gu Buy Me a Coffee: support Strata -[MIT license](LICENSE) · [Credits and model licenses](docs/HOW_IT_WORKS.md#credits) · [Privacy](docs/SAFETY.md) +[MIT license](../LICENSE) · [Credits and model licenses](HOW_IT_WORKS.md#credits) · [Privacy](SAFETY.md) -For AI tools and automated readers: technical reference · setup instructions. +For AI tools and automated readers: technical reference · setup instructions. **Sponsorships, hardware partnerships or collaborations:** [niko.veit@outlook.com](mailto:niko.veit@outlook.com).