diff --git a/CHANGELOG.md b/CHANGELOG.md index 0993ba04..e1eedecc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,13 +11,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.63.0] — 2026-10-21 + +Against engine **0.62.0** (the JVM ↔ Kotlin/Wasm parity release: every kernel on the wasm tier, resident +activations, fused attention / norms / softmax through FFM on the JVM and in the SIMD128 sidecar, browser IO +parity, `skainet-data-source` on JS and wasm). Headline: **expert heads and routing for Laya** — a decision +head fine-tuned on top of the frozen encoder loads on its own, any number of heads share one encoder pass +per question, a prompted router picks one per request, and a mixture stands behind the typed decision +contract. Verified against the published layaMOE evaluation. Plus the whisper StableHLO export as a shipped +CLI. + +Version note: every API change is additive. The Laya Q8_0 lane on the desktop JVM with +`skainet-backend-native-cpu` on the classpath needs engine 0.62.1 (SKaiNET #1436: the native Q8_0 / Q4_0 / +Q5_0 / Q5_1 bindings zeroed every row but the last of a multi-row product); the FP32 lane and the browser's +Q8_0 lane are unaffected. + ### Added +- **iree-android: a Whisper session over the `WhisperExportCli` graphs** — `IreeWhisperSession` with the prebuilt `libskainet_whisper_session.so`, the README section and `IreeWhisperContractTest` (#512). +- **Whisper export ships as `WhisperExportCli`** in the whisper module's `jvmMain` (was `jvmTest`), like `MoonshineV2ExportCli`: `WhisperExportHarness` is configurable (`WhisperConfig` + `WhisperExportOptions`: audio context, KV length, prompt length, manifest language, parameter scope), `configFromHf` reads geometry and vocabulary from a Hugging Face `config.json` (tiny, base, small and `*.en`), and the Gradle task `exportWhisper` is env-driven (`WHISPER_MODEL_DIR`, `WHISPER_OUT_DIR`, `WHISPER_AUDIO_CTX`, `WHISPER_MAX_POSITIONS`, `WHISPER_LANGUAGE`), so a blueprint can produce the encoder, masked prefill and KV-step graphs with one merged `params.irpa` from the released artifact (#510). + - **Expert heads and routing for Laya** (`sk.ainet.models.laya`): a Laya decision head can be loaded on its own and run on the base model's encoder output. `LayaExpertLoader.load(metaJson, weights, base, ctx)` binds an `expert.safetensors` (the `type_emb.*` / `head.*` / `scorer.*` tensors of a head copied from the base checkpoint and fine-tuned for a domain) with its `expert.json` (name, domains, calibration temperatures) into a `LayaExpert`; `LayaWeightPreparation.prepareHead` and `LayaDecisionModel.buildHead` are the steps underneath, and a head prepared on its own reproduces the base model bit for bit. `LayaDecisionModel.logitsFrom` / `logitsBatchFrom` and `LayaAgent.encodeFor` / `logitsFrom` run any `LayaDecisionHead` on an encoder output computed once, so several heads answer the same sequences for one encoder pass each. `LayaRouterSpec` is the prompted router (the base head answers a choice over kinds of text with descriptions; kind → expert; threshold; `general` fallback), `LayaMixtureAgent.decide(state, questions, expert, compare)` routes and answers with the chosen head (forced head reported as overridden; `compare` adds the base head's answers on the same encoder output; encoder and head timings), and `LayaMixtureDecisionAdapter` puts a mixture behind the typed decision contract with `onRouted` reporting the head that answered. Verified against the published evaluation of layaMOE (gated `LayaMoeEvalTest`, fixtures under `laya-moe/`, Apache-2.0). - `LayaAgent.decide` / `decideAll` / `logitsFor` overloads that take a head and a decoder; `LayaAgent.decodeAll`; `LayaDecisionModel.encodeBatch(sequences, padId)` returning a `LayaEncodedBatch`; `LayaDecisionModel.defaultHead`. The existing signatures are unchanged. ### Changed +- Engine pin **SKaiNET 0.62.0** (`skainet-bom:0.62.0` re-exported by the BOM); kotest 6.2.6. The llama `ForwardScopeSteadyStateTest` pins the scoped decode to within four ulp of the unscoped one instead of raw-bit identity: since 0.62.0 the JVM ops route slab-backed and heap activations through kernels whose accumulation order can differ by one ulp. - `LayaEncoderConfig.requireSupported()` accepts `position_embedding_type: "absolute"` beside `"sans_pos"`: ModernBERT positions are rotary whatever the field says, and transformers 5 exports (`convaiinnovations/laya-typed-decisions`) write `absolute`. The English checkpoint loads and its `sequences.json` section reproduces (gated on `LAYA_TYPED_DECISIONS_PATH`). ## [0.62.2] — 2026-10-18 diff --git a/README.md b/README.md index a6e1e282..89f15101 100644 --- a/README.md +++ b/README.md @@ -113,7 +113,9 @@ Honest status — see the project-status note at the top of this README. ## Current release -The current release is **0.62.2**, against **SKaiNET 0.61.0**: a transformers-only fix, the Android runtime's streaming Moonshine libraries are rebuilt with the utterance-start fix (#500; 0.62.0 and 0.62.1 packed the old build, which garbles transcripts on device). It builds on **0.62.1**, a transformers-only fix on 0.62.0, whose JVM `LayaModelLoader.load(directory, …)` had become an extension function that existing callers did not import; it is a member again (`@JvmOverloads`, so Java callers keep the 0.61 signature). +The current release is **0.63.0**, against **SKaiNET 0.62.0**: expert heads and routing for Laya — a decision head fine-tuned on top of the frozen encoder (`expert.safetensors` + `expert.json`) loads on its own with `LayaExpertLoader`, any number of heads share one encoder pass per question (`LayaDecisionModel.logitsFrom`), `LayaRouterSpec` + `LayaMixtureAgent` route a request by asking the base head what kind of text it is, and `LayaMixtureDecisionAdapter` puts the mixture behind the typed decision contract; verified against the published layaMOE evaluation. The ModernBERT-large `laya-typed-decisions` checkpoint loads (`position_embedding_type: absolute`). The whisper StableHLO export ships as `WhisperExportCli`. Engine 0.62.0 brings the JVM ↔ Kotlin/Wasm parity: every kernel on the wasm tier, resident activations, fused attention / norms / softmax, browser IO parity. The Laya Q8_0 lane on the desktop JVM with the native provider needs engine 0.62.1 (SKaiNET #1436). + +It builds on **0.62.2**, against **SKaiNET 0.61.0**: a transformers-only fix, the Android runtime's streaming Moonshine libraries are rebuilt with the utterance-start fix (#500; 0.62.0 and 0.62.1 packed the old build, which garbles transcripts on device). It builds on **0.62.1**, a transformers-only fix on 0.62.0, whose JVM `LayaModelLoader.load(directory, …)` had become an extension function that existing callers did not import; it is a member again (`@JvmOverloads`, so Java callers keep the 0.61 signature). It builds on **0.62.0**, against **SKaiNET 0.61.0**: @@ -301,7 +303,7 @@ The recommended way to consume is via the BOM. It pins every published `skainet- ```kotlin dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.62.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.0")) // Versions resolved from the BOM: implementation("sk.ainet.transformers:skainet-transformers-core") diff --git a/docs/modules/ROOT/pages/how-to/export-moonshine-v2-streaming.adoc b/docs/modules/ROOT/pages/how-to/export-moonshine-v2-streaming.adoc index 649baf05..0905a91c 100644 --- a/docs/modules/ROOT/pages/how-to/export-moonshine-v2-streaming.adoc +++ b/docs/modules/ROOT/pages/how-to/export-moonshine-v2-streaming.adoc @@ -32,7 +32,7 @@ From your own build, resolve the published module and run its main class: ---- val exportTool by configurations.creating dependencies { - exportTool(platform("sk.ainet.transformers:skainet-transformers-bom:0.62.2")) + exportTool(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.0")) exportTool("sk.ainet.transformers:skainet-transformers-inference-moonshine") } tasks.register("exportMoonshine") { diff --git a/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc b/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc index 4e5f2373..f773cce3 100644 --- a/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc +++ b/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc @@ -24,7 +24,7 @@ a DSL decoder is future work. [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.62.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.0")) implementation("sk.ainet.transformers:skainet-transformers-inference-moonshine") } ---- diff --git a/docs/modules/ROOT/pages/reference/typed-decisions.adoc b/docs/modules/ROOT/pages/reference/typed-decisions.adoc index 85c3d29e..63b6dddb 100644 --- a/docs/modules/ROOT/pages/reference/typed-decisions.adoc +++ b/docs/modules/ROOT/pages/reference/typed-decisions.adoc @@ -12,7 +12,7 @@ below put that on the JVM path with the same contract for every model. [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.62.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.0")) implementation("sk.ainet.transformers:skainet-transformers-decisions-api") // the contract, no engine dependency implementation("sk.ainet.transformers:skainet-transformers-decisions-runtime") // validation + acceptance policy implementation("sk.ainet.transformers:skainet-transformers-inference-laya") // Laya as a DecisionAdapter diff --git a/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc b/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc index 298aa380..31331792 100644 --- a/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc +++ b/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc @@ -55,7 +55,7 @@ dependencies { // self-registers it on ART at process start — nothing to call. runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu") - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.62.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.0")) implementation("sk.ainet.transformers:skainet-transformers-core") implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-inference-llama") diff --git a/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc b/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc index ed5f9b52..17bd41e0 100644 --- a/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc +++ b/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc @@ -25,7 +25,7 @@ In your `build.gradle.kts`: [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.62.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.0")) implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-agent") @@ -41,7 +41,7 @@ Or in Maven (Maven needs the `-jvm` classifier suffix on platform artifacts): sk.ainet.transformers skainet-transformers-bom - 0.62.2 + 0.63.0 pom import diff --git a/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc b/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc index fc8505f3..f244bdda 100644 --- a/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc +++ b/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc @@ -34,7 +34,7 @@ encoder output to the advertised dimensionality. The runtime applies it automati [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.62.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.0")) implementation("sk.ainet.transformers:skainet-transformers-providers") } ---- diff --git a/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc b/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc index dae36972..2beaa414 100644 --- a/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc +++ b/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc @@ -52,7 +52,7 @@ The pieces you need live in three modules: [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.62.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.0")) implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-agent") diff --git a/gradle.properties b/gradle.properties index d50475d9..b0dbdad1 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.transformers -VERSION_NAME=0.62.2 +VERSION_NAME=0.63.0 POM_DESCRIPTION=SKaiNET-transformers diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index a4633ee0..b321844c 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -1,5 +1,5 @@ [versions] -skainet = "0.61.0" +skainet = "0.62.0" agp = "9.4.1" jacksonDatabind = "2.22.3" jsonSchemaValidator = "3.0.8" diff --git a/llm-inference/llama/src/jvmTest/kotlin/sk/ainet/models/llama/ForwardScopeSteadyStateTest.kt b/llm-inference/llama/src/jvmTest/kotlin/sk/ainet/models/llama/ForwardScopeSteadyStateTest.kt index 2ccf9e54..1e133ae0 100644 --- a/llm-inference/llama/src/jvmTest/kotlin/sk/ainet/models/llama/ForwardScopeSteadyStateTest.kt +++ b/llm-inference/llama/src/jvmTest/kotlin/sk/ainet/models/llama/ForwardScopeSteadyStateTest.kt @@ -19,9 +19,11 @@ import sk.ainet.lang.types.FP32 * #343 steady-state pins for the per-step [sk.ainet.lang.memory.ForwardScope] * in [OptimizedLLMRuntime]'s DIRECT decode path: * - * 1. **Bit-identity** — a scoped decode produces exactly the logits of an + * 1. **Identity to within a few ulp** — a scoped decode produces the logits of an * unscoped one (`forwardSlabFloats = 0`), step for step. The slab changes - * where activations live, never what they hold. + * where activations live, never what they hold; since engine 0.62.0 the JVM + * kernels chosen for slab-backed and heap activations may accumulate in a + * different order, so the pin allows four ulp instead of raw-bit equality. * 2. **Flat peak** — `peakFloats` stops growing once decode is warm. The KV * history lives outside the scope (the cache detaches it to ambient * storage), so per-step slab use grows only with the KV *views* concat @@ -105,11 +107,16 @@ class ForwardScopeSteadyStateTest { val b = unscoped.forward(tokenId).data.copyToFloatArray() assertEquals(b.size, a.size) for (i in a.indices) { - // Bit-identical, not approximately equal: same kernels, same - // values, different storage. - assertEquals( - b[i].toRawBits(), a[i].toRawBits(), - "step=$step logit[$i]: scoped=${a[i]} unscoped=${b[i]}", + // Equal to within a few ulp. Up to engine 0.61 this was raw-bit identity (same + // kernels, same values, different storage); since engine 0.62.0 the JVM ops route + // slab-backed and heap activations through kernels whose accumulation order can + // differ (the fused native attention and norm kernels take heap operands, a + // segment-backed activation is bridged first), which moves a logit by one ulp. + // The slab still changes where activations live, never what they mean. + val tolerance = 4 * Math.ulp(b[i]) + assertTrue( + a[i] == b[i] || kotlin.math.abs(a[i] - b[i]) <= tolerance, + "step=$step logit[$i]: scoped=${a[i]} unscoped=${b[i]} (tolerance $tolerance)", ) }