Skip to content

feat: LayaMoE — Laya expert heads with a prompted router on desktop, Android and the web - #46

Merged
michalharakal merged 5 commits into
developfrom
feature/layamoe
Oct 21, 2026
Merged

michalharakal merged 5 commits into
developfrom
feature/layamoe

Conversation

@michalharakal

Copy link
Copy Markdown
Contributor

Summary

LayaMoE — a new official example: domain experts on top of the Laya decision model, a port of layaMOE (Apache-2.0, vishalmysore; an unofficial experiment on ConvAI Innovations' laya-typed-decisions) onto SKaiNET-transformers 0.63.0's expert-head API. One frozen ModernBERT-large encoder, the safety and customer_ops heads (26.5M parameters each, fine-tuned on synthetic domain data), a prompted router (the base head answers "What kind of text is this?" over ten kinds, five map to an expert, threshold 0.3, the original head as the fallback), the seven presets of the original page, and the published 108-case evaluation. Sequence-level routing, one head per request — not a sparse mixture of experts, and the page says so.

Full KMP. Checkpoint spec and download (per-file hf:// through skainet-data-source), the mixture (LayaExpertLoader, LayaMixtureAgent), the presets, the question JSON (the Python SDK shape), the worker protocol, the view model and the Compose UI are common code. Platform seams: where the model bytes come from (desktop cache dir, Android filesDir, the browser's Cache API + JsBlobRandomAccessSource), which kernels to install (native FFM, JNI NEON, the SIMD128 sidecar), and the Web Worker bootstrap. Desktop runs FP32 (reference) or Q8_0; Android and the browser run Q8_0 at load. The CLI has ask, preset, bench and eval.

Honest numbers (12-core laptop, README Measured): cli eval over all 108 cases reproduces layaMOE's published table to the answer — general 59.6 %, routed 67.3 %, oracle 68.9 %; with an expert 55.2 → 67.7 %, without 66.7 % either way. Desktop FP32 with the native kernels: load 13 s, the "drop a production table" preset 13.0 s (three questions plus the router); Q8_0 on 0.62.1: 25.3 s (the eager JVM packed kernels are a memory measure). Browser, headless Chromium, SIMD128 with threads in a Web Worker: 1.05 GB first download in 95 s, Q8_0 build 104 s, the same preset 31.9 s, answers within quantization noise of the desktop (destructive 67.2 % vs 65.1 %). A worker start-up race (the page's LOAD arriving before the Kotlin/Wasm handler exists) is closed by a HELLO handshake.

Found on the way, fixed upstream: engine 0.62.0's native FFM bindings of the Q8_0 / Q4_0 / Q5_0 / Q5_1 kernels zeroed every row but the last of a multi-row product (SKaiNET #1436, released in 0.62.1, the pin here); and the ModernBERT-large checkpoint needed position_embedding_type: absolute accepted (transformers #511).

Pins: SKaiNET 0.62.1, SKaiNET-transformers 0.63.0, Kotlin 2.4.21 (the sidecar needs at least the engine's), Compose Multiplatform 1.11.1 (the skainet-ui match). webapp.json (id layamoe) for examples.skainet.sk, .github/workflows/layamoe-ci.yml, THIRD_PARTY_LICENSES/NOTICE (Laya / ConvAI, ModernBERT-large / Answer.AI + LightOn, layaMOE heads, router, presets and evaluation cases / vishalmysore, coi-serviceworker / MIT), root README entry with the web screenshot.

Verification

  • Against Maven Central (no local lane): :shared:jvmTest (13: question JSON, presets, router spec, DTOs, worker protocol, view model with a fake client), :shared:testAndroidHostTest, :cli:test, :cli:installDist, :composeApp:compileKotlinJvm, :androidApp:assembleDebug (arm64-v8a APK), :shared:compileKotlinWasmJs, :webWorker:compileProductionExecutableKotlinWasmJs, :composeApp:compileKotlinWasmJs — the CI set.
  • Desktop: cli preset --index 0 --compare routes to safety and gives destructive 0.651 / needs_human 0.233 / risk 2.11, the figures the layaMOE README quotes for that case; cli eval over the 108 cases (table in the README); cli preset --q8 on 0.62.1 answers like FP32.
  • Web: wasmJsBrowserDistribution served on localhost under the coi-serviceworker, driven by a headless-Chromium script: cache, Q8_0 build, a routed request, the screenshot in docs/screenshots/web.png.
  • Not run on a device: Android (APK builds; the memory note on low-RAM devices is in the app).

…uter on desktop, Android and the web

Port of layaMOE (Apache-2.0) on the SKaiNET-transformers expert-head API:
one laya-typed-decisions encoder, the safety and customer_ops heads, the
prompted router, the seven presets and the 108-case evaluation. Common
code for the checkpoint spec, download, mixture, presets, question JSON,
worker protocol, view model and Compose UI; platform actuals only for the
model bytes (cache dir, filesDir, Cache API + Blob), the kernels and the
Web Worker bootstrap. CLI with ask / preset / bench / eval. Pinned to
transformers 0.63.0-SNAPSHOT from Maven local until the release.
The page sends LOAD only after the worker says HELLO (its handler is
installed after the module instantiates; an earlier LOAD was lost and the
page waited forever). Header layout (the shared Powered-by line gets its own
row), ASCII arrows for Skiko's browser fonts, the README's measured section
(full 108-case evaluation reproducing the published table, desktop FP32 and
Q8_0 costs, the browser run) and the note on the engine 0.62.0 Q8_0 bug fixed
by SKaiNET PR #1436.
@michalharakal
michalharakal merged commit 43d619c into develop Oct 21, 2026
1 check passed
@michalharakal
michalharakal deleted the feature/layamoe branch October 21, 2026 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant