Skip to content

Kev: add pinned CPU export and FP32 pointer-head validation - #170

Draft
psv666 wants to merge 1 commit into
InferMatrixLab:mainfrom
psv666:codex/kev-export-head
Draft

psv666 wants to merge 1 commit into
InferMatrixLab:mainfrom
psv666:codex/kev-export-head

Conversation

@psv666

@psv666 psv666 commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Purpose

Kev-4B has no preparation or native readout in this repository. Add the first independent slice of #27: a hash-pinned CPU exporter that merges every language LoRA projection, plus a Rust FP32 bilinear pointer head checked against the released head and the official reference class.

The export preserves the base tokenizer's delimiter IDs and the trained head/calibration. This PR does not add a worker or claim full-model parity; #27 remains open for prompt/row processing, shared Qwen execution and serving. #129 already carries selected-position readouts, so this slice does not introduce a competing shared API.

Summary

pinned base + Kev adapter/head
  -> verify every input hash
  -> merge 496 LoRA tensors into 248 BF16 language projections
  -> preserve FP32 pointer head, base tokenizer and calibration manifest

decision/option hidden states -> learned FP32 pointer logits
                            -> saved temperature + whole-question softmax

The exporter includes all five GDN projection types. Tests and small reference outputs live under root tests/kev/; a CPU-only CI job runs exporter checks, and the Rust tests join the workspace suite. Recipes and model status explicitly identify the preparation-only scope.

Required human contributor answers

The contributor supplied these answers in the task on 2026-10-25 (translated):

  1. Outcome: Add native model support for System1-Omni users that agrees with official reference results and has reproducible acceptance.
  2. Highest-leverage variable: Reference fidelity and reproducible native coverage.
  3. How the work moves it: Reuse existing backends and independent small PRs to lower integration and review cost. The requested first Kev slice is 4B, merge plus pointer head.
  4. Customer and actual need: System1-Omni users need native model support with reference-consistent results.

Is a PR needed? The contributor explicitly requested implementation PRs where appropriate. This scoped repository change provides the preparation/readout prerequisite for #27; no tracking-only or empty worker PR is proposed.

Bug-fix evidence: N/A; new model preparation, not a bug fix.

Test plan and results

Base cd302b6e3d7723ebfc33bba9698da2c1e881f203; head 4be35df43514b57636ab2926c63a52782f62b8c1. Local Linux x86_64, Python 3.12.13, Torch 2.8.0+cpu, safetensors 0.8.0, Rust 1.99.1. CPU checks only; no GPU inference or performance campaign.

Passed:

  • cargo fmt --all --check
  • cargo clippy --workspace --locked --all-targets --features omni-laya/serve -- -D warnings
  • cargo test --workspace --locked --features omni-laya/serve: 350 passed, 66 ignored, zero failures. The first attempt hit an existing CLM test's fixed temporary-directory permissions; rerunning with a fresh task-specific TMPDIR passed without changing CLM.
  • cargo build --workspace --release --locked --features omni-laya/serve
  • cargo test --locked -p omni-kev-native -- --list: both ordinary tests and the opt-in released-head test discovered.
  • python -m unittest discover -s tests/kev -p 'test_*.py' -v: 6 passed; no weights or GPU required.
  • ruff check recipe/kev/export.py tests/kev --select E9,F,I, mkdocs build --strict, and git diff --check.
  • Full CPU export of jaredpalmer/kev-4b@139fdd94f1b6a6ad80cc15e08fcb99cac885a101 with Qwen/Qwen3.5-4B-Base@1001bb4d826a52d1f399e183466143f4da7b741b: 248 merged targets, including 24 of each of the five GDN projections. Independently rechecked all seven output hashes, all four head tensors exactly, and all five delimiter IDs through the exported tokenizer.
  • KEV_HEAD=<export>/head.safetensors cargo test --release --locked -p omni-kev-native --test head -- --ignored --nocapture: 1 passed against the pinned upstream class for 1/2/7/31/255 options. Maximum absolute logit error 3.12924385e-7, probability error 2.98023224e-8. Predeclared gates: logits 1e-4 + 1e-5*abs(reference); probabilities 1e-5 absolute.

Current-head GitHub CI also passed: Rust, benchmark tooling, the new Kev exporter tests, CUDA contract checks and CUDA compilation (CI run), plus the strict Docs build. Deployment was skipped as configured for pull requests. CUDA compilation is not GPU inference validation.

Demo / evidence

Before: no Kev export, head or registered tests.
After: real pinned artifacts export successfully and the Rust head reproduces the official class on deterministic hidden inputs. tests/kev/data/head.json is the maintained expected-output fixture with source/head hashes and generation provenance; tests/kev/generate_head_fixture.py executes the hash-verified official PointerHead class. The recipe contains all reproduction commands. Model weights and local execution logs are not committed.

Video and performance charts: N/A. Full-backbone BF16-merge decision parity, prompt/token fixtures, GPU execution, readiness and worker/frontend inference remain unverified/not implemented, not passing. No speed or model-quality claim is made.

Merge Danger

Door: two-way. Additive, model-local preparation and head code; no shared runtime/backend changes.

Blast Radius: Kev. The CPU export CI job adds a Torch CPU installation; existing workers are unchanged. The manifest is a preparation format, not yet a serving-loader contract.

Self-review

Full diff, ownership, hash gates, head arithmetic, fixture provenance/consumers and documentation reviewed locally. 711 changed authored Python/Rust lines; 1765 total changed lines including the small expected-output fixture, license, docs and wiring. Human review remains pending; this PR stays draft and does not close #27.

  • I have reviewed the full diff and addressed the issues I found.
  • I have checked that the change follows the project's architecture and stays focused on the stated purpose.
  • I have run the checks appropriate to this change and reported commands, results, and anything I could not verify above.
  • I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.

Signed-off-by: psv666 <2693925048@qq.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[New Model]: Support Kev (0.5B–9B) — second Qwen3.5 decision model, reusing the qwen3_5 CUDA backend

1 participant