[hsliuustc0106]

Required human contributor answers

Answer every question before implementation and PR submission, including drafts.
Reuse explicit, still-applicable human answers with a traceable source reference;
ask the human for anything missing and wait. An agent must not invent answers.
Blank fields, placeholders, or an agent-written rationale do not satisfy this gate.
Share only information and source links appropriate for this PR's audience.

1. What outcome are you trying to achieve?

   Answer: Move the default Clef-Flash serve path off Python ThreadingHTTPServer and onto the shared Rust runtime, while keeping /v1/systemone and /health behavior correct. Source: the task that opened this branch (draft PR only; do not mark ready).

2. What is the highest-leverage variable for that outcome?

   Answer: Where the first CUDA touch happens. On the L20X-labeled host at main a41e379, HTTP serve was about 162–175 ms and the same executor on a main or persistent thread was about 48–65 ms. The difference is the first CUDA touch on a fresh HTTP thread, about 110 ms. Public231 C1 on that host was about 167/214 ms versus vLLM about 32/83 ms. Source: the same task. Not re-measured in this PR.

3. How will this work move that variable rather than create busywork?

   Answer: Admit requests on omni-runtime's SerialScheduler and score them on one persistent executor thread, reusing the existing Clef executor. That removes the ephemeral-thread CUDA tax. A full native joint-head port is a later slice; it is not required to move this variable. Source: the same task (Hongsheng chose the Rust migration, not a Python-only dedicated-thread patch).

4. Who is the customer or user, and what do they actually need?

   Answer: Operators serving Clef-Flash. They need the current request and response envelope, and a default path that does not pay the ephemeral-thread CUDA tax. Lab L20X smoke and Public231 C1 of this server are a separate follow-up. Source: the same task.

After answering the four questions: is a PR needed at all to achieve this outcome?

Human decision and rationale (or explicit source reference): Yes. The task asks for a draft PR on InferMatrixLab/system1-omni and says not to mark it ready, merge, or post review comments.

For a bug fix: link actual observed failure evidence.
Include the observed result of a minimal reproduction, failing regression test,
or relevant trace. Label untested diagnoses as unverified. For a non-bugfix,
state why this is not applicable.

Evidence: Operator report in the task, L20X-labeled host, main a41e379. Public231 C1: omni Clef about 167/214 ms versus vLLM about 32/83 ms. HTTP about 162–175 ms versus main-thread or persistent-thread serve about 48–65 ms. This environment did not re-run that measurement. The regression tests added here prove the new default path does not serve CUDA work on a fresh ThreadingHTTPServer thread; they do not measure latency.

Purpose

frontend.clef used ThreadingHTTPServer, so each request ran on a new thread. PyTorch CUDA state is thread-local, and the first touch on that thread cost about 110 ms. StartLux already avoids that by serving through Rust. This PR makes the same shape the default for Clef.

Design: hybrid first slice, not a native joint head. The joint schema head is the checkpoint's joint_schema_model.py plus joint_head.safetensors. The shared Qwen CUDA executor scores a different head and is not used. Reimplementing that head is a separate milestone. This slice is the largest change that removes the thread tax without leaving a second HTTP server as the default:

• omni-clef is the HTTP server. It speaks GET /health and POST /v1/systemone.
• One request is admitted at a time through omni_runtime::SerialScheduler (same FIFO queue as StartLux and Laya). A second request waits. The previous handler returned HTTP 503 while the lock was held.
• The admitted closure only writes a length-prefixed frame and waits. It does not call CUDA.
• One Python process, python -m models.clef.worker, loads the pinned snapshot, warms up, and scores every request on its main thread. That is the thread that initialized CUDA.
• python -m frontend.clef execs omni-clef. It does not bind a port and it does not fall back to a Python HTTP server.
• StartLux is unchanged.

The port opens only after the executor's startup forward returns. Response JSON from the executor is forwarded unchanged, including answer order. Transport errors keep the previous {"detail": ...} text for the cases the old handler tested (400, 404, 408, 411, 413, 415, 422, 500). A negative Content-Length is rejected by hyper as HTTP 400 with an empty body before that handler runs; a declared length above 8 MiB is still 413 with the same detail string.

Residual versus vLLM, unverified on this commit. After the thread tax, the operator's persistent-thread serve was about 48–65 ms against a vLLM p50 of about 32 ms, roughly 1.55×. That remainder is prefill and the joint head, not this PR. The recorded Public231 primary is still tip 594528a9 (p50 164.969 ms, p95 216.755 ms) from the old Python HTTP server. Do not cite those numbers as a result for omni-clef.

Still not done

• Native joint-head CUDA on the shared Qwen executor, with numerical parity.
• CUDA graphs, torch.compile, and cross-request batching.
• A bounded admission queue. The scheduler waits; it does not shed load.
• Lab L20X smoke and a Public231 C1 paste for this binary. That paste is the next primary, compared with 594528a9.

Test Plan

System1-Omni Version / Commit: a7d3f2b2661099e8fa162aab43aa9a541edf6669 on cursor/clef-rust-runtime-734c, base main 05075d0f36e399e3a001ffedea400c666990cda9.

From the repository root, after installing pytest:

PYTHONPATH=src python -m pytest tests/clef -q
PYTHONPATH=src python -m compileall -q src/models/clef src/frontend/clef.py tests/clef
cargo fmt --all --check
cargo clippy --workspace --locked --all-targets --features omni-laya/serve -- -D warnings
cargo test --workspace --locked --features omni-laya/serve

No checkpoint and no GPU. The Rust serve tests spawn tests/clef/fake_worker.py, which implements the same frame protocol and reports its pid and thread.

Test Result

Run in this environment (no GPU):

• PYTHONPATH=src python -m pytest tests/clef -q — 36 passed.
• compileall on the Clef Python tree — passed.
• cargo test --locked -p omni-clef — 11 passed, including concurrent posts sharing one executor pid and its main thread, serialization across a delay, a stalled body not holding admission, warmup-before-return, and a dead executor not hanging the next request.
• cargo clippy --workspace --locked --all-targets --features omni-laya/serve -- -D warnings — passed.
• cargo test --workspace --locked --features omni-laya/serve — passed.
• cargo fmt --all --check — passed.

Not run: mkdocs build --strict. Lab L20X load, smoke, answer parity, and Public231 C1. CPU tests do not establish that the joint-head forward got faster.

Demo / evidence

N/A for a screen recording: this change is a serve-path and admission change, and the GPU campaign was out of scope.

Unverified: Public231 C1 and a single-request smoke on the L20X-labeled host for commit a7d3f2b. The 110 ms thread diagnosis and the ~1.55× residual versus vLLM are the operator report cited above, not a result of this commit. Historical p50/p95 figures in the recipe remain the previous ThreadingHTTPServer worker.

How to run, from the repository root, after the recipe's Python install and snapshot download:

cargo build --release --locked -p omni-clef
./target/release/omni-clef \
  --model <snapshot> \
  --variant clef-flash \
  --device cuda \
  --gdn-prefill fla \
  --host 127.0.0.1 \
  --port 8000

GET /health is absent until the startup forward returns.

Self-review

Before marking this PR ready for review or requesting maintainer review, complete
the self-review checklist.
Keep the PR in draft while this work is incomplete.
For agent assistance, use the optional precheck-pr skill.

• I have reviewed the full diff and addressed the issues I found.
• I have checked that the change follows the project's architecture and stays focused on the stated purpose.
• I have run the checks appropriate to this change and reported commands, results, and anything I could not verify above.
• I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.