From 2812144ea51562e0660e539daf9c3d6656d9e70a Mon Sep 17 00:00:00 2001 From: "Mr.Dong" <1042542469@qq.com> Date: Sun, 18 Oct 2026 01:46:02 +0800 Subject: [PATCH 1/7] vela: add the 2.0 0.3B reference worker A Python reference worker on /v1/systemone that runs the pinned vllm-sr/Vela-2.0-0.3B@d6f03aa9 package's own runtime for text state and all five question kinds (choice/noul/score plus the span/set extensions), served forward-untouched with the upstream whole-request 422/413 error surface, per-member calibration, upstream state truncation, and the /v1/models /health /ready bodies verbatim. The hybrid members and a native executor are future work reusing this wire module. - src/models/vela: shared wire module (envelope preamble with upstream's detail triples, SystemOneError mapping, package gate with a 7-file full-hash pin table, response finishing and the confidence formula mirrors) - src/frontend/vela.py: worker with single-flight 503 lock, warmup before readiness, VELA2_API_KEY 401 gate - recipe/vela: requirements, three example requests (card PII, set threshold override, CJK offsets), compare_reference.py driving the pinned vela2_serve app and the worker pipeline over one engine with exact-match parity - tests/vela: 94 CPU-only contract tests (no weights, no torch) and the frozen oracle fixtures as the reference baseline - docs: supported-models rows, README, recipe index, mkdocs nav Validated on CPU FP32: startup with all-kind warmup, the example and rejection parity fixtures, exact-match reference parity, byte-identical Rust-frontend forwarding. --- README.md | 5 +- docs/supported-models.md | 4 + mkdocs.yml | 1 + recipe/README.md | 4 + recipe/vela/compare_reference.py | 275 ++++++++++++++ recipe/vela/example-request-cjk.json | 32 ++ .../vela/example-request-set-threshold.json | 16 + recipe/vela/example-request.json | 51 +++ recipe/vela/reference.md | 266 ++++++++++++++ recipe/vela/requirements.txt | 15 + recipe/vela/test_compare_reference.py | 164 +++++++++ src/frontend/vela.py | 223 ++++++++++++ src/models/vela/README.md | 194 ++++++++++ src/models/vela/__init__.py | 7 + src/models/vela/engine.py | 278 +++++++++++++++ src/models/vela/postprocess.py | 144 ++++++++ src/models/vela/preprocess.py | 42 +++ src/models/vela/protocol.py | 125 +++++++ tests/vela/__init__.py | 0 .../fixtures/oracle-example-request-cjk.json | 175 +++++++++ .../oracle-example-request-set-threshold.json | 81 +++++ .../vela/fixtures/oracle-example-request.json | 175 +++++++++ tests/vela/test_engine_manifest.py | 219 ++++++++++++ tests/vela/test_postprocess.py | 167 +++++++++ tests/vela/test_preprocess.py | 63 ++++ tests/vela/test_protocol.py | 186 ++++++++++ tests/vela/test_worker.py | 336 ++++++++++++++++++ tests/vela/test_worker_server.py | 282 +++++++++++++++ 28 files changed, 3529 insertions(+), 1 deletion(-) create mode 100644 recipe/vela/compare_reference.py create mode 100644 recipe/vela/example-request-cjk.json create mode 100644 recipe/vela/example-request-set-threshold.json create mode 100644 recipe/vela/example-request.json create mode 100644 recipe/vela/reference.md create mode 100644 recipe/vela/requirements.txt create mode 100644 recipe/vela/test_compare_reference.py create mode 100644 src/frontend/vela.py create mode 100644 src/models/vela/README.md create mode 100644 src/models/vela/__init__.py create mode 100644 src/models/vela/engine.py create mode 100644 src/models/vela/postprocess.py create mode 100644 src/models/vela/preprocess.py create mode 100644 src/models/vela/protocol.py create mode 100644 tests/vela/__init__.py create mode 100644 tests/vela/fixtures/oracle-example-request-cjk.json create mode 100644 tests/vela/fixtures/oracle-example-request-set-threshold.json create mode 100644 tests/vela/fixtures/oracle-example-request.json create mode 100644 tests/vela/test_engine_manifest.py create mode 100644 tests/vela/test_postprocess.py create mode 100644 tests/vela/test_preprocess.py create mode 100644 tests/vela/test_protocol.py create mode 100644 tests/vela/test_worker.py create mode 100644 tests/vela/test_worker_server.py diff --git a/README.md b/README.md index e4316082..b2327ce0 100644 --- a/README.md +++ b/README.md @@ -164,7 +164,9 @@ Python screenshot worker, Valen-Preview-0923 has a Python reference worker for text and one-image state, CLM has a stub-encoder contract recipe, and OmniJev-4B v1.1 has its checkpoint export and CPU contract, with no worker yet. D3 (Decision 3.0) has a Python reference worker that runs the checkpoint's own -pinned runtime for text state: +pinned runtime for text state, and Vela 2.0 0.3B has one that serves all five +SystemOne question kinds (`choice`/`noul`/`score` plus the `span`/`set` +extensions) on the checkpoint's own pinned runtime: | Model | Status | | --- | --- | @@ -179,6 +181,7 @@ pinned runtime for text state: | LFM2.5-350M | [Transformers worker](recipe/lfm2/README.md); text state and independent choice questions | | OmniJev-4B v1.1 | [Checkpoint export and CPU contract](recipe/omnijev/README.md) only; the native worker is in progress ([#114](https://github.com/ThinkFlowLab/system1-omni/issues/114)) | | D3 (Decision 3.0) | [Python reference worker](recipe/d3/reference.md) on the checkpoint's pinned runtime; `choice`/`noul`/`score` text state, media refused until stage 2; author-run on A100 (evidence in the integration PR) | +| Vela 2.0 0.3B | [Python reference worker](recipe/vela/reference.md) on the checkpoint's pinned runtime; all five kinds (`choice`/`noul`/`score`/`span`/`set`) for text state; CPU FP32 validated (author-run, evidence in the integration PR) | [Supported models and hardware](docs/supported-models.md) lists the devices and where each worker has been run. diff --git a/docs/supported-models.md b/docs/supported-models.md index 3ca05477..2cd5e290 100644 --- a/docs/supported-models.md +++ b/docs/supported-models.md @@ -24,6 +24,7 @@ Models that are being added are also tracked in issues labeled [new model](https | CLM-v0.1-8B | [Native Rust engine](../src/models/clm/README.md), `clm-run`, in front of a Qwen3-8B `/v1/embeddings` server | Engine-only agreement **4.5e-06** on identical vectors ([#29](https://github.com/ThinkFlowLab/system1-omni/pull/29), [validation](../recipe/clm/native/VALIDATION.md)); contract checks documented ([#23](https://github.com/ThinkFlowLab/system1-omni/pull/23)) | **Validated on compute capability 8.9** ([#29](https://github.com/ThinkFlowLab/system1-omni/pull/29), [validation](../recipe/clm/native/VALIDATION.md)) | Unverified | Rust to build and a separate Qwen3-8B `/v1/embeddings` server for the encoder; the engine itself has no device-specific code | | OmniJev-4B v1.1 | None yet: [checkpoint export and CPU contract](../recipe/omnijev/README.md); the native worker is in progress ([#114](https://github.com/ThinkFlowLab/system1-omni/issues/114)) | Request contract, preparation, heads and finishing checked against the reference ([#120](https://github.com/ThinkFlowLab/system1-omni/pull/120)); no inference | Not yet | Not supported | Python 3.10 reference environment for the export and fixtures | | D3 (Decision 3.0) | [Python reference worker](../recipe/d3/reference.md) running the checkpoint's pinned runtime | Unverified | Author-run on NVIDIA A100 (sm_80): three-kind serving smoke with byte-identical frontend forwarding and reference parity (exact decisions, max delta 0.0, 348 == 348 tokens); evidence in the integration PR. Owner: @MrDongsls. | Unverified | Python 3.12, the [pinned runtime requirements](../recipe/d3/requirements.txt), the [pinned package](https://huggingface.co/vllm-sr/d3) (~52 GB) | +| Vela 2.0 0.3B | [Python reference worker](../recipe/vela/reference.md) running the checkpoint's pinned runtime | **CPU FP32 validated** (author-run, x86 WSL2, 24 cores): startup with all-kind warmup, the five charter fixtures, and exact-match reference parity against the pinned upstream server on all five (three 200s byte-equal incl. key order, agreed 422/413); byte-identical frontend forwarding; evidence in the integration PR. Owner: @MrDongsls. | Unverified (bf16-autocast is an untested deployment option) | Not supported | Python 3.12, the [pinned requirements](../recipe/vela/requirements.txt), the [pinned package](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) (1.24 GB FP32 weights; ~4.4 GB with ONNX/assets) | - **Validated:** covered by the recipe on `main` or by the checks in the linked merged pull request. - **Author-run:** recorded by the recipe author on the stated hardware and not independently reproduced; the row names its owner. @@ -34,6 +35,9 @@ The Cua-S1 workers answer `choice` questions only. LAYA's English worker, Open-Jev and CLM support `choice`, `score`, and `noul` text questions. D3's reference worker serves the same three kinds for text state; its media inputs stay refused for now and will be integrated soon. +Vela 2.0 0.3B's reference worker serves all five kinds (`choice`, `noul`, +`score`, plus the `span`/`set` extensions with their Noul views) for text +state on CPU FP32; the hybrid members (0.8B/4B/9B) are separate slices. CLM has a row per path: the merged `clm-serve` recipe exercises those answer shapes with stub embeddings and says nothing about decision quality, while the native engine's CUDA row is the path through a real Qwen3-8B encoder. Its engine-only figure comes from the same comparison run diff --git a/mkdocs.yml b/mkdocs.yml index 055b7c24..7e912aeb 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -76,4 +76,5 @@ nav: - CLM stub-encoder contract: recipe/clm/README.md - LFM2.5-350M choice worker: recipe/lfm2/README.md - D3 reference worker: recipe/d3/reference.md + - Vela 2.0 0.3B reference worker: recipe/vela/reference.md - Contributing: CONTRIBUTING.md diff --git a/recipe/README.md b/recipe/README.md index 32f83cd0..3950951d 100644 --- a/recipe/README.md +++ b/recipe/README.md @@ -28,6 +28,10 @@ For a first real decision, follow the [complete CPU walkthrough](../docs/getting - [D3 reference worker](d3/reference.md): download the pinned vllm-sr/d3 package, verify its runtime manifest, and serve `choice`/`noul`/`score` text decisions through the Python worker and the Rust frontend (serving and parity validated on A100). +- [Vela 2.0 0.3B reference worker](vela/reference.md): download the pinned + vllm-sr/Vela-2.0-0.3B package, pass its full-hash package gate, and serve all + five SystemOne question kinds (`choice`/`noul`/`score` plus the `span`/`set` + extensions) through the Python worker and the Rust frontend on CPU FP32. - [LFM2.5-350M choice worker](lfm2/README.md): score text decisions with independent questions, hybrid cache branching and configurable candidate batches. diff --git a/recipe/vela/compare_reference.py b/recipe/vela/compare_reference.py new file mode 100644 index 00000000..46398d47 --- /dev/null +++ b/recipe/vela/compare_reference.py @@ -0,0 +1,275 @@ +#!/usr/bin/env python3 +"""Compare the Vela 2.0 0.3B worker pipeline against the pinned upstream server. + +Runs each fixture request through two paths that share one loaded pinned +engine, so the comparison isolates this repository's admission and +orchestration from the checkpoint's own code: + +- reference: the pinned package's own ``vela2_serve.py`` app, driven + through FastAPI's test client exactly as a client would (routing, body + parsing, whole-request 422/413 mapping included); +- worker: the production ``frontend.vela.decide`` pipeline (protocol + envelope admission, pinned ``system_one_request`` execution, response + finishing). + +Both paths execute the same verified runtime, and the worker delegates +response construction to the same pinned code — so the criterion is the +charter's reference parity: exact match, including JSON key order, with +rejections agreeing on status and body (``{"detail": [{"loc", "msg", +"type"}]}``). ``--export`` additionally freezes the oracle fixture — +request, SystemOne response and the predict()-level answers — for the +future native encoder slices, and writes it only after a passing, +non-rejected comparison. Requires the pinned package, the reference +dependencies, and fastapi+uvicorn+httpx; see recipe/vela/reference.md. + +The default fixture set is the recipe's three example requests plus +five synthetic rejections: an invalid span question (whole-request +422), a schema that alone exceeds the 8,192-token window (413), and +the three envelope 422s (missing field, extra field, wrong model +name) that freeze the worker's preamble admission against the pinned +runtime's. CPU FP32 is the oracle setting; GPU bf16-autocast is a +deployment option, not this tool's default. +""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +MODEL_NAME = "vllm-sr/Vela-2.0-0.3B" + + +def parse_args(argv=None): + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument("--model", required=True, help="pinned Vela 2.0 0.3B package directory") + parser.add_argument("--device", default="cpu", help="torch device (default: the CPU FP32 oracle)") + parser.add_argument( + "--fixture", + action="append", + default=None, + help="request body file (repeatable; default: the recipe examples plus the " + "synthetic invalid-span and over-limit rejections)", + ) + parser.add_argument( + "--export", + default=None, + help="write the frozen oracle fixture (request, SystemOne response, predict-level " + "answers per passing 200 case); only passing, non-rejected comparisons are written", + ) + return parser.parse_args(argv) + + +def over_limit_body(): + """A schema that alone cannot fit the 8,192-token window. + + 255 choice options with ~300-character descriptions: far past + ``max_length`` before any state is rendered, which is exactly the + upstream 413 condition (questions + schema alone; state text would be + truncated, never 413'd). + """ + + criteria = { + f"option_{index}": ( + f"a deliberately long description repeating filler text to overflow the " + f"window for option {index}: " + "filler " * 40 + ) + for index in range(255) + } + return { + "model": MODEL_NAME, + "state": "short", + "questions": { + "big": {"type": "choice", "instructions": "Pick one.", "criteria": criteria} + }, + } + + +def invalid_span_body(): + """A span question whose criteria is a list, not a label mapping.""" + + return { + "model": MODEL_NAME, + "state": "short", + "questions": { + "bad": { + "type": "span", + "instructions": "Which spans?", + "criteria": ["PERSON", "EMAIL_ADDRESS"], + } + }, + } + + +def missing_model_body(): + """An envelope missing the model field (the preamble's 'missing' 422).""" + + return {"state": "short", "questions": {"q": {"type": "noul", "instructions": "any?"}}} + + +def extra_field_body(): + """An envelope with a field the preamble does not permit ('extra_forbidden').""" + + return { + "model": MODEL_NAME, + "state": "short", + "questions": {"q": {"type": "noul", "instructions": "any?"}}, + "images": [], + } + + +def wrong_model_body(): + """Another member's name on this runtime ('value_error.model' 422).""" + + return { + "model": "vllm-sr/Vela-2.0-0.8B", + "state": "short", + "questions": {"q": {"type": "noul", "instructions": "any?"}}, + } + + +def default_cases(): + return [ + ("example-request", HERE / "example-request.json"), + ("example-request-set-threshold", HERE / "example-request-set-threshold.json"), + ("example-request-cjk", HERE / "example-request-cjk.json"), + ("invalid-span-422", invalid_span_body()), + ("over-limit-413", over_limit_body()), + ("missing-model-422", missing_model_body()), + ("extra-field-422", extra_field_body()), + ("wrong-model-422", wrong_model_body()), + ] + + +def load_cases(paths): + if not paths: + cases = default_cases() + else: + cases = [(Path(path).name, path) for path in paths] + return [ + (name, body if isinstance(body, dict) else json.loads(Path(body).read_text(encoding="utf-8"))) + for name, body in cases + ] + + +def reference_response(client, body): + """One request through the pinned upstream server app. + + Returns the (status, parsed-body) pair a client observes. + """ + + response = client.post("/v1/systemone", json=body) + return response.status_code, response.json() + + +def worker_response(raw, executor, decide): + """One request through the production worker pipeline. + + Returns (200, response) or the (status, body) rejection a client + would observe. + """ + + try: + return 200, decide(raw, executor) + except Exception as exc: # noqa: BLE001 - reported through the comparison + status = getattr(exc, "status", 500) + body = getattr(exc, "body", None) + return status, body if body is not None else {"detail": str(exc)} + + +def compare_case(name, worker, reference): + """Exact-match comparison of the two observed results. + + Returns (report lines, passed). Key order is part of the match: + insertion order is the pinned runtime's answer/question order. + """ + + lines = [f"case: {name}", "reference path: pinned vela2_serve app (FastAPI test client)"] + problems = [] + if worker == reference: + status = worker[0] + lines.append(f"both paths: HTTP {status}") + if status == 200: + answers = worker[1].get("answers", {}) + usage = worker[1].get("usage", {}) + lines.append(f"answers: {len(answers)} key(s) in pinned order") + lines.append(f"input_tokens: {usage.get('input_tokens')} == reference") + else: + lines.append(f"rejection body: {json.dumps(worker[1], ensure_ascii=False)[:160]}") + lines.append("exact match (status, body, key order): PASS") + return lines, True + problems.append("results differ") + if worker[0] != reference[0]: + problems.append(f"status {worker[0]} != {reference[0]}") + lines.append(f"worker: HTTP {worker[0]}") + lines.append(f"reference: HTTP {reference[0]}") + lines.append("exact match (status, body, key order): FAIL: " + "; ".join(problems)) + return lines, False + + +def export_allowed(ok, worker): + """The oracle freezes served 200 responses, never a rejection.""" + + return ok and worker[0] == 200 + + +def export_oracle(name, body, response, predict, path): + """Freeze one passing case for the future native encoder slices.""" + + oracle = { + "case": name, + "request": body, + "system_one": response, + "predict": predict, + } + Path(path).write_text(json.dumps(oracle, indent=1, ensure_ascii=False) + "\n", encoding="utf-8") + return oracle + + +def main(argv=None): + args = parse_args(argv) + sys.path.insert(0, str(HERE.parents[1] / "src")) + from fastapi.testclient import TestClient + + from frontend.vela import decide + from models.vela.engine import Vela2Executor + + executor = Vela2Executor(args.model, device=args.device) + import vela2_inference + import vela2_serve + + app = vela2_serve.create_app(executor.engine, model_name=executor.model_name, api_key=None) + client = TestClient(app) + + exported = 0 + failed = 0 + try: + for name, body in load_cases(args.fixture): + raw = json.dumps(body).encode("utf-8") + worker = worker_response(raw, executor, decide) + reference = reference_response(client, body) + lines, ok = compare_case(name, worker, reference) + print("\n".join(lines)) + if not ok: + failed += 1 + elif args.export and export_allowed(ok, worker): + req, _plan = vela2_inference.system_one_to_predict(body["state"], body["questions"]) + target = Path(args.export) + if target.is_dir(): + target = target / f"oracle-{name}.json" + export_oracle(name, body, worker[1], executor.engine.predict(req), target) + exported += 1 + print() + finally: + executor.close() + if args.export: + print(f"oracle cases exported: {exported}" if exported else + "oracle export skipped: no passing 200-response comparison to freeze") + print("compare_reference: ok" if not failed else f"compare_reference: {failed} FAILED") + return 1 if failed else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/recipe/vela/example-request-cjk.json b/recipe/vela/example-request-cjk.json new file mode 100644 index 00000000..bd3ab688 --- /dev/null +++ b/recipe/vela/example-request-cjk.json @@ -0,0 +1,32 @@ +{ + "model": "vllm-sr/Vela-2.0-0.3B", + "state": { + "request": "客户张伟（zhang.wei@example.cn）于2026年10月1日在上海市浦东新区世纪大道100号办理了业务，联系电话13800138000。", + "source": "客户的业务申请需要在三个工作日内审核完成。", + "answer": "客户的业务已经即时完成了。" + }, + "questions": { + "pii": { + "type": "span", + "instructions": "Which spans are personal information?", + "criteria": { + "PERSON": "a person's name", + "EMAIL_ADDRESS": "an e-mail address", + "PHONE_NUMBER": "a telephone number", + "STREET_ADDRESS": "a street address or postal address" + }, + "over": "request" + }, + "unsupported": { + "type": "span", + "instructions": "Which spans of the answer are not supported by the context?", + "criteria": { + "unsupported": "a claim not supported by the context" + } + }, + "billing": { + "type": "noul", + "instructions": "这个请求是否与账单有关？" + } + } +} diff --git a/recipe/vela/example-request-set-threshold.json b/recipe/vela/example-request-set-threshold.json new file mode 100644 index 00000000..901927fc --- /dev/null +++ b/recipe/vela/example-request-set-threshold.json @@ -0,0 +1,16 @@ +{ + "model": "vllm-sr/Vela-2.0-0.3B", + "state": "My card was charged twice and the parcel never arrived.", + "questions": { + "issues": { + "type": "set", + "instructions": "Which issues does the customer report?", + "criteria": { + "billing": "payments, charges, refunds or invoices", + "shipping": "delivery of an order or a parcel", + "login": "signing in, passwords or account access" + }, + "threshold": 0.15 + } + } +} diff --git a/recipe/vela/example-request.json b/recipe/vela/example-request.json new file mode 100644 index 00000000..8e0557b1 --- /dev/null +++ b/recipe/vela/example-request.json @@ -0,0 +1,51 @@ +{ + "model": "vllm-sr/Vela-2.0-0.3B", + "state": { + "request": "Hi, I'm Tom Baker (tom.baker@example.com). What is the maximum daily dose of paracetamol for an adult?", + "source": "For adults, the maximum dose of paracetamol is 4 grams in 24 hours, taken as 500 mg to 1 g every 4 to 6 hours.", + "answer": "Adults can take up to 6 grams of paracetamol in 24 hours, in doses of 500 mg to 1 g every 4 to 6 hours." + }, + "questions": { + "pii": { + "type": "span", + "instructions": "Which spans are personal information?", + "criteria": { + "AGE": "a person's age", + "CREDIT_CARD": "a payment card number", + "DATE_TIME": "a date, time or date of birth", + "DOMAIN_NAME": "an internet domain name or website address", + "EMAIL_ADDRESS": "an e-mail address", + "GPE": "a country, city, state or other geopolitical place", + "IBAN_CODE": "an international bank account number (IBAN)", + "IP_ADDRESS": "an IPv4 or IPv6 address", + "NRP": "a nationality, religious or political group", + "ORGANIZATION": "the name of a company, institution or other organisation", + "PERSON": "a person's name", + "PHONE_NUMBER": "a telephone number", + "STREET_ADDRESS": "a street address or postal address", + "TITLE": "a personal or professional title such as Dr. or Mrs.", + "US_DRIVER_LICENSE": "a US driver's licence number", + "US_SSN": "a US social security number", + "ZIP_CODE": "a postal or ZIP code" + }, + "over": "request" + }, + "halu": { + "type": "span", + "instructions": "Which spans of the answer are not supported by the context?", + "criteria": { + "unsupported": "a claim not supported by the context" + } + }, + "domain": { + "type": "choice", + "instructions": "Which subject area is this request about?", + "over": "request", + "criteria": { + "health": "medicine, clinical practice, nutrition, ageing or sexual health", + "math": "arithmetic, algebra, geometry, statistics or other mathematics", + "other": "a subject that fits none of the listed areas" + } + } + } +} diff --git a/recipe/vela/reference.md b/recipe/vela/reference.md new file mode 100644 index 00000000..877a9d7a --- /dev/null +++ b/recipe/vela/reference.md @@ -0,0 +1,266 @@ +# Vela 2.0 0.3B reference worker + +This recipe serves Vela 2.0 0.3B through the Python reference worker and +the existing Rust frontend. The model-side implementation is under +[`src/models/vela/`](../../src/models/vela/), and the worker is +[`src/frontend/vela.py`](../../src/frontend/vela.py). + +Run the commands from the repository root with Python 3.12. The default +and evidence-backed setting is CPU FP32 — the reference oracle; a CUDA +device works for serving but its bf16-autocast backbone is a deployment +option, not the parity baseline. The recipe records the procedure; +measured results and verbatim run output live in the pull request +description, not here. + +## Scope + +The slice is: + +- package: [`vllm-sr/Vela-2.0-0.3B`](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) + at the pinned revision below; +- the Python reference worker behind the Rust frontend, executing the + checkpoint's own pinned runtime (`vela2_inference.py` and siblings) + after a full-hash package gate; +- text/JSON state and all five typed kinds — `choice`, `noul`, `score`, + `span`, `set` — in one request, with the span/set response extensions + (`spans`/`sets`/`thresholds` and their Noul views) served + forward-untouched; +- FP32 parameters, one bidirectional forward per assembled sequence, + calibrated readouts; zero generated tokens. + +Out of scope for this slice: the hybrid members (0.8B/4B/9B — their +reference workers are a separate slice), the ONNX backend, any native +Rust/CUDA executor (the frozen RFC's ModernBERT-profile route), and +`noul_calibration` (a hybrid-only feature that does not exist on 0.3B). + +## Architecture and ownership + +~~~text +Rust frontend + -> Python worker + -> admit: JSON transport + upstream envelope preamble (protocol) + -> verify: config identity pin, full SHA-256 of the 7-file table, + exact weights size, then import it (engine) + -> execute: the checkpoint's own vela2_inference system_one_request + -> finish: response-order/key-set checks + formula mirrors (postprocess) +~~~ + +The ownership boundary follows +[`docs/architecture.md`](../../docs/architecture.md) and the +[model contract](../../src/models/vela/README.md): + +- `protocol.py` owns the envelope preamble with upstream's exact 422 + triples; deeper validation (state parts, question semantics, the 413) + is the pinned runtime's, verbatim; +- `preprocess.py` maps the runtime's `SystemOneError` to 422/413; +- `engine.py` owns the pinned runtime and the package gate; +- `postprocess.py` owns the pure formulas and response finishing; +- `frontend.vela` owns HTTP lifecycle, the single-flight 503 lock, the + `VELA2_API_KEY` 401 gate and warmup-before-readiness; +- The Rust frontend forwards `/v1/systemone` and `/health` unchanged. + +## Pinned artifacts + +| Artifact | Value | +| --- | --- | +| Package | [vllm-sr/Vela-2.0-0.3B](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) | +| Pinned revision | `d6f03aa9baca4f9017fa8f8e9c2764d3bed3f409` | +| Package gate | config structural identity + a 7-file SHA-256 table pinned per revision in `engine.py`, full hash before import, no unlisted importables in the package root, exact 1,236,473,252-byte weights check | +| License | Apache-2.0; tokenizer material under Gemma Terms of Use | + +The gate's file table is checked in (not read from the package, which +ships no manifest), so a later snapshot — self-consistent or not — +cannot pass as the pinned revision. Re-check before changing the pin: + +~~~sh +curl -sS "https://huggingface.co/api/models/vllm-sr/Vela-2.0-0.3B/revision/d6f03aa9baca4f9017fa8f8e9c2764d3bed3f409" | jq -r '.sha' +sha256sum weights/vela-2.0-0.3b/{vela2_inference.py,modeling_vela2.py,configuration_vela2.py,vela2_serve.py,config.json,calibration.json,tokenizer.json} +~~~ + +## Install and prepare the artifacts + +~~~sh +python3.12 -m venv .venv +.venv/bin/python -m pip install --upgrade pip +.venv/bin/python -m pip install -r recipe/vela/requirements.txt pytest +~~~ + +~~~sh +export VELA_PKG="$PWD/weights/vela-2.0-0.3b" +export VELA_REV=d6f03aa9baca4f9017fa8f8e9c2764d3bed3f409 +mkdir -p weights +.venv/bin/hf download vllm-sr/Vela-2.0-0.3B --revision "$VELA_REV" --local-dir "$VELA_PKG" +~~~ + +The full package is about 4.4 GB (the 1.24 GB FP32 safetensors, a 34 MB +tokenizer, the ONNX exports and doc assets); serving needs roughly +2 GB of RAM for the FP32 model plus working memory for the batch rows. +The worker refuses to start when the config is not the pinned member's, +when any of the 7 gated files fails its hash, when the package root +holds an importable entry the table does not name, or when the weights +file is not the pinned size — that gate runs before any package code is +imported. + +## Start the worker + +~~~sh +PYTHONPATH=src .venv/bin/python -m frontend.vela \ + --model "$VELA_PKG" --device cpu --port 8000 +~~~ + +The worker verifies the package, loads the FP32 model, runs a real +warmup request covering every question kind, and only then binds the +port: + +~~~sh +curl -sS -i http://127.0.0.1:8000/health # {"status": "ok"} (upstream verbatim) +curl -sS -i http://127.0.0.1:8000/ready # {"ready": true} +curl -sS -i http://127.0.0.1:8000/v1/models +~~~ + +Optional: set `VELA2_API_KEY` to require `Authorization: Bearer +<$VELA2_API_KEY>` on `/v1/systemone` and `/v1/models` (401 with the +upstream message otherwise), and `VELA2_MODEL_NAME` (or `--name`) to +serve under a different name. One request is served at a time; a +concurrent one gets 503. + +## Example requests + +Three complete runnable requests ship with the recipe: + +- [`example-request.json`](example-request.json) — the model card's PII + example: a span question over the 17 trained PII labels (the PII + length-rule threshold is applied and echoed), a hallucination span + question over the answer part, and a domain `choice` question; +- [`example-request-set-threshold.json`](example-request-set-threshold.json) + — a `set` question with a per-question `threshold` override (the + applied value 0.15 is echoed in `thresholds`; the pinned predict + level marks overrides with rule `given` on span answers); +- [`example-request-cjk.json`](example-request-cjk.json) — CJK state + exercising Unicode code-point offsets (one unit per CJK character). + +~~~sh +curl -sS http://127.0.0.1:8000/v1/systemone \ + -H 'Content-Type: application/json' \ + --data-binary @recipe/vela/example-request.json +~~~ + +Expected response shape (a serving smoke test, not an accuracy result; +the exact numbers come from the checkpoint and are frozen by the parity +tool below): + +~~~json +{ + "model": "vllm-sr/Vela-2.0-0.3B", + "answers": { + "pii": {"type": "noul", "noul": 0.0}, + "halu": {"type": "noul", "noul": 0.0}, + "domain": {"type": "choice", "choice": "…", "confidence": 0.0, "probabilities": {"…": 0.0}} + }, + "usage": {"input_tokens": 0, "output_tokens": 0}, + "spans": {"pii": [{"label": "…", "start": 0, "end": 0, "text": "…", "probability": 0.0}]}, + "thresholds": {"pii": 0.0, "halu": 0.55} +} +~~~ + +## Start and verify the Rust frontend + +~~~sh +cargo build --release --locked +OMNI_JEV_BIND=127.0.0.1:8080 \ +OMNI_JEV_BACKEND_URL=http://127.0.0.1:8000 \ + target/release/omni-jev +~~~ + +Send the same request through port 8080 and compare byte-for-byte: + +~~~sh +curl -fsS http://127.0.0.1:8000/v1/systemone \ + -H 'Content-Type: application/json' \ + --data-binary @recipe/vela/example-request.json > /tmp/vela-worker.json +curl -fsS http://127.0.0.1:8080/v1/systemone \ + -H 'Content-Type: application/json' \ + --data-binary @recipe/vela/example-request.json > /tmp/vela-frontend.json +cmp /tmp/vela-worker.json /tmp/vela-frontend.json +~~~ + +Worker responses are forwarded unchanged. Frontend-generated 502/504 +responses are frontend errors, not model responses. + +## Error behavior + +The request, response and error contract — the two error layers, the +five answer shapes, usage counting and the state-truncation order — is +the [model contract](../../src/models/vela/README.md). In short: +transport errors follow the shared worker discipline; every invalid +field rejects the whole request with the upstream 422 +`{"detail": [{"loc", "msg", "type"}]}`; 413 `too_long` fires only when +questions plus schema alone exceed 8,192 tokens; and over-limit state +text is **truncated as the reference truncates it** (shrink part first, +floor 64 then 1, then protected parts; windowed re-coverage of cut +labelled parts) — a deliberate, loudly documented divergence from the +repository's no-silent-truncation convention, with the guardrail risk +recorded in the model contract. + +## Repository-side validation + +~~~sh +PYTHONPATH=src .venv/bin/python -m pytest tests/vela -q +PYTHONPATH=src .venv/bin/python -m compileall -q src/models/vela src/frontend/vela.py tests/vela +.venv/bin/python recipe/vela/test_compare_reference.py +cargo fmt --all --check +cargo clippy --workspace --locked --all-targets --features omni-laya/serve -- -D warnings +cargo test --workspace --locked --features omni-laya/serve +cargo build --workspace --release --locked --features omni-laya/serve +~~~ + +Run these from the repository root. The recorded outputs of a run belong +in the pull request description, not in this recipe. + +## Reference parity (author-side, opt-in) + +[`compare_reference.py`](compare_reference.py) drives the fixture set — +the three example requests and five synthetic rejections (an +invalid-span 422, an over-limit 413, and the missing-field / +extra-field / wrong-model envelope 422s) — through two paths that +share one loaded engine: the pinned +`vela2_serve.py` app via FastAPI's test client, and the production +`frontend.vela.decide` pipeline. The criterion is the charter's +reference parity: **exact match** including JSON key order for 200s, +and agreement on status and body for rejections (needs `pip install +fastapi uvicorn httpx` on top of the requirements file). +`--export tests/vela/fixtures/oracle-.json` freezes each passing +200 case — request, SystemOne response and predict()-level answers — as +the frozen comparison baseline for the frozen RFC's native encoder +slices, and writes it only after a passing comparison. + +~~~sh +PYTHONPATH=src .venv/bin/python recipe/vela/compare_reference.py \ + --model "$VELA_PKG" --device cpu \ + --export tests/vela/fixtures/ +~~~ + +Run it by hand and record the verbatim output, host, package revision +and dependency versions in the pull request description. CPU FP32 is +the oracle setting; keep flash-linear-attention out of the fixture path +(it does not exist on this encoder member anyway) and do not substitute +GPU bf16-autocast numbers for the baseline. + +## Troubleshooting and limits + +- The package directory must be the pinned revision's download; the + worker refuses foreign configs, hash mismatches, unlisted importables + and wrong weights sizes at startup, before importing anything. +- The 8 MiB body cap is slice-scoped (upstream sets no total-body + limit); a valid request bounded by the 8,192-token window is far + below it. +- `/health` and `/ready` are upstream's minimal bodies on this member, + not the richer shape of some other workers in this repository. +- Long states are truncated (see Error behavior); if a consumer must + never miss tail text, check `usage.input_tokens` against the expected + budget or enforce limits upstream of the worker. +- GPU bf16-autocast serving (`--device cuda`) is untested in this + slice's evidence; treat it as a deployment option to validate + separately. +- GPU evidence for the supported-models table is recorded in the + integration pull request, not in this recipe. diff --git a/recipe/vela/requirements.txt b/recipe/vela/requirements.txt new file mode 100644 index 00000000..9f2cb6cf --- /dev/null +++ b/recipe/vela/requirements.txt @@ -0,0 +1,15 @@ +# Reference-worker dependencies for Vela 2.0 0.3B (torch backend, CPU FP32 +# oracle setting). The upstream package ships no requirements file; its +# vela2_inference.py header names numpy + tokenizers, and the torch backend +# adds torch + transformers + safetensors. Versions are the ones the worker +# was validated with (Python 3.12); torch CPU wheels are enough — the GPU +# bf16-autocast deployment setting is not part of this slice's evidence. +torch==2.11.0 +transformers==5.19.0 +safetensors==0.8.0 +tokenizers==0.23.2 +numpy==2.5.3 +# Downloads of the pinned revision (hf download). +huggingface_hub==1.33.0 +# The reference comparison additionally needs the upstream server stack: +# pip install fastapi uvicorn httpx diff --git a/recipe/vela/test_compare_reference.py b/recipe/vela/test_compare_reference.py new file mode 100644 index 00000000..3db5eceb --- /dev/null +++ b/recipe/vela/test_compare_reference.py @@ -0,0 +1,164 @@ +#!/usr/bin/env python3 +"""Regression tests for recipe/vela/compare_reference.py. + +Checks the pure comparison, case-loading and export helpers without the +pinned package, weights, torch or fastapi: exact-match verdicts +including key order, rejection agreement, the default fixture set's +shape (the invalid-span 422 and over-limit 413 synthetic cases), the +oracle export gate, and the oracle export structure. + + python recipe/vela/test_compare_reference.py +""" + +from __future__ import annotations + +import json +import sys +import tempfile +from pathlib import Path + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE)) + +import compare_reference as tool + + +def response(answers=None, input_tokens=17): + value = { + "model": tool.MODEL_NAME, + "answers": answers if answers is not None else {"q": {"type": "noul", "noul": 0.75}}, + "usage": {"input_tokens": input_tokens, "output_tokens": 0}, + } + if answers is None: + value["thresholds"] = {"q": 0.3} + return value + + +def main() -> None: + results: list[tuple[str, bool, str]] = [] + + def check(name: str, condition: bool, detail: str = "") -> None: + results.append((name, condition, detail)) + + # 1. Matching 200 responses pass; every divergence fails. + reference = response() + lines, ok = tool.compare_case("fixture", (200, response()), (200, reference)) + report = "\n".join(lines) + check( + "exact 200 match passes", + ok and "exact match (status, body, key order): PASS" in report, + report, + ) + + lines, ok = tool.compare_case("fixture", (200, response(input_tokens=18)), (200, reference)) + check("token divergence fails", not ok, "\n".join(lines)) + + reordered = {"q": {"type": "noul", "noul": 0.75}} + other_first = {"other": {"type": "noul", "noul": 0.5}, "q": {"type": "noul", "noul": 0.75}} + lines, ok = tool.compare_case("fixture", (200, response(other_first)), (200, response(reordered))) + check("answer-set divergence fails", not ok, "\n".join(lines)) + + drifted = {"q": {"type": "noul", "noul": 0.7500000001}} + lines, ok = tool.compare_case("fixture", (200, response(drifted)), (200, reference)) + check("numeric drift fails (exact-match criterion)", not ok, "\n".join(lines)) + + # 2. Rejections agree only on both status and body. + rejection = {"detail": [{"loc": ["body", "questions", "bad", "criteria"], + "msg": "span criteria must be an object with 1 to 255 entries", + "type": "value_error"}]} + lines, ok = tool.compare_case("fixture", (422, rejection), (422, rejection)) + check("matching 422 rejections pass", ok, "\n".join(lines)) + + lines, ok = tool.compare_case("fixture", (413, rejection), (422, rejection)) + check("status mismatch fails", not ok, "\n".join(lines)) + + other = {"detail": [{"loc": ["body", "questions"], "msg": "different", "type": "value_error"}]} + lines, ok = tool.compare_case("fixture", (422, other), (422, rejection)) + check("body mismatch fails", not ok, "\n".join(lines)) + + lines, ok = tool.compare_case("fixture", (200, response()), (422, rejection)) + check("worker answering a rejected request fails", not ok, "\n".join(lines)) + + # 3. The default case set carries the charter's fixture list shape. + cases = tool.default_cases() + names = [name for name, _ in cases] + check( + "default cases are the three examples plus five rejections", + names == ["example-request", "example-request-set-threshold", "example-request-cjk", + "invalid-span-422", "over-limit-413", + "missing-model-422", "extra-field-422", "wrong-model-422"], + str(names), + ) + missing = dict(cases[5][1]) + check( + "the missing-model case is an envelope without the model field", + set(missing) == {"state", "questions"}, + json.dumps(missing)[:120], + ) + extra = dict(cases[6][1]) + check( + "the extra-field case carries a field the preamble forbids", + "images" in extra and set(extra) == {"model", "state", "questions", "images"}, + json.dumps(extra)[:120], + ) + wrong = dict(cases[7][1]) + check( + "the wrong-model case names the 0.8B member", + wrong["model"] == "vllm-sr/Vela-2.0-0.8B", + wrong["model"], + ) + invalid = dict(cases[3][1]) + check( + "the invalid-span case is a span question with list criteria", + invalid["questions"]["bad"]["type"] == "span" + and isinstance(invalid["questions"]["bad"]["criteria"], list), + json.dumps(invalid)[:200], + ) + over = dict(cases[4][1]) + check( + "the over-limit case is questions+schema overflow by construction", + len(over["questions"]["big"]["criteria"]) == 255, + json.dumps(over)[:120], + ) + + # 4. The oracle export freezes the full native-comparison surface. + body = {"model": tool.MODEL_NAME, "state": "s", + "questions": {"q": {"type": "noul", "instructions": "x"}}} + with tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "oracle.json" + oracle = tool.export_oracle( + "case", body, response(), {"answers": [{"id": "q", "type": "choice"}]}, path + ) + reread = json.loads(path.read_text(encoding="utf-8")) + check( + "oracle export round-trips", + oracle == reread + and oracle["request"] == body + and oracle["system_one"]["answers"]["q"]["noul"] == 0.75 + and oracle["predict"]["answers"][0]["id"] == "q", + json.dumps(oracle)[:200], + ) + + check("export allowed for a passing 200 case", + tool.export_allowed(True, (200, response()))) + check("export refused for a failed comparison", + not tool.export_allowed(False, (200, response()))) + check("export refused for an agreed rejection", + not tool.export_allowed(True, (422, rejection))) + + failed = 0 + for name, passed, detail in results: + print(f" {'ok ' if passed else 'FAIL'} {name}") + if not passed: + failed += 1 + print(" " + detail.strip().replace("\n", "\n ")[:400]) + print( + "compare_reference tests: ok" + if not failed + else f"compare_reference tests: {failed} FAILED" + ) + sys.exit(1 if failed else 0) + + +if __name__ == "__main__": + main() diff --git a/src/frontend/vela.py b/src/frontend/vela.py new file mode 100644 index 00000000..87e56cc1 --- /dev/null +++ b/src/frontend/vela.py @@ -0,0 +1,223 @@ +"""Python reference worker for Vela 2.0 0.3B. + +The Rust frontend remains the public transport layer. This worker owns +only HTTP lifecycle and request orchestration; wire admission lives in +models.vela.protocol, the pinned-runtime error mapping in +models.vela.preprocess, the pinned runtime and its package gate in +models.vela.engine, and response finishing in models.vela.postprocess. + +The error surface has two layers: transport errors follow the shared +Python-worker discipline, and every request rejection the pinned runtime +makes — an invalid field, an invalid question, a questions-overflow — is +the upstream whole-request 422 ``{"detail": [{"loc", "msg", "type"}]}`` +(or 413 ``too_long``), passed through verbatim. There are no per-question +errors. Over-limit state text is truncated exactly as the pinned +reference truncates it (see the model contract); only questions plus +schema that cannot fit the 8,192-token window are refused. +""" + +from __future__ import annotations + +import argparse +import hmac +import json +import logging +import os +import socket +import threading +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from typing import Any + +from models.vela.engine import Vela2Executor +from models.vela.postprocess import finish_response +from models.vela.preprocess import map_runtime_error, to_request_error +from models.vela.protocol import MODEL_NAME, MAX_BODY, RequestError, parse_body, admit_request + +LOG = logging.getLogger(__name__) + +# Upstream vela2_serve.py /v1/models entry, kept verbatim so clients see +# the reference server's description and release date. +MODELS_ENTRY = { + "name": None, # the served model name, filled per request + "release_date": "2026-09-29", + "description": ( + "Vela 2.0 0.3B: SystemOne Choice / Noul / Score " + "with span and set extensions, 8,192-token context" + ), +} +UNAUTHORIZED_BODY = {"detail": "Missing or invalid API key"} + + +def decide(raw: bytes, executor: Vela2Executor) -> dict[str, Any]: + """Admit, execute, and finish one complete request.""" + + body = admit_request(parse_body(raw), executor.served) + questions = body["questions"] + try: + response = executor.system_one(body) + except executor.SystemOneError as exc: + raise to_request_error(exc) from exc + except ValueError as exc: + # The pinned runtime is the validation authority for what the wire + # layer cannot foresee (state rendering, question semantics, the + # window budget); its rejections are client errors, not failures. + raise map_runtime_error(exc) from exc + return finish_response(questions, response) + + +class WorkerServer(ThreadingHTTPServer): + daemon_threads = False + + def __init__(self, address, executor: Vela2Executor, api_key: str | None = None): + self.executor = executor + self.api_key = api_key + self.inference_lock = threading.Lock() + self._executor_closed = False + super().__init__(address, Handler) + + def server_close(self): + super().server_close() + if not self._executor_closed: + self.executor.close() + self._executor_closed = True + + +class Handler(BaseHTTPRequestHandler): + def setup(self): + super().setup() + self.connection.settimeout(15) + + def log_message(self, format, *args): + # Do not log paths, prompts, or arbitrary request headers. + pass + + def send_json(self, status: int, value: dict): + raw = json.dumps(value, ensure_ascii=False, allow_nan=False).encode("utf-8") + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(raw))) + self.end_headers() + try: + self.wfile.write(raw) + except (BrokenPipeError, ConnectionResetError): + pass + + def _send_request_error(self, exc): + if exc.body is not None: + self.send_json(exc.status, exc.body) + else: + self.send_json(exc.status, {"detail": str(exc)}) + + def _unauthorized(self) -> bool: + """Upstream vela2_serve.py: no key configured means no gate; with + VELA2_API_KEY set, exactly ``Authorization: Bearer `` passes.""" + + key = self.server.api_key + if not key: + return False + got = self.headers.get("authorization", "") + if hmac.compare_digest(got, f"Bearer {key}"): + return False + self.send_json(401, UNAUTHORIZED_BODY) + return True + + def do_GET(self): + path = self.path + if path == "/health": + # Upstream verbatim; the worker binds the port only after + # loading and warmup, so answering at all means ready. + self.send_json(200, {"status": "ok"}) + elif path == "/ready": + self.send_json(200, {"ready": True}) + elif path == "/v1/models": + if self._unauthorized(): + return + entry = dict(MODELS_ENTRY) + entry["name"] = self.server.executor.model_name + self.send_json(200, {"models": [entry]}) + else: + self.send_json(404, {"detail": "unknown route"}) + + def do_POST(self): + if self.path != "/v1/systemone": + self.send_json(404, {"detail": "unknown route"}) + return + if self.headers.get("Transfer-Encoding"): + self.send_json(411, {"detail": "Content-Length is required; chunked requests are unsupported"}) + return + lengths = self.headers.get_all("Content-Length", []) + if len(lengths) != 1: + self.send_json(411, {"detail": "one Content-Length is required"}) + return + try: + length = int(lengths[0]) + except ValueError: + self.send_json(400, {"detail": "invalid Content-Length"}) + return + if length < 0 or length > MAX_BODY: + self.send_json(413, {"detail": "request exceeds the 8 MiB body limit"}) + return + if self.headers.get_content_type() != "application/json": + self.send_json(415, {"detail": "Content-Type must be application/json"}) + return + if self._unauthorized(): + return + if not self.server.inference_lock.acquire(blocking=False): + self.send_json(503, {"detail": "worker busy"}) + return + try: + raw = self.rfile.read(length) + if len(raw) != length: + self.send_json(400, {"detail": "incomplete body"}) + return + self.send_json(200, decide(raw, self.server.executor)) + except RequestError as exc: + self._send_request_error(exc) + except (TimeoutError, socket.timeout): + self.send_json(408, {"detail": "request body timed out"}) + except Exception as exc: + LOG.error("vela inference failed: %s", type(exc).__name__) + self.send_json(500, {"detail": "inference failed"}) + finally: + self.server.inference_lock.release() + + +def parse_args(argv=None): + parser = argparse.ArgumentParser(description="Python reference worker for Vela 2.0 0.3B") + parser.add_argument("--model", required=True, help="pinned Vela 2.0 0.3B package directory") + parser.add_argument("--device", default="cpu", help="torch device (reference contract: cpu)") + parser.add_argument("--batch-size", type=int, default=None) + parser.add_argument("--name", default=os.environ.get("VELA2_MODEL_NAME", MODEL_NAME), + help="model name accepted in the request's model field") + parser.add_argument("--host", default="127.0.0.1") + parser.add_argument("--port", type=int, default=8000) + return parser.parse_args(argv) + + +def main(argv=None): + args = parse_args(argv) + logging.basicConfig(level=logging.INFO) + executor = Vela2Executor( + args.model, + device=args.device, + model_name=args.name, + batch_size=args.batch_size, + ) + server = None + try: + executor.warmup() + server = WorkerServer((args.host, args.port), executor, + api_key=os.environ.get("VELA2_API_KEY") or None) + LOG.info("%s worker ready on %s:%s", executor.model_name, args.host, args.port) + server.serve_forever() + except KeyboardInterrupt: + pass + finally: + if server is not None: + server.server_close() + else: + executor.close() + + +if __name__ == "__main__": + main() diff --git a/src/models/vela/README.md b/src/models/vela/README.md new file mode 100644 index 00000000..4a5b4857 --- /dev/null +++ b/src/models/vela/README.md @@ -0,0 +1,194 @@ +# Vela 2.0 0.3B reference contract + +This directory contains the System1-Omni integration slice for +[Vela 2.0 0.3B](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) +(`vela2-unified`). It is a reference-serving boundary only: a Python +worker that runs the checkpoint's own pinned runtime for text state and +all five typed question kinds (`choice`, `noul`, `score`, `span`, `set`) +on `/v1/systemone`. Native execution is not part of this slice; per the +frozen RFC this worker is the Phase-1 entry that freezes the +text/token_ids input, the position-level final-hidden readout contract, +the error/length boundaries and the parity baseline for later +ModernBERT-profile work. + +The shared Vela 2.0 wire module is deliberately shaped for the family: +the envelope admission (`protocol`), the pinned-runtime error mapping +(`preprocess`) and response finishing (`postprocess`) are member-neutral; +this slice ships the 0.3B pinned executor (`engine`). The hybrid members +(0.8B/4B/9B) reuse the same wire layer with their own executors. + +## Pinned artifacts + +| Artifact | Value | +| --- | --- | +| Package | [vllm-sr/Vela-2.0-0.3B](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) | +| Pinned revision | `d6f03aa9baca4f9017fa8f8e9c2764d3bed3f409` | +| Identity | `vela2-unified` / `Vela2Model`, 309,114,369 parameters, all FP32 (1,236,473,252-byte safetensors) | +| Runtime | the checkpoint's own flat files: `vela2_inference.py`, `modeling_vela2.py`, `configuration_vela2.py`, `vela2_serve.py` | +| Calibration | the package's own `calibration.json` (never cross-copied between members) | +| License | Apache-2.0; the inherited tokenizer material carries Gemma Terms of Use | +| Runtime dependencies | [recipe/vela/requirements.txt](../../recipe/vela/requirements.txt) (torch, transformers, safetensors, tokenizers, numpy) | + +The package ships no manifest, so the executor's startup gate checks the +config's structural identity (model type, architecture, `max_length` +8,192, the 8 marker ids), then hashes every small file the runtime reads +or executes — the four runtime modules, `config.json`, +`calibration.json` and `tokenizer.json` — against the table checked in +for the pinned revision, before anything is imported, and refuses +importable entries the table does not name (`model.safetensors` is +checked by exact size; the runtime's strict safetensors load rejects +truncated or structurally mismatched weights, but not same-size +bit-level corruption). +A later self-consistent snapshot rewrites its own files and fails the +checked-in pin. + +## Current serving scope + +- The request envelope is upstream's three-field form, exactly: + `{"model", "state", "questions"}`. Any other key is a whole-request + 422 `extra_forbidden`; a missing key is a 422 `missing`; `model` must + be non-blank and equal the served name (`vllm-sr/Vela-2.0-0.3B`, or + `VELA2_MODEL_NAME`/`--name`), otherwise 422 `value_error.model`. +- `state` is a plain string or a JSON object/array. An object is mapped + onto typed parts (`user`/`context`/`answer`) by the pinned runtime's + key roles (request/user/prompt → user; source/context/document/tools/ + previous_answer → context; answer/response → answer; unknown keys → + context); non-string values render as canonical JSON. A string holding + a JSON object is read as that object. +- `questions` is a non-empty object of question ids to questions. All + five kinds are served forward-untouched; per-question limits (choice + 2–255 options, score 2–10 distinct levels, span/set 1–255 criteria) + and `over` resolution are the pinned runtime's validation, verbatim. +- `over` names a state key, a part name, or a list of them; default is + every part (span: the answer part if present, else user, else the + first part). `threshold` is allowed only on span/set and must be a + number in [0, 1]. Set-label Noul views (`".