Documentation publication: Omni PR #126 adds the roadmap page and README/navigation links. Read the documentation checklist. Use this issue for task ownership and completion; keep the documentation snapshot aligned when scope or priorities change.

System1-Omni roadmap

The next phase focuses on decision-inference performance, broader model support,
and community adoption through System1-Agents. This TODO list covers model
support, engine features, and agent integration over a roughly three-month
planning horizon. Milestones define order of work, not release dates or promised
speedups.

Planning snapshot: 2026-10-08, based on
Omni main at 4a79980
and
Agents main at 5d89f85.
Check the supported-model matrix for available paths.
Omni #83 tracks implementation separately from hardware validation.

Working on a TODO

Each checkbox describes a remaining deliverable, even when prerequisites are
merged. Related PRs are proposals or foundations, not instructions to merge them
unchanged. Claim work in the linked issue, record an owner and dependencies, and
split large items into focused PRs. Where there is no dedicated issue, open one
using the linked design context.

Complete an item only when its scoped implementation is merged, its acceptance
checks pass, and its recipe or evidence is available. Preserve model-specific
prompts, heads, calibration, and probabilities. Follow the
architecture contracts,
contribution checks, and
benchmark protocol.

Model support

• M1. Open-Jev-9B / 27B: Maintain correctness and performance baselines
  across mixed choice/score/noul, prompt lengths, question/candidate counts,
  and concurrency; publish parity gates, memory, and repeated-run results.
  Merged foundations: Omni #55, #92, and #102.
  Broader benchmark tooling: #116.
• M2. Cua-S1 4B: Publish a reproducible native screenshot serving recipe
  and integrated reference-parity coverage; qualify language and vision graph
  replay separately. Native vision is merged in Omni #64.
  Remaining scope: #10, #103, and #108.
• M3. Decider-2B: Complete request/response compilation and eager native
  serving; qualify batching, graph replay, and prefix reuse as separate changes.
  Related: Omni #57, #94, #110, #111,
  #112, and #113.
• M4. JEMM: Complete native text/image serving, verified export and
  calibration, typed errors, and parity for supported multi-image requests.
  Related: Omni #121, #118, and #122.
• M5. OmniJev-4B: Complete native multimodal execution after the contract,
  preparation, head, and export work; validate readout, abstention, and probability
  semantics against the pinned reference. Related: Omni #114 and #120.
• M6. openjev/openjev: Complete native execution and serving after CPU
  request compilation; qualify this checkpoint's scoring/calibration independently
  of Open-Jev-9B/27B. Related: Omni #95 and #124.
• M7. JEV-27B-VL: Complete and validate native multimodal serving and
  prefix caching, with bounded memory and independent branch state.
  Related: Omni #96.
• M8. CLM: Complete real-encoder serving and verify the trained heads;
  define and validate bounded candidate caching with cold/warm measurements.
  Checkpoint/scoring support is merged in Omni #28; remaining work:
  #29 and the design in #9.
• M9. LAYA: Complete resource/handle reuse, grouped uploads, failed graph
  cleanup, bounded workspace caching, and explicit build targets; publish matched
  comparisons with the original runtime. Native inference is merged in Omni #90.
  Related: #66, #39, #71, #72, #73,
  #74, and #100.
• M10. Kev-4B: Implement verified LoRA merging, tokenizer boundaries,
  request compilation, and the trained pointer head using shared Qwen execution.
  Related: Omni #27. #69 is proposed head-kernel work,
  not a complete integration.
• M11. LFM2.5-350M: Complete reference-worker integration and candidate
  branching/cache-isolation checks. Evaluate native execution on a frozen workload;
  larger variants need separate scope and validation.
  Related: Omni #31 and #32.
• M12. Valen: Complete the reference worker and serving recipe, then use
  it to study shared-observation execution.
  Related: Omni #84, #115, and #85.
• M13. Additional candidates: Select owned first integrations for Julia-1,
  AFM-DE, Intern-Decision, or larger variants. Freeze artifacts and establish
  reference behavior before committing to native support. Selection context:
  Omni #9, Agents #25, and Agents #52.
  Intern-Decision has benchmark context, not an Omni integration issue.

For every promoted model, update the supported-model matrix and #83 with its
checkpoint/revision, question types, modalities, worker path, tested hardware,
reference parity, and reproducible measurements. Reference serving, native
execution, and optimized execution are separate milestones.

Feature support

• F1. Shared serving contract: Agree on JSON errors, readiness, overload,
  model discovery, and capabilities; document model-specific confidence and token
  accounting. Related: Omni #61 and the client foundation in Agents #35.
• F2. Bounded admission: Converge pending queue-limit proposals on one
  policy per executor, including capacity, overload responses, cancellation, and
  resource retention after dispatch.
  Related: Omni #35, #82, and #109.
• F3. Request-local observation reuse: Execute compatible prefixes once,
  then branch attention, convolution, and recurrent state correctly. Validate
  changed observations, ordering, compatibility, memory budgets, and parity.
  Related: Omni #85, #97, and #99.
• F4. Processing orchestration and batching: Complete the worker-side
  processing/executor contract; extend within-request packing before bounded
  cross-request batching. Define compatibility, token budgets, maximum wait, and
  output reconstruction; require a batch-capable executor.
  Related foundations: Omni #34, #102, and #111.
  Cross-request batching needs a dedicated follow-up issue.
• F5. Shared vision and CUDA Graphs: Qualify shared vision and
  language/vision graph replay, mixed shapes, changed inputs, capture failures,
  cache budgets, and eager fallback.
  Related: Omni #117, #123, #103, #108, and #106.
• F6. Performance and quality scorecard: Publish parity/accuracy,
  calibration where applicable, p50/p95, requests/s, decisions/s, peak memory,
  startup, and first-inference results. Compare reference and optimized backends
  on matched workloads; retain failures and observed variability.
  Related: Omni #116, #93, #39, and Agents #52.
• F7. Build and contribution path: Compile every declared CUDA backend
  in CI; provide reproducible exports, worker builds, launch recipes, and claimable
  tasks for contracts, kernels, and independent hardware validation.
  Related: Omni #77, #101, and #83.
• F8. Serving provenance and timing: Define checkpoint/backend identity,
  request IDs, and queue/inference/proxy timing across workers and the frontend;
  test identity across restarts and routing. Design context:
  Omni #61 and Agents #35.
  Open a focused issue before extending the common response.
• F9. Additional hardware and precision: Scope native Metal and
  quantization as separate pilots with one model, hardware target, reference,
  numerical gate, and memory/performance criterion each. Native Metal remains
  planned; Python MPS is a foundation, not native Metal support.
  Related context: Omni #9 and #30. Each pilot needs a design issue.

Integration with System1-Agents

• A1. Named served models: Extend served-client plumbing to named
  Omni-backed models with local endpoints, optional authentication, correct model
  labels, and model-specific serializers. Exercise Open-Jev first and CLM when its
  worker is ready. Related: Agents #35, #24, #46, and Omni #87.
  Open a focused issue for the common Omni served adapter.
• A2. Served screenshot decisions: Send browser/desktop images through
  the decision-model interface; validate IDs/probabilities, preserve capture
  identity and window binding, reject stale targets, and keep image bytes out of
  observation logs. Related: Agents #44, #14, #30, and #33.
  Local Cua-S1 in merged #32 does not complete the HTTP screenshot bridge.
• A3. Typed score questions: Add typed score questions/answers,
  validation, serialization, and one rubric-based workflow. Agents currently
  exposes choice and noul. Design context:
  Agents roadmap
  and Omni #61. Open a dedicated Agents issue before implementation.
• A4. Front registration and capabilities: Register served backends in
  applicable CLI, MCP, tool, browser, desktop, and rail paths; reject unsupported
  modalities/question types before inference.
  Related: Agents #18, #35, #46, and #44.
• A5. Operational compatibility: Align readiness, retries, overload,
  deadlines, and confidence thresholds with each worker's semantics. Reuse the
  configurable deadline already merged; verify unavailable, overloaded, invalid,
  and slow-response behavior.
  Related: Omni #61, Agents #36, and #53.
• A6. Episode provenance: Record model/checkpoint, revision, worker
  backend, request ID, and available server timings across supported fronts.
  Verify identity changes and separate capture, decision, and action time.
  Depends on F8; client foundation: Agents #35.
• A7. Joint evaluation and demos: Run pinned Agents/Omni revisions on
  repeatable text, browser, and desktop tasks. Publish task success, every failure,
  inference/episode time, commands, traces, and honestly timed replays. Related:
  Omni #87, Agents #13, #44, and #52.
  Keep public decision-set scores separate from live task completion.

Milestones and dependencies

Models can progress independently once prerequisites are ready. Select the next
integration by contributor ownership and validated readiness rather than promising
every candidate in one release. F9 and unowned M13 candidates stay exploratory
until their first deliverable is scoped.

Track performance against strong baselines, validated model/modality coverage,
the effort to add a model, and independent reproductions. Report variability and
negative results; faster kernels alone do not establish faster or more reliable
agents.