flujo-app · Nov 3, 2026 · Nov 3, 2026 · Nov 3, 2026
diff --git a/.github/workflows/public-sdk.yml b/.github/workflows/public-sdk.yml
@@ -23,7 +23,9 @@ jobs:
           cache: npm
       - run: npm ci
       - run: npm audit --omit=dev --audit-level=high
-      - run: node --test test/cloud-lite.test.mjs test/cloud-lite-provider.test.mjs test/model-provider.test.mjs test/public-sdk.test.mjs test/template-catalog.test.mjs test/factory-team-template.test.mjs test/flujo-workspace.test.mjs test/savia-case-plan.test.mjs test/savia-topology.test.mjs test/managed-fleet.test.mjs test/managed-fleet-real-module.test.mjs test/managed-fleet-provider-closure.test.mjs test/provider-cell-retirement.test.mjs test/presentation.test.mjs test/presentation-modal.test.mjs test/presentation-closure.test.mjs test/observatory-client.test.mjs test/workflows.test.mjs test/workflow-native-wake.test.mjs test/workflow-m0-lifecycle.test.mjs test/workflow-m1-portable-artifacts.test.mjs test/workflow-m2-ack-recovery.test.mjs test/workflow-m2-authority-recovery.test.mjs
+      - run: node --test test/cloud-lite.test.mjs test/cloud-lite-provider.test.mjs test/model-provider.test.mjs test/selfhost-modal.test.mjs test/public-sdk.test.mjs test/template-catalog.test.mjs test/factory-team-template.test.mjs test/flujo-workspace.test.mjs test/savia-case-plan.test.mjs test/savia-topology.test.mjs test/managed-fleet.test.mjs test/managed-fleet-real-module.test.mjs test/managed-fleet-provider-closure.test.mjs test/provider-cell-retirement.test.mjs test/presentation.test.mjs test/presentation-modal.test.mjs test/presentation-closure.test.mjs test/observatory-client.test.mjs test/workflows.test.mjs test/workflow-native-wake.test.mjs test/workflow-m0-lifecycle.test.mjs test/workflow-m1-portable-artifacts.test.mjs test/workflow-m2-ack-recovery.test.mjs test/workflow-m2-authority-recovery.test.mjs
+      - name: Self-hosted model settings (Python, offline)
+        run: python3 -B -m unittest discover -s selfhost/modal -p "test_*.py"
       - name: Install and exercise the packed public package
         shell: bash
         run: |
@@ -34,6 +36,7 @@ jobs:
           node "$RUNNER_TEMP/consumer/node_modules/flujo-factory/bin/factory-public.mjs" --help
           node "$RUNNER_TEMP/consumer/node_modules/flujo-factory/bin/factory-lite.mjs" --help
           node "$RUNNER_TEMP/consumer/node_modules/flujo-factory/bin/factory-evidence.mjs" --help
+          node "$RUNNER_TEMP/consumer/node_modules/flujo-factory/bin/factory-selfhost.mjs" profiles
           FACTORY_TEST_MCP="$RUNNER_TEMP/consumer/node_modules/flujo-factory/bin/factory-mcp.mjs" node --test --test-name-pattern="public MCP stdio" test/public-sdk.test.mjs
       - name: Generic workflow and evidence recovery regressions
         run: node --test test/generic-swarm-evidence.test.mjs
diff --git a/PUBLIC_SDK.md b/PUBLIC_SDK.md
@@ -1,18 +1,21 @@
 # Public local Factory SDK, CLI, and MCP
 
-Version 0.4.0 is distributed as a versioned package in
-[GitHub Releases](https://github.com/flujo-app/FACTORY/releases/tag/v0.4.0):
+Version 0.5.0 is distributed as a versioned package in
+[GitHub Releases](https://github.com/flujo-app/FACTORY/releases/tag/v0.5.0):
 
 ```sh
-npm install https://github.com/flujo-app/FACTORY/releases/download/v0.4.0/flujo-factory-0.4.0.tgz
+npm install https://github.com/flujo-app/FACTORY/releases/download/v0.5.0/flujo-factory-0.5.0.tgz
 ```
 
-0.4.0 lets cloud lite workers run on a self-hosted OpenAI-compatible model,
-clone with a read-only deploy key, prepare agent folders, take operator-driven
-jobs in a narrower sandbox, and record far less trace volume. See
-[Self-hosted models and operator-driven jobs](#self-hosted-models-and-operator-driven-jobs).
-All new plan settings are optional; existing plans behave as before, except
-that streaming deltas are no longer traced unless `traceDeltas` is set.
+FACTORY can serve an open-weight model on your own Modal GPUs and run workers on it.
+`ModalSelfHostedModel` and `factory-selfhost` deploy, verify, stop and retire a vLLM endpoint
+with Qwen3.8-27B (H100) or Nemotron 3.5 Super (B300) profiles; see
+[selfhost/README.md](selfhost/README.md). Its endpoint description is a model provider that
+cloud lite workers (Codex) and local lite workers (Codex or Claude Code) use directly. See
+[Self-hosted models and operator-driven jobs](#self-hosted-models-and-operator-driven-jobs)
+for the provider, read-only deploy keys, prepare hooks, operator-driven jobs and quieter
+traces added in 0.4.0. All of it is optional; existing plans behave as before, except that
+streaming deltas are no longer traced unless `traceDeltas` is set.
 
 `FactoryControl` and `executeEffect` are public exports for integrations that
 retain FACTORY's durable admission while using an existing fleet executor.
@@ -259,6 +262,10 @@ public journal, source commit or trace. Store the fleet directory privately.
 
 #### Self-hosted models and operator-driven jobs
 
+To run the model yourself, `ModalSelfHostedModel` (CLI `factory-selfhost`) deploys a
+bundled vLLM profile on your Modal account and returns this description from `endpoint()`,
+with its key from `key()`; see [selfhost/README.md](selfhost/README.md).
+
 Lite workers can run on a self-hosted, OpenAI-compatible model instead of the
 subscription login. Codex reaches it through the Responses API
 (`/v1/responses`), which vLLM serves for every model. Add `modelProvider` to the

diff --git a/bin/factory-selfhost.mjs b/bin/factory-selfhost.mjs
@@ -0,0 +1,68 @@
+#!/usr/bin/env node
+import { writeFileSync } from 'node:fs';
+import { resolve } from 'node:path';
+import { parseArgs } from 'node:util';
+import { ModalSelfHostedModel, selfHostedProfiles } from '../src/selfhost/modal.mjs';
+
+const usage = `Usage: factory-selfhost <command> [options]
+
+A self-hosted model (vLLM) on your Modal account, owned and journaled in a private folder.
+Uses the active Modal profile and a Python that has the modal package.
+
+  profiles                         list model profiles with list-price estimates
+  deploy   --dir <dir> [--profile qwen|nemotron] [--set key=value ...] [--prefetch] [--no-verify]
+                                   create the key, deploy, and verify the served model
+  status   --dir <dir>             phase, model, GPU and price (no key or URL)
+  endpoint --dir <dir> [--out f]   model-provider description for workers (no key)
+  key      --dir <dir> --out <f>   write the endpoint key to a private file
+  verify   --dir <dir>             check the served model again
+  stop     --dir <dir>             GPU off now; deploy again to resume
+  retire   --dir <dir>             delete the app, secret, volumes and key, and check it
+
+Options: --python <path> (default python3, or python on Windows), --prefix <name> (default factory).
+--set values are JSON when they parse (--set replicas=2 --set variant="fast"), text otherwise.
+Running replicas cost money; idle ones stop after the profile's scale-down time.
+`;
+
+const { values, positionals } = parseArgs({ allowPositionals: true, options: {
+  dir: { type: 'string' }, profile: { type: 'string' }, set: { type: 'string', multiple: true }, prefetch: { type: 'boolean' },
+  'no-verify': { type: 'boolean' }, out: { type: 'string' }, python: { type: 'string' }, prefix: { type: 'string' }, help: { type: 'boolean' } } });
+const [command] = positionals;
+const print = value => process.stdout.write(`${JSON.stringify(value, null, 2)}\n`);
+
+async function main() {
+  if (!command || values.help || command === 'help') return process.stdout.write(usage);
+  if (command === 'profiles') return print(selfHostedProfiles());
+  if (!values.dir) throw new Error('--dir <private folder> is required.');
+  const model = new ModalSelfHostedModel({ directory: resolve(values.dir), ...(values.python ? { python: values.python } : {}), ...(values.prefix ? { namePrefix: values.prefix } : {}) });
+  const controller = new AbortController();
+  process.once('SIGINT', () => controller.abort());
+  const signal = controller.signal;
+  if (command === 'status') return print(model.status());
+  if (command === 'endpoint') {
+    const description = model.endpoint();
+    if (values.out) { writeFileSync(values.out, JSON.stringify(description, null, 2)); return print({ written: values.out }); }
+    return print(description);
+  }
+  if (command === 'key') {
+    if (!values.out) throw new Error('--out <private file> is required; the key is never printed.');
+    writeFileSync(values.out, await model.key(), { mode: 0o600 });
+    return print({ written: values.out });
+  }
+  if (command === 'deploy') {
+    const overrides = {};
+    for (const entry of values.set ?? []) {
+      const at = entry.indexOf('=');
+      if (at < 1) throw new Error(`--set needs key=value: ${entry}`);
+      const raw = entry.slice(at + 1);
+      try { overrides[entry.slice(0, at)] = JSON.parse(raw); } catch { overrides[entry.slice(0, at)] = raw; }
+    }
+    return print(await model.deploy({ profile: values.profile ?? 'qwen', overrides, prefetch: values.prefetch === true, verify: values['no-verify'] !== true, signal }));
+  }
+  if (command === 'verify') return print(await model.verify({ signal }));
+  if (command === 'stop') return print(await model.stop({ signal }));
+  if (command === 'retire') return print(await model.retire({ signal }));
+  throw new Error(`Unknown command: ${command}`);
+}
+
+main().catch(error => { process.stderr.write(`${error.message}\n`); process.exitCode = 1; });
diff --git a/package-lock.json b/package-lock.json
diff --git a/package.json b/package.json
@@ -1,6 +1,6 @@
 {
   "name": "flujo-factory",
-  "version": "0.4.0",
+  "version": "0.5.0",
   "description": "Durable FLUJO swarm engine and Factory SDK, CLI, and MCP server",
   "license": "MIT",
   "repository": {
@@ -19,12 +19,18 @@
     "bin/factory-lite.mjs",
     "bin/factory-cloud-mcp.mjs",
     "bin/factory-cloud-runtime.mjs",
+    "bin/factory-selfhost.mjs",
     "bin/factory.mjs",
     "scripts/watch-pilot.mjs",
     "scripts/pilot.mjs",
     "scripts/run-tests.mjs",
     "scripts/local-flujo-live-smoke.mjs",
     "scripts/savia-local-live-smoke.mjs",
+    "selfhost/README.md",
+    "selfhost/modal/*.py",
+    "selfhost/modal/prices.json",
+    "selfhost/modal/profiles/*.json",
+    "!selfhost/modal/test_*.py",
     "README.md",
     "PUBLIC_SDK.md",
     "WORKFLOWS.md",
@@ -45,7 +51,8 @@
     "factory-evidence": "./bin/factory-evidence.mjs",
     "factory": "./bin/factory-public.mjs",
     "factory-mcp": "./bin/factory-mcp.mjs",
-    "factory-lite": "./bin/factory-lite.mjs"
+    "factory-lite": "./bin/factory-lite.mjs",
+    "factory-selfhost": "./bin/factory-selfhost.mjs"
   },
   "type": "module",
   "engines": {

diff --git a/scripts/verify-public-sdk.mjs b/scripts/verify-public-sdk.mjs
@@ -98,3 +98,10 @@ const endpoint = { ...providerPlan.modelProvider, model: 'served-model', apis: [
 assert.deepEqual(sdk.providerPrograms(endpoint), ['codex', 'claude']);
 assert.match(sdk.codexProviderConfig(endpoint), /wire_api = "responses"/);
 assert.equal(sdk.claudeProviderEnvironment(endpoint, { key: 'package-smoke-key' }).ANTHROPIC_BASE_URL, 'https://model.example');
+// Self-hosted model serving ships its Python sources and profiles, but not its tests.
+assert.deepEqual(sdk.selfHostedProfiles().map(profile => profile.id), ['nemotron', 'qwen']);
+for (const file of ['app.py', 'settings.py', 'discovery.py', 'deploy.py', 'prices.json', 'profiles/qwen.json', 'profiles/nemotron.json']) {
+  assert.equal(existsSync(join(sdk.SELFHOST_MODAL_DIR, file)), true, `${file} is missing`);
+}
+assert.equal(existsSync(join(sdk.SELFHOST_MODAL_DIR, 'test_settings.py')), false);
+assert.equal(typeof sdk.ModalSelfHostedModel, 'function');
diff --git a/selfhost/README.md b/selfhost/README.md
@@ -0,0 +1,104 @@
+# Self-hosted models on Modal
+
+FACTORY can serve an open-weight model on your own Modal GPUs with vLLM, so workers run on it
+instead of a vendor API: source code and prompts stay in your account. The served endpoint
+speaks the OpenAI Responses API (Codex), the Anthropic Messages API (Claude Code) and Chat
+Completions, and plugs straight into `plan.modelProvider` and the local lite workers.
+
+This folder is separate from `modal/`, which holds the earlier pilot definitions.
+
+## Profiles
+
+| Profile | Model (served name) | GPU | Price while running | Context | Notes |
+|---|---|---|---|---|---|
+| `qwen` (default) | Qwen3.8-27B FP8 (`qwen3.8-27b`) | 1 × H100 80 GB | ~$4.84/h | 262,144 | vLLM 0.30.0; parsers `qwen3_xml` / `qwen3`; thinking on |
+| `nemotron` | Nemotron 3.5 Super VL 120B-A12B NVFP4 (`nemotron-3.5-super-vl`) | 1 × B300 288 GB | ~$8.43/h | 262,144 | vLLM 0.32.0 release image (pinned by digest); parsers `qwen3_coder` / `nemotron_v3`; text only |
+
+Prices are Modal list prices (GPU plus the reserved CPU and memory, `prices.json`, checked
+2026-11-01), per running replica. They are estimates, not invoices; storage, egress and image
+builds are extra. A replica with no requests stops after the profile's scale-down time (10 or
+8 minutes) and costs nothing while stopped. Each replica is its own web function with at most
+one GPU container (`replicas` 1–4), so the GPU count cannot grow by itself.
+
+The Nemotron profile has named flag sets (`variant`), all checked against vLLM 0.32.0, the
+current stable release:
+
+| Variant | Adds | Use |
+|---|---|---|
+| `base` | FP32 Mamba state | reference |
+| `fast` | async scheduling, FlashInfer Mamba kernel | 128–200 long conversations per GPU, full precision |
+| `mtp1` (default) | `fast` + multi-token prediction, 1 draft token | up to ~128 conversations; each agent much faster |
+| `mtp3` | `fast` + 3 draft tokens | up to ~32 conversations |
+| `mtp1-fp16` | `mtp1` + FP16 Mamba state with stochastic rounding (lower precision) | MTP speed at ~190 conversations |
+
+All Nemotron variants use prefix caching for the hybrid model (`--mamba-cache-mode align`,
+`--enable-mamba-shared-prefix-checkpoint` with MTP) and load the language model only.
+
+## Measured (long agent conversations, 300-token answers, histories 11k–100k tokens)
+
+| Setup | Comfortable agents (turn ≤ 10 s, cache hits ≥ 85%) | Each agent | $ per agent-hour |
+|---|---|---|---|
+| Qwen, 1 × H100 | 8 | 39 tok/s | 0.61 |
+| Nemotron `mtp1`, 1 × B300 | 64 (128 busy) | 57 tok/s (32 busy) | 0.13 (0.066 busy) |
+| Nemotron `fast` / `mtp1-fp16`, 1 × B300 | 192 busy | 14 / 23 tok/s | 0.044 |
+
+On one H100 the Qwen model's GPU cache (about 660k tokens) cannot hold more than about ten such
+histories: cache hits fall from about 90% to 15% and answers wait 40–140 s. The B300 holds
+23 million cache tokens, about 35 times more. In a real 190-agent coding run (10 lite-worker
+machines × 19 Codex sessions, `fast`), one B300 carried about 170 requests at once with none
+queued. The GPU cache was at most 37% full and 84% of 464 million input tokens came from the
+prompt cache. The run wrote 12.5 million output tokens in 2 h 18 min for about $20 of GPU time.
+Cold starts: Qwen 4–5 min (first 7.5 min); Nemotron about 9.5 min (weights from the volume plus
+kernel compilation).
+
+## Use it
+
+Needs Python 3.10+ with `pip install modal` and a signed-in Modal profile
+(`modal profile current`). The private folder holds the journal and, by default, the key; keep
+it out of Git.
+
+```sh
+factory-selfhost profiles
+factory-selfhost deploy --dir ./private/model --profile nemotron --prefetch   # CPU weight download first, then deploy and verify
+factory-selfhost endpoint --dir ./private/model --out endpoint.json            # provider description, no key
+factory-selfhost key --dir ./private/model --out ./private/model-key           # backend only
+factory-selfhost stop --dir ./private/model                                    # GPU off now; deploy again to resume
+factory-selfhost retire --dir ./private/model                                  # delete app, secret, volumes and key
+```
+
+```js
+import { ModalSelfHostedModel, FactoryCloudLiteFleet } from 'flujo-factory';
+
+const model = new ModalSelfHostedModel({ directory: '/absolute/private/model' });
+await model.deploy({ profile: 'qwen', overrides: { replicas: 2 } });
+const modelProvider = model.endpoint();              // validated provider description
+await fleet.deploy({ ...plan, model: modelProvider.model, subagentModel: modelProvider.model, modelProvider },
+  { image, bundlePath, providerKey: await model.key() });
+// ...
+await model.retire();
+```
+
+`deploy` creates an owned app name (`<prefix>-<profile>-<12 hex>`), checks the active Modal
+profile and workspace, and generates a fresh key that only Modal's secret and your credential
+store hold. It then deploys and proves each replica serves the expected model: the catalogue
+lists it, and one non-thinking completion returns an exact challenge. Modal's 303 result links
+are followed on the same origin only, so the POST is never repeated and the key never goes to
+another host. Any uncertain remote outcome leaves the model `unknown`. It is never retried
+automatically; `retire` reconciles it. `stop` and `retire` refuse a different Modal workspace,
+and `retire` checks that the secret and both volumes are gone.
+`adopt({ appName, profileName, workspaceName })` takes over an owned app that an earlier
+version of your app created, in the `unknown` phase, so `retire` can clean it up.
+
+Overrides are explicit settings fields, for example `{ replicas: 2 }`, `{ variant: 'fast' }`,
+`{ enable_thinking: false }` or `{ scaledown_minutes: 2 }`. Unknown fields, unpinned images and
+settings that would not fit the GPU are refused before anything is created.
+
+## Things to know
+
+- vLLM's `--api-key` protects `/v1/*`; `/health` and `/metrics` answer without it (counters only).
+- A request to a cold replica is held, not refused, until the container is up.
+- Qwen3.8's chat template accepts reasoning efforts `low`, `medium` and `xhigh` and rejects
+  `high`. Nemotron's template ignores the effort; `enable_thinking` switches thinking.
+- Spread workers over replicas (`baseUrls[i % n]`), so each worker keeps its prompt cache on one GPU.
+- Offline checks: `python -B -m unittest discover -s selfhost/modal -p "test_*.py"` and
+  `node --test test/selfhost-modal.test.mjs`. Neither makes a cloud call.