Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,63 @@
# Changelog

## v2.7.0

Routing, concurrency and cost. The factory now picks a model per issue type and stage, shares
one machine's slots and daily budget across every repo it watches, and can prove non-test work
(a blog post, a docs page) without ever writing a failing test.

- **`routes` in `.factory/config.json`.** Each type label (`bug`, `feature`, `docs`, `security`,
`dependency`, or one a repo adds, e.g. `content`) can name its own agent per stage and its own
routed skills. `resolveRoute(agents, stages, routes, stage, type)` resolves
`routes[type].stages[stage]` → `stages[stage]` → `stages.default` → `"claude"`; triage always
uses `stages.triage`, since the type isn't known yet. `template/.factory/config.example.json`
ships the recommended default: triage on Haiku, plan on Opus, build and verify on Sonnet, PR
on Haiku, with `docs` builds on Haiku. With no config, the Claude CLI's own default model still
applies.
- **`src/context.ts`: a context pack, built once per stage.** `buildContextPack(stage, issue, cwd,
skills)` assembles AGENTS.md, the route's skills, `ARCHITECTURE.md` if the repo has one, and the
files named in `plan.json`, capped at 64 KiB; whatever doesn't fit is named, never truncated
mid-file. Claude gets it appended after `STAGE_GUIDANCE` on the same `--append-system-prompt`;
every other agent gets it inlined ahead of the artifact contract in its rendered prompt.
- **Pricing for the current models.** `claude-sonnet-5` and `claude-opus-5-5` price correctly
instead of costing NULL; `research/pricing/anthropic.md` captures the source table these prices
(and the already-priced `claude-haiku-4-5`) come from.
- **Concurrency belongs to the machine, not the repo.** `src/machine.ts` reads slots from
`FACTORY_SLOTS`, or `FACTORY_HOME/machine.json` (`{ "slots": N, "dailyUsd": N }`), validated at
boot. A repo's own `concurrency` becomes a cap on top of that: the limit is whichever is
smaller. Slots are leases in `machine.db` (`BEGIN IMMEDIATE`), reclaimed when a holder's pid has
died, so two watchers on one machine (splitbill-demo and lwp-website) share one pool instead of
each assuming the whole machine. `factory doctor` prints a suggested slot count from cores and
free memory, as advice only.
- **Continuous dispatch.** The watcher no longer waits for the whole pool to drain before polling
again; it keeps polling while jobs run and picks up new work the moment a slot frees, tracking
in-flight issues so nothing double-starts.
- **Spend caps.** `spend.perIssueUsd` and `dailyUsd` (repo and machine) are checked before every
stage, against the running sum of `stage_runs.cost_usd`. Over a cap, the issue parks with a
"budget" inbox item and intake pauses while the daily cap is hit; a stage with unknown cost
counts against a separate token cap so an unpriced model can't spend past every limit for free.
- **`proof: "test" | "check"` on the plan.** `test` (the default) keeps today's rule: build writes
a failing test first, verify checks it catches the bug. `check` names the commands that prove
each acceptance criterion instead: build runs them, verify re-runs them and judges the diff, and
never asks for a test. Both skills read `proof` off the plan artifact, never off the type name, so
a repo can declare work `proof: check` under any type, including one it added itself. The type
list is config-driven everywhere now; `TYPE_LABELS` is the default, not the only list.

Structural tests: every `(type, stage)` pair in `routes` and `TYPE_LABELS` resolves to a defined
agent; `loadConfig` refuses at boot when a routed skill has no `SKILL.md` on disk, instead of
no-opping mid-run in the context pack; a `proof: check` plan (Markdown files, a prose-lint gate,
no test file) ships exactly like a `proof: test` plan; the app-agnostic grep now covers every
file under `template/.claude/skills` and `template/.claude/agents` for a named stack tool
(`npm`, `bun`, `pytest`, `cargo`, `go test`), since those are the files an agent actually reads at
runtime, unlike `gates.sh` or the CI workflow, which are allowed to be stack-specific; a property
test holds running jobs to the machine's slots across two watcher processes and confirms a freed
slot is picked up within one poll.

Not in v2.7.0:
- The lwp-website pilot itself, `merge.policy`, and anything that merges (v2.8).
- A live agent run against splitbill-demo exercising the new routes and concurrency: pending your
reset OK.

## v2.6.2

Any repo, safely. No new features: this closes the gaps a second repo (splitbill-demo plus
Expand Down
13 changes: 10 additions & 3 deletions bin/factory
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import { plain } from "../src/display";
import { act, buildInbox, inboxPositionals, type InboxAction } from "../src/inbox";
import { reset, rebaseline } from "../src/reset";
import { runDoctor, fixDoctor } from "../src/doctor";
import { loadMachineConfig, MachineLeases, MachineSpend } from "../src/machine";
import { createDashboard } from "../dashboard/server";
import { versionOf, which } from "../src/probes";
import { workspacesDir as defaultWorkspacesDir, defaultStatePath } from "../src/paths";
Expand Down Expand Up @@ -77,12 +78,16 @@ function buildWatchDeps(cloneDir: string, config: FactoryConfig): WatchDeps {
github: new GitHub(),
git: new Git(new GitCommandRunner()),
state: new FactoryState(flag("db") ?? process.env.FACTORY_DB_PATH ?? defaultStatePath(process.env, config.repo)),
executor: new CommandExecutor(config.agents, config.stages),
executor: new CommandExecutor(config.agents, config.stages, config.routes),
gateRunner: new ShellGateRunner(),
rechecker: verifierIsClaude(config) ? new ClaudeRechecker(new BinaryRunner("claude"), cloneDir) : undefined,
cloneDir,
workspacesDir: flag("workspaces") ?? defaultWorkspacesDir(process.env, config.repo),
setupRunner: new ShellSetupRunner(),
// Real cross-process leases: every real CLI invocation (watch, run, tick)
// shares this machine's FACTORY_HOME/machine.db, so a second repo's
// watcher on the same machine is respected (plan v2.7.0 item 5).
machine: { leases: new MachineLeases(), config: loadMachineConfig(), spend: new MachineSpend() },
};
}

Expand Down Expand Up @@ -306,17 +311,19 @@ async function cmdDoctor(): Promise<void> {
factoryMode: process.env.FACTORY_MODE,
agents: config.agents,
stages: config.stages,
routes: config.routes,
templateSkills: templateSkills(),
legacyStatePath: defaultStatePath(process.env),
legacyWorkspacesDir: defaultWorkspacesDir(process.env),
configuredSlots: loadMachineConfig().slots,
},
);
const allOk = checks.every((c) => c.ok || c.warn);
if (has("json")) console.log(successJson({ checks }, allOk));
else for (const c of checks) console.log(` [${c.ok ? "ok" : c.warn ? "warn" : "FAIL"}] ${c.name} — ${c.detail}`);
if (!allOk && has("fix")) {
console.error("factory doctor --fix: creating missing labels");
await fixDoctor(github, config.repo);
await fixDoctor(github, config.repo, config.routes);
}
if (!allOk && !has("fix")) process.exit(EXIT.checksFailed);
}
Expand Down Expand Up @@ -382,7 +389,7 @@ async function cmdVerifyAgent(): Promise<void> {
const loaded = await loadConfig(cloneDir);
const config: FactoryConfig = { ...loaded, ...configFor(name) };
const out = resolve(flag("out") ?? `tests/fixtures/agents/${name}`);
const deps = { ...buildWatchDeps(cloneDir, config), executor: new CommandExecutor(config.agents, config.stages, new FixtureRecorder(out, name)) };
const deps = { ...buildWatchDeps(cloneDir, config), executor: new CommandExecutor(config.agents, config.stages, config.routes, new FixtureRecorder(out, name)) };
const outcome = await advanceIssue(deps, config, await deps.github.getIssue(config.repo, issueNumber));
if (outcome === "awaiting-approval") {
console.log(`Plan gate: read the plan on #${issueNumber}, approve it (/factory approve), then run this command again to finish and record the rest.`);
Expand Down
5 changes: 5 additions & 0 deletions dashboard/analytics.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ export interface Analytics {
readonly spend30dUsd: number;
readonly byStage: readonly Bucket[];
readonly byAgent: readonly Bucket[];
readonly byModel: readonly Bucket[];
}

const FINISHED = new Set(["shipped", "failed", "rejected", "cancelled", "needs-human"]);
Expand Down Expand Up @@ -56,5 +57,9 @@ export function analytics(runs: readonly Run[], stageRuns: readonly StageRun[],
spend30dUsd: spendSince(30),
byStage: bucketBy(stageRuns, (r) => r.stage),
byAgent: bucketBy(stageRuns, (r) => r.agent),
// A run with no price on record (no agent-reported total, no pricing.ts
// entry) is still counted, under "unknown" — never dropped silently
// (plan v2.7.0 item 7: spend per model, visible even when incomplete).
byModel: bucketBy(stageRuns, (r) => r.model ?? "unknown"),
};
}
2 changes: 1 addition & 1 deletion dashboard/public/app.js
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ function analyticsView() {
h("div", { class: "kpis" },
[[a.successRate == null ? "None yet" : `${Math.round(a.successRate * 1000) / 10}%`, "Runs that shipped, of those finished"], [String(a.runs), "Runs started"], [money(a.spend7dUsd), "Spent in 7 days"], [money(a.spend30dUsd), "Spent in 30 days"]]
.map(([v, l]) => h("div", { class: "kpi" }, h("span", { class: "value" }, v), h("span", { class: "label" }, l)))),
bars("Cost by stage", a.byStage), bars("Cost by agent", a.byAgent))));
bars("Cost by stage", a.byStage), bars("Cost by agent", a.byAgent), bars("Cost by model", a.byModel))));
}

/* ---- Agents ---- */
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "software-factory",
"version": "2.6.2",
"version": "2.7.0",
"private": true,
"type": "module",
"description": "GitHub-native SDLC loop for coding agents: triage, plan, build, verify, PR.",
Expand Down
20 changes: 20 additions & 0 deletions research/pricing/anthropic.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Anthropic pricing capture

Source: https://www.anthropic.com/pricing (redirects to https://claude.com/pricing).
Captured: 2026-09-27, via WebFetch.

USD per million tokens.

| Model | Input | Output | Cache read | Cache write |
|---|---|---|---|---|
| Opus 5.5 | 4 | 20 | 0.20 | 5 |
| Sonnet 5 | 2 | 10 | 0.20 | 2.50 |
| Haiku 4.5 | 1 | 5 | 0.10 | 1.25 |
| Fable 5.1 | 10 | 50 | 0.25 | 12.50 |
| Opus 5 / Opus 4.8 / 4.7 / 4.6 / 4.5 (legacy) | 5 | 25 | 0.50 | 6.25 |
| Sonnet 4.6 / 4.5 (legacy) | 3 | 15 | 0.30 | 3.75 |
| Fable 5 (legacy) | 10 | 50 | 1 | 12.50 |

`src/pricing.ts` already carries `claude-haiku-4-5`, `claude-sonnet-4-5`, `claude-opus-4-5` under
these exact numbers (they match this capture; no change needed there). This capture adds
`claude-opus-5-5` and `claude-sonnet-5`, which were previously unpriced (NULL cost).
35 changes: 30 additions & 5 deletions src/agents/executor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,15 @@ import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { runDir } from "../artifacts";
import type { RouteConfig } from "../config";
import { buildContextPack } from "../context";
import { aggregateStageEvents, type Executor, type StageEvent, type StageRunOptions, type StageRunResult } from "../executor";
import { sanitizeEnv } from "./env";
import { renderPrompt } from "./prompt";
import { PRESETS } from "./presets";
import { structuredCommand } from "./structured";
import { replySchema } from "../schemas";
import { typesFor } from "../labels";
import { writeReply } from "./reply";
import type { FixtureRecorder } from "./record";
import { type AgentConfig, type AgentPreset, type StageAgents, stagePolicy } from "./types";
Expand All @@ -30,8 +33,26 @@ export interface ResolvedAgent {
readonly preset?: AgentPreset;
}

// `routes[type].stages[stage]`, then `stages[stage]`, then `stages.default`,
// then "claude" (plan v2.7.0 item 1). Triage runs before the type is known,
// so it always uses `stages.triage` regardless of what `type` is passed.
export function resolveRoute(
agents: Record<string, AgentConfig>,
stages: StageAgents,
routes: Readonly<Record<string, RouteConfig>> | undefined,
stage: keyof StageAgents & string,
type: string | undefined,
): ResolvedAgent {
const routed = stage !== "triage" && type ? routes?.[type]?.stages?.[stage] : undefined;
const name = routed ?? stages[stage] ?? stages.default ?? "claude";
return resolveAgentByName(agents, name, stage);
}

export function resolveAgent(agents: Record<string, AgentConfig>, stages: StageAgents, stage: keyof StageAgents & string): ResolvedAgent {
const name = stages[stage] ?? stages.default ?? "claude";
return resolveRoute(agents, stages, undefined, stage, undefined);
}

function resolveAgentByName(agents: Record<string, AgentConfig>, name: string, stage: string): ResolvedAgent {
const config = agents[name];
if (!config) throw new Error(`stage ${stage} uses agent "${name}", which is not in config.agents`);
const preset = config.preset ? PRESETS[config.preset] : undefined;
Expand All @@ -58,11 +79,12 @@ export class CommandExecutor implements Executor {
constructor(
private readonly agents: Record<string, AgentConfig>,
private readonly stages: StageAgents,
private readonly routes: Readonly<Record<string, RouteConfig>> = {},
private readonly recorder?: FixtureRecorder,
) {}

async runStage(opts: StageRunOptions): Promise<StageRunResult> {
const agent = resolveAgent(this.agents, this.stages, opts.stage);
const agent = resolveRoute(this.agents, this.stages, this.routes, opts.stage, opts.type);
const scratch = mkdtempSync(join(tmpdir(), `factory-scratch-${opts.issue}-`));
try {
return await this.spawnStage(opts, agent, scratch);
Expand All @@ -79,10 +101,13 @@ export class CommandExecutor implements Executor {
const readOnly = !stagePolicy(opts.stage).write && agent.preset?.returnsArtifact === true && !agent.config.command;
let argv: readonly string[];
let stdin: string | undefined;
const types = typesFor(this.routes);
const skills = opts.type ? this.routes[opts.type]?.skills ?? [] : [];
const contextPack = await buildContextPack(opts.stage, opts.issue, opts.cwd, skills);
if (agent.preset?.ownsPrompt) {
({ argv, stdin } = agent.preset.command(opts, agent.config, ""));
({ argv, stdin } = agent.preset.command(opts, agent.config, contextPack));
} else {
const prompt = await renderPrompt(opts, readOnly);
const prompt = await renderPrompt(opts, readOnly, types, contextPack);
if (agent.config.command) {
const promptFile = join(scratch, "prompt.md");
writeFileSync(promptFile, prompt);
Expand All @@ -93,7 +118,7 @@ export class CommandExecutor implements Executor {
let schemaFile: string | undefined;
if (readOnly && agent.config.outputSchema) {
schemaFile = join(scratch, "reply.schema.json");
writeFileSync(schemaFile, JSON.stringify(replySchema(opts.stage)));
writeFileSync(schemaFile, JSON.stringify(replySchema(opts.stage, types)));
}
({ argv, stdin } = agent.preset.command(opts, agent.config, prompt, { schemaFile }));
} else {
Expand Down
12 changes: 9 additions & 3 deletions src/agents/presets/claude.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,11 @@ export function parseStreamJsonLine(line: string): StageEvent[] {
return events;
}

export function claudeArgs(opts: StageRunOptions): string[] {
// `contextPack` (buildContextPack, plan v2.7.0 item 3) rides the same
// --append-system-prompt as STAGE_GUIDANCE, appended after it. --help gives
// no guarantee that repeating the flag concatenates rather than overriding,
// so this sends one flag with one value rather than risk losing STAGE_GUIDANCE.
export function claudeArgs(opts: StageRunOptions, contextPack = ""): string[] {
return [
"-p",
`/factory-${opts.stage} ${opts.issue}`,
Expand All @@ -87,7 +91,7 @@ export function claudeArgs(opts: StageRunOptions): string[] {
"--settings",
stageSettings(opts.stage, opts.issue, opts.agentCommands),
"--append-system-prompt",
STAGE_GUIDANCE,
contextPack ? `${STAGE_GUIDANCE}\n\n${contextPack}` : STAGE_GUIDANCE,
"--no-session-persistence",
"--max-budget-usd",
String(opts.maxBudgetUsd),
Expand All @@ -104,7 +108,9 @@ export const claudePreset: AgentPreset = {
skillsDir: ".claude/skills",
contextFile: "CLAUDE.md",
ownsPrompt: true,
command: (opts, agent) => ({ argv: ["claude", ...claudeArgs(opts), ...(agent.model ? ["--model", agent.model] : [])] }),
// `prompt` is the context pack for an ownsPrompt agent: Claude runs its own
// skill and ignores everything else spawnStage would otherwise put there.
command: (opts, agent, prompt) => ({ argv: ["claude", ...claudeArgs(opts, prompt), ...(agent.model ? ["--model", agent.model] : [])] }),
parseLine: parseStreamJsonLine,
isUsageCandidate: (line) => isUsageResultCandidate(line, "result"),
};
16 changes: 11 additions & 5 deletions src/agents/prompt.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,17 @@ export function stripFrontmatter(text: string): string {
return text.startsWith("---\n") ? text.slice(text.indexOf("\n---", 3) + 4).replace(/^\n+/, "") : text;
}

export function artifactContract(opts: Pick<StageRunOptions, "stage" | "issue">, readOnly = false): string {
// `types` bounds the triage schema's `type` enum to the repo's actual type
// list (typesFor(config.routes)); omitted, it falls back to TYPE_LABELS.
export function artifactContract(opts: Pick<StageRunOptions, "stage" | "issue">, readOnly = false, types?: readonly string[]): string {
if (readOnly) {
return [
"## Artifact contract",
"",
"This stage is read-only: you cannot write files. Do not try. Your final message must be ONE JSON object and nothing else, with no code fence:",
`{"artifact": <the structured result, exactly as the instructions above describe for ${JSON_FILENAMES[opts.stage]}>, "comment": "<the comment the runner posts on the issue, in markdown>", "question": "<only when you cannot proceed without a human answer>"}`,
"",
`The artifact must match this JSON Schema: ${JSON.stringify(stageSchema(opts.stage))}`,
`The artifact must match this JSON Schema: ${JSON.stringify(stageSchema(opts.stage, types))}`,
"",
"You have no GitHub access and cannot push or merge; the runner does that.",
].join("\n");
Expand All @@ -29,15 +31,18 @@ export function artifactContract(opts: Pick<StageRunOptions, "stage" | "issue">,
"## Artifact contract",
"",
`You run in the issue's worktree. Write your results as files in $FACTORY_ARTIFACT_DIR (${runDir(opts.issue)}/):`,
`- ${JSON_FILENAMES[opts.stage]}: the structured result, exactly as the instructions above describe. Schema: ${JSON.stringify(stageSchema(opts.stage))}`,
`- ${JSON_FILENAMES[opts.stage]}: the structured result, exactly as the instructions above describe. Schema: ${JSON.stringify(stageSchema(opts.stage, types))}`,
`- ${COMMENT_FILENAMES[opts.stage]}: the comment the runner posts on the issue.`,
"- question-comment.md: only when you cannot proceed without an answer from a human.",
"",
"You have no GitHub access and cannot push or merge; the runner does that. Finish by exiting.",
].join("\n");
}

export async function renderPrompt(opts: StageRunOptions, readOnly = false): Promise<string> {
// `contextPack` (buildContextPack, plan v2.7.0 item 3) is inlined here for
// every non-Claude agent; Claude gets the same text through
// --append-system-prompt instead (see agents/presets/claude.ts).
export async function renderPrompt(opts: StageRunOptions, readOnly = false, types?: readonly string[], contextPack = ""): Promise<string> {
const skill = await readFile(`${opts.cwd}/.claude/skills/factory-${opts.stage}/SKILL.md`, "utf8").catch(() => {
throw new Error(`stage skill .claude/skills/factory-${opts.stage}/SKILL.md not found in ${opts.cwd}; run \`factory install --update\``);
});
Expand All @@ -46,7 +51,8 @@ export async function renderPrompt(opts: StageRunOptions, readOnly = false): Pro
"",
stripFrontmatter(skill).trim(),
"",
artifactContract(opts, readOnly),
...(contextPack ? [contextPack, ""] : []),
artifactContract(opts, readOnly, types),
"",
].join("\n");
}
Loading
Loading