[1992 by GPT-5.6 Sol] Add Web UI, CLI, playground, observability, and documentation for synthetic pools

docs/docs/concepts/glossary.md

@@ -31,6 +31,18 @@ title: Glossary
 
 **Task** — one unit of agent work with its own record: prompt, isolated run, logs, usage, commits, and resulting PR or follow-up.
 
+**Synthetic agent** — a provider-neutral virtual agent whose models route to configured direct agent/model members. See [Synthetic Pools](../features/synthetic-pools.md).
+
+**Synthetic model** — a virtual model ID exposed by a synthetic agent in normal model selectors.
+
+**Pool member** — one direct-agent alias and supported physical model participating in a synthetic model.
+
+**Priority tier** — all eligible pool members at one priority; only the highest currently eligible tier participates in selection.
+
+**Usage cap** — an optional session or weekly usage percentage above which a capped pool member becomes ineligible.
+
+**Failover** — retrying the same call and workspace on another eligible pool member after a retryable physical failure.
+
 **Ultrafix** — the automated review-fix loop: `/review` scores the PR, fixes are applied, and cycles repeat until the target score, cycle limit, or a human stop. See [PR Comment Commands](../features/pr-commands.md#ultrafix).
 
 **Worktree** — the dedicated Git working directory each task gets, paired with its own branch and container, so parallel tasks never collide and the main checkout stays untouched.

docs/docs/features/agents-and-models.md

@@ -21,6 +21,8 @@ Use routing when you want to:
 - Fall back to another provider when rate limits or quota are tight
 - Preserve the same PR follow-up workflow across providers
 
+For virtual routing across several configured direct agents, see [Synthetic Pools](./synthetic-pools.md).
+
 ## Supported Agents
 
 | Agent | Type | Docker image | Existing host credentials |

docs/docs/features/propr-cli.md

@@ -268,12 +268,18 @@ propr agent add --file agent-config.json     # From a JSON file (or `-` for stdi
 propr agent enable my-agent                  # Enable / disable without deleting
 propr agent disable my-agent
 propr agent delete my-agent --force
+
+propr agent pool list --json > pools.json
+propr agent pool apply pools.json       # Also accepts '-' for stdin
+propr agent pool delete balanced-pool
 ```
 
 Agent types: `claude`, `codex`, `antigravity`, `opencode`, `vibe`.
 
 See [Agents and Models](./agents-and-models.md) for the model catalog, label formats, and per-agent credential setup, including the OpenCode host-authentication steps and the `XDG_DATA_HOME` requirement for file-based OpenCode auth.
 
+Synthetic pool commands replace one complete, nested configuration document. JSON from `pool list --json` can be passed unchanged to `pool apply`; validation failures retain the backend's nested field message. See [Synthetic Pools](./synthetic-pools.md) for schemas and routing behavior.
+
 ## To-Dos
 
 ```bash

docs/docs/features/synthetic-pools.md

@@ -0,0 +1,114 @@
+---
+title: Synthetic Pools
+---
+
+# Synthetic Pools
+
+Synthetic pools give a stable virtual agent/model identity to a set of existing direct agent accounts. They are useful for rotating between two accounts from one provider, balancing capacity, or failing over to a different provider without changing repository, planner, review, or issue configuration.
+
+## Concepts
+
+- A **synthetic agent** is a virtual coding agent. It has an alias and one or more synthetic models but no provider credentials of its own.
+- A **synthetic model** is a virtual model ID exposed in ProPR's instance catalog and model selectors.
+- A **pool member** is one direct-agent alias and one physical model supported by that direct agent. A synthetic agent can never be a member of another pool.
+- A **priority tier** is the set of currently eligible members with the same priority, from 0 through 100. Routing considers only the highest eligible tier.
+- A **usage cap** makes a member ineligible when its current session or weekly usage reaches a configured percentage.
+- **Failover** retries a synthetic call on another eligible member after a retryable physical failure.
+
+Synthetic choices use a neutral layers icon in the UI because the pool is not owned by a provider. Task lists keep their model column concise by showing the virtual model. Playground results, task details, task-history attempts, and LLM logs also show the physical agent/model that actually ran.
+
+## Configure in the Web UI
+
+Installation administrators can open **Coding Agents → Synthetic Pools** to create, edit, enable, disable, or delete pools. Each virtual model supports **Round robin** or **Usage based** routing, an enabled state, and one or more direct members. Each member has an enabled state, priority, and optional session and weekly maximum percentages.
+
+The member picker contains only configured direct agents and their supported physical models. Disabled direct agents remain visible for correcting existing configuration but are not eligible at runtime. Demo mode is read-only and disables every mutation.
+
+Backend validation is authoritative. A rejected save keeps the editor and unsaved values open and associates a validation message with its nested model/member field when the response contains a field path.
+
+### Same-provider round robin
+
+Create two direct Codex agents, such as `codex-account-a` and `codex-account-b`, using separate credential directories. Add both with the same physical model to one enabled virtual model, give both priority 100, and choose **Round robin**. Successful calls rotate between the two accounts using a cursor shared by the workers.
+
+### Usage-based selection
+
+**Usage based** still honors strict priority first. Within the highest eligible tier it selects the member with the most normalized headroom below its configured caps. If no caps are configured, all members have equal headroom; use round robin when deterministic rotation is the goal.
+
+## Primary and fallback recipe
+
+For cross-provider primary/fallback routing:
+
+1. Add the primary member at priority 100.
+2. Optionally set its weekly maximum to 80%.
+3. Add the fallback member at priority 0.
+4. Use either strategy; strategy only chooses among members inside the selected priority tier.
+
+The priority-0 member is not mixed into normal traffic. It becomes eligible for selection only when every higher-priority member is disabled, capped, unavailable, too small for the call's context, or has failed during that call. This priority-100 primary plus priority-0 fallback pattern is the recommended way to reserve fallback capacity.
+
+## Context-aware early selection
+
+ProPR can select a route early so planning and task setup retain one stable physical choice. Before the first physical invocation it finalizes the required prompt plus output reserve. If the selected model's context limit is too small, ProPR reselects without counting that member as a failed attempt.
+
+Every later failover applies the same context requirement. A smaller-context fallback can therefore be skipped even when it is healthy: sending a prompt that cannot fit would only create a misleading provider failure.
+
+## Usage data and degraded pools
+
+A capped member requires fresh Agent Tank data whose name exactly matches the direct-agent alias. Missing, refreshing, stale, provider-wide-only, or differently named data makes that capped member ineligible. The default freshness window is five minutes and can be changed with `SYNTHETIC_USAGE_FRESHNESS_MS`.
+
+Uncapped pools do not require Agent Tank. If no member of a synthetic model is currently eligible, the pool reports **Degraded**. This does not mark its unrelated direct agents unhealthy; direct-agent health remains independent.
+
+## Failure retries and workspace preservation
+
+A retryable physical error fails over to the next eligible, not-yet-attempted member. Every physical attempt is recorded as a separate history entry with the virtual identity, physical agent/model, attempt number, and selection reason. These attempts remain part of one task: ProPR does not create extra tasks or extra worktrees.
+
+Implementation retries reuse the same task workspace and branch, so edits made before a provider failure remain available to the fallback. Explicit user cancellation, security-policy failures, invalid configuration, and prompts that exceed the context limit are not retried on another member.
+
+## CLI
+
+The CLI manages the same complete configuration document:
+
+```bash
+propr agent pool list
+propr agent pool list --json > pools.json
+propr agent pool apply pools.json
+cat pools.json | propr agent pool apply -
+propr agent pool delete balanced-pool
+propr agent pool delete balanced-pool --json
+```
+
+`pool list --json` emits `{ "synthetic_agents": [...] }`. That file can be passed unchanged to `pool apply`; `apply` also accepts the array itself. Full-document replacement keeps nested multi-model configuration unambiguous and makes review, backup, and automation straightforward. Backend validation messages, including nested field paths, are printed without being rewritten.
+
+An abbreviated two-tier document looks like this (IDs must be UUIDs):
+
+```json
+{
+  "synthetic_agents": [{
+    "id": "11111111-1111-4111-8111-111111111111",
+    "alias": "balanced-pool",
+    "enabled": true,
+    "defaultModel": "balanced",
+    "models": [{
+      "id": "balanced",
+      "displayName": "Balanced",
+      "enabled": true,
+      "strategy": "usage_based",
+      "members": [
+        {
+          "id": "22222222-2222-4222-8222-222222222222",
+          "directAgentAlias": "codex-primary",
+          "model": "gpt-5.6-sol",
+          "enabled": true,
+          "priority": 100,
+          "usageLimits": { "weeklyMaxPercent": 80 }
+        },
+        {
+          "id": "33333333-3333-4333-8333-333333333333",
+          "directAgentAlias": "claude-fallback",
+          "model": "claude-sonnet-5",
+          "enabled": true,
+          "priority": 0
+        }
+      ]
+    }]
+  }]
+}
+```

docs/docs/features/web-ui.md

@@ -58,6 +58,8 @@ See [Repository Knowledge](./repository-knowledge.md) and [Branch Configuration]
 
 **Coding Agents** (`/ai-agents`) is an administrator-only split view: configure agent aliases and their models on one side, and a **playground** to test an agent interactively on the other. When adding Claude, Codex, Antigravity, or OpenCode, choose a new-account login or reuse an existing config. New-account login creates an isolated ProPR-managed credential directory, so multiple accounts of the same provider can coexist without entering host paths. The login dialog starts the configured agent image, displays the CLI's authorization link and instructions, and accepts requested confirmation codes or terminal menu input without requiring the agent CLI on the host. Existing entries also include **Log in**. The dialog includes Up, Down, and Enter controls for provider and login-method menus; Escape or backdrop dismissal cancels its temporary container. Vibe uses an API key or pre-populated config instead of this interactive flow. See [Agents And Models](./agents-and-models.md).
 
+Administrators can switch the configuration pane to **Synthetic Pools** to combine direct agent/model pairs behind virtual models with strict priority tiers, usage caps, round-robin or usage-based routing, and failover. Synthetic models also appear in the playground, which reports the virtual choice and physical member used. See [Synthetic Pools](./synthetic-pools.md).
+
 ## LLM Log
 
 **LLM Log** (`/llm-logs`) shows every model call with expandable rows and filters by execution type, model, status, and work type. What each record contains and how to use the page for cost analysis is covered in [Metrics](../operations/metrics.md).

docs/sidebars.ts

@@ -82,6 +82,7 @@ const sidebars: SidebarsConfig = {
       label: 'Reference',
       items: [
         'features/agents-and-models',
+        'features/synthetic-pools',
         'features/propr-cli',
       ],
     },

packages/api/routes/agentRoutes.ts

@@ -11,6 +11,7 @@ import {
   toProprOpenCodeModelId,
   type Agent,
   type AgentRegistry,
+  SyntheticAgent,
 } from '@propr/core';
 import { AGENT_DEFAULTS, isManagedAgentConfigPath } from '@propr/shared';
 import { requireManageAgents } from '../permissionGuards.js';
@@ -19,6 +20,7 @@ const execFileAsync = promisify(execFile);
 
 interface AgentChatQuery {
   agentId: string;
+  syntheticConfigId?: string;
   model?: string;
 }
 
@@ -35,6 +37,31 @@ interface AgentChatResult {
   response?: string;
   error?: string;
   durationMs: number;
+  syntheticConfigId?: string;
+  virtualAgentAlias?: string;
+  virtualModel?: string;
+  physicalAgentAlias?: string;
+  physicalModel?: string;
+  attemptNumber?: number;
+}
+
+interface ChatRoutingMetadata {
+  virtualAgentAlias?: string;
+  virtualModel?: string;
+  physicalAgentAlias?: string;
+  physicalModel?: string;
+  attemptNumber?: number;
+}
+
+function chatRoutingFields(metadata: Record<string, unknown> | undefined): ChatRoutingMetadata {
+  if (!metadata) return {};
+  return {
+    virtualAgentAlias: typeof metadata.virtualAgentAlias === 'string' ? metadata.virtualAgentAlias : undefined,
+    virtualModel: typeof metadata.virtualModel === 'string' ? metadata.virtualModel : undefined,
+    physicalAgentAlias: typeof metadata.physicalAgentAlias === 'string' ? metadata.physicalAgentAlias : undefined,
+    physicalModel: typeof metadata.physicalModel === 'string' ? metadata.physicalModel : undefined,
+    attemptNumber: typeof metadata.attemptNumber === 'number' ? metadata.attemptNumber : undefined,
+  };
 }
 
 function resolveHostPath(configPath: string): string {
@@ -130,6 +157,58 @@ function canonicalChatModel(agent: Agent, model: string | undefined): string {
     : fallbackModel;
 }
 
+async function executeChatQuery(
+  registry: AgentRegistry,
+  query: AgentChatQuery,
+  prompt: string,
+  context: string | undefined,
+): Promise<AgentChatResult> {
+  const requestedAgentId = query.syntheticConfigId || query.agentId;
+  const agent = await resolveChatAgent(registry, requestedAgentId);
+
+  if (!agent) {
+    return {
+      agentId: requestedAgentId,
+      model: query.model || 'default',
+      error: 'Agent not found',
+      durationMs: 0,
+    };
+  }
+
+  const start = Date.now();
+  const routingSession = agent instanceof SyntheticAgent
+    ? agent.beginRoutingSession(query.model)
+    : undefined;
+
+  try {
+    const analysisResult = routingSession
+      ? await routingSession.analyze(prompt, { context, model: query.model })
+      : await agent.analyze(prompt, { context, model: query.model });
+    const routing = chatRoutingFields(routingSession?.routingMetadata);
+    return {
+      agentId: requestedAgentId,
+      ...(query.syntheticConfigId ? { syntheticConfigId: query.syntheticConfigId } : {}),
+      agentAlias: agent.config.alias,
+      model: routing.virtualModel || canonicalChatModel(agent, analysisResult.modelUsed || query.model),
+      ...routing,
+      response: analysisResult.response,
+      error: analysisResult.success === false ? (analysisResult.error || 'Analysis failed') : undefined,
+      durationMs: Date.now() - start,
+    };
+  } catch (error) {
+    const routing = chatRoutingFields(routingSession?.routingMetadata);
+    return {
+      agentId: requestedAgentId,
+      ...(query.syntheticConfigId ? { syntheticConfigId: query.syntheticConfigId } : {}),
+      agentAlias: agent.config.alias,
+      model: routing.virtualModel || canonicalChatModel(agent, query.model),
+      ...routing,
+      error: (error as Error).message,
+      durationMs: Date.now() - start,
+    };
+  }
+}
+
 export function createAgentRoutes() {
   const router = Router();
 
@@ -171,38 +250,7 @@ export function createAgentRoutes() {
       // use the same agent credentials concurrently.
       const results: AgentChatResult[] = [];
       for (const query of queries) {
-          const agent = await resolveChatAgent(registry, query.agentId);
-
-          if (!agent) {
-            results.push({
-              agentId: query.agentId,
-              model: query.model || 'default',
-              error: 'Agent not found',
-              durationMs: 0
-            });
-            continue;
-          }
-
-          const start = Date.now();
-          try {
-            const analysisResult = await agent.analyze(prompt, { context, model: query.model });
-            results.push({
-              agentId: query.agentId,
-              agentAlias: agent.config.alias,
-              model: canonicalChatModel(agent, analysisResult.modelUsed || query.model),
-              response: analysisResult.response,
-              error: analysisResult.success === false ? (analysisResult.error || 'Analysis failed') : undefined,
-              durationMs: Date.now() - start
-            });
-          } catch (err) {
-            results.push({
-              agentId: query.agentId,
-              agentAlias: agent.config.alias,
-              model: canonicalChatModel(agent, query.model),
-              error: (err as Error).message,
-              durationMs: Date.now() - start
-            });
-          }
+        results.push(await executeChatQuery(registry, query, prompt, context));
       }
 
       res.json({ results });