Skip to content
Open
4 changes: 2 additions & 2 deletions docs/docs/features/agents-and-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ Reasoning flags require Claude Code >= 2.1.68 and Codex CLI >= 0.144.0. Saving a

## Model Labels

Every model has a GitHub label of the form `llm-<agent>-<model-alias>`. Add one to an issue (together with your trigger label, such as `AI`) to route that issue to the model. The same identity follows the run through the system: task records show the selected agent and model, and branch names include the model identifier for traceability.
Every model has a GitHub label of the form `llm-<agent>-<model-alias>`. For ordinary issues, add the trigger label (such as `AI`) by itself and ProPR uses the configured default model. Add one model label only when you want an explicit override. The same identity follows the run through the system: task records show the selected agent and model, and branch names include the model identifier for traceability.

Adding **several** model labels to one issue fans the work out into one job per model label: each model gets its own run, its own worktree and branch, and its own pull request. Compare the PRs, merge the best one, and close the rest. If a `base-<branch>` label is also present, the fan-out is per base × per model.

Expand All @@ -84,7 +84,7 @@ The same aliases work in PR comments (the `llm-` prefix is optional; the raw cat

```
/switch claude-opus5 # future follow-ups on this PR use this model
/use codex-gpt56-sol # one follow-up with this model
/use codex-gpt56-sol # change this PR's model label
/review claude-opus5 codex-gpt56-sol # independent reviews from two models
```

Expand Down
13 changes: 6 additions & 7 deletions docs/docs/features/pr-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,15 @@ To **take over an existing PR** for ongoing work (so that natural follow-up comm
| `/fix` | You want to apply a `/review`'s pending suggestions | Yes | [`/fix`](#fix) |
| `/merge` | You want the base branch merged into the PR branch | Maybe, if conflicts need resolution | [`/merge`](#merge) |
| `/switch <model-id>` | You want future PR work to use a different model | No, unless you include follow-up instructions | [`/switch`](#switch) |
| `/use <model-id>` | You want one immediate follow-up run with a temporary model | Yes | [`/use`](#use) |
| `/use <model-id>` | You want to change the PR's model label | No | [`/use`](#use) |
| `/ultrafix` | You want an automated review-fix loop | Yes | [`/ultrafix`](#ultrafix) |

## Syntax Rules

- The slash command must be on the first line of the PR comment. A comment with leading blank lines or text before the command is treated as a normal follow-up comment.
- Arguments go on the same line as the command (for example `/review llm-claude-opus5` or `/ultrafix goal=8 max=10`).
- Lines below the command become extra instructions for the run.
- Both top-level PR comments and line-level review comments are processed; line-level comments carry their file, line, and diff context to the agent.
- Lines below the command become extra instructions when the command queues work. `/use` ignores trailing text because it only changes a label.
- Both top-level PR comments and line-level review comments are processed; when a command queues work, line-level comments carry their file, line, and diff context to the agent.

## Model IDs

Expand Down Expand Up @@ -170,14 +170,13 @@ Without instructions, `/switch` only updates the label and makes no code changes

### `/use`

`/use` runs one immediate follow-up task with a temporary model:
`/use` changes the PR's managed model label:

```text
/use <model-id>
Please investigate the flaky test failure and update the PR.
```

The PR's model label keeps its current value. Later work returns to the PR's configured model unless you use `/switch` or another `/use`. Like `/switch`, `/use` takes one model argument, and the agent sees only your instructions, without the command syntax.
ProPR resolves a supported short alias or full `llm-*` label, removes the PR's other managed model labels, and adds the selected canonical label. All unrelated labels are preserved. `/use` does not queue work; any trailing text is ignored. Post a separate follow-up comment when you want the selected model to do work.

### Choosing A Model

Expand All @@ -187,7 +186,7 @@ Use routing when:

- A model is better suited to the task
- The current model is stuck
- You want a one-off second opinion
- You want future work on the PR to use a different model
- You need to work around provider capacity or rate limits

## Ultrafix And Branch Updates
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/operations/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ Then check credentials, branch settings, and agent configuration — the usual c
Recovery runs through the PR conversation:

- Add a clearer follow-up comment with stronger instructions.
- `/switch <model-id>` to change the PR's model going forward, or `/use <model-id>` for a one-off task with a different model.
- `/use <model-id>` to change the PR's model label without queuing work, or `/switch <model-id>` when you also need its optional follow-up behavior.
- `/review` then `/fix`, or `/ultrafix` for an automated review-fix loop (remove the `ultrafix` PR label to stop it).
- Re-run with a smaller scope — see [Work Splitting](../features/work-splitting.md).
- Undo a bad commit with `propr task revert owner/repo <pr> <sha> <issue>`, which runs a signed system task (authorized via `SYSTEM_TASK_SECRET`) that resets the branch and force-pushes.
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/tutorials/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Use slash commands only for specific actions:
- `/fix` applies unprocessed AI review comments generated by `/review`.
- `/merge` merges the base branch into the PR branch, attempts automatic conflict resolution, and reports back.
- `/switch <model-id>` changes the PR's model label going forward.
- `/use <model-id>` runs one follow-up task with that model without changing the PR's model.
- `/use <model-id>` changes the PR's managed model label without queuing work.
- `/ultrafix` runs a review-fix loop. Parameters: `goal=<score>`, `max=<cycles>`, `pause=<seconds>`, `model=<model-id>`, for example `/ultrafix goal=9 max=5`. It waits for CI checks and PR inactivity between cycles. The `ultrafix` PR label is the circuit breaker — remove it to stop the loop.

See [PR Slash Commands](../features/pr-commands.md).
Expand Down
193 changes: 182 additions & 11 deletions packages/core/src/webhook/commentEventHandler.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,12 @@ import { handleMergeCommand } from './mergeConflictDetector.js';
import { parseSlashCommand, buildCommandMeta } from './slashCommandParser.js';
import type { CommandMeta, UltrafixCommandMeta } from './slashCommandParser.js';
import { safeUpdateLabels } from '../utils/github/labelOperations.js';
import { resolveModelAlias } from '../config/modelAliases.js';
import { resolveModelAlias, resolveReviewModels } from '../config/modelAliases.js';
import { MODEL_INFO_MAP } from '../config/modelDefinitions.js';
import { getBotUsername } from '../daemon/configLoader.js';
import { AgentRegistry } from '../agents/AgentRegistry.js';
import type { DeliveryDisposition } from '../intake/routingWebSocketProtocol.js';
import { buildAgentModelLlmLabel, buildDynamicLlmLabel, MAX_GITHUB_LABEL_LENGTH, shortHash } from '@propr/shared';

export interface UltrafixDeps {
loadUltrafixRatingGoal: () => Promise<number>;
Expand Down Expand Up @@ -283,19 +284,11 @@ async function handleSlashCommand(opts: SlashCommandHandlerOptions): Promise<voi
return;
}

if (commandMeta.mode === 'use' && commandMeta.models.length === 0) {
correlatedLogger.warn({ pullRequestNumber: prNumber, commentId: comment.id, commentAuthor }, '/use command requires a model argument, ignoring');
if (commandMeta.mode === 'use') {
await handleUseCommand({ commandMeta, comment, commentAuthor, eventContext, payload, config, correlationId, correlatedLogger });
return;
}

if (commandMeta.mode === 'use' && commandMeta.models.length > 0) {
const resolvedModel = resolveModelAlias(commandMeta.models[0]);
if (!await isKnownOrConfiguredModel(resolvedModel)) {
correlatedLogger.warn({ pullRequestNumber: prNumber, invalidModels: [resolvedModel] }, '/use command contains unrecognized model(s), ignoring');
return;
}
}

const manualTakeover = await fenceManualCommand({ commandMeta, comment, eventContext, config, correlatedLogger });

correlatedLogger.info({ pullRequestNumber: prNumber, commentId: comment.id, commentAuthor, command: commandMeta.mode }, `/${commandMeta.mode} command detected, enqueuing job`);
Expand Down Expand Up @@ -325,6 +318,184 @@ async function handleSlashCommand(opts: SlashCommandHandlerOptions): Promise<voi
await enqueueNewCommentJob(strippedComment, commentAuthor, eventContext, { payload, redisClient, PR_FOLLOWUP_TRIGGER_KEYWORDS: config.PR_FOLLOWUP_TRIGGER_KEYWORDS, MODEL_LABEL_PATTERN: config.MODEL_LABEL_PATTERN, correlationId, commandMeta, commentRevisionIdentity: manualTakeover?.commentRevisionIdentity });
}

type UseCommandOptions = Omit<SlashCommandHandlerOptions, 'parsedCommand'> & { commandMeta: CommandMeta & { mode: 'use' } };

function buildPrefixedDynamicModelLabel(prefix: string, agentAlias: string, modelId: string): string | null {
const canonicalLabel = `${prefix}${agentAlias}~${modelId}`;
if (canonicalLabel.length <= MAX_GITHUB_LABEL_LENGTH) return canonicalLabel;

const hash = shortHash(modelId);
const maxAliasLength = Math.max(1, MAX_GITHUB_LABEL_LENGTH - `${prefix}~-x-${hash}`.length);
const sanitizedAlias = agentAlias
.replace(/[^a-zA-Z0-9_.-]/g, '-')
.slice(0, maxAliasLength)
.replace(/[^a-zA-Z0-9]+$/, '');
const labelAlias = sanitizedAlias || 'agent'.slice(0, maxAliasLength);
const modelPrefixBudget = MAX_GITHUB_LABEL_LENGTH - `${prefix}${labelAlias}~-${hash}`.length;
const fallbackPrefix = 'model'.slice(0, Math.max(1, modelPrefixBudget));
const modelPrefix = modelId
.replace(/[^a-zA-Z0-9_.-]/g, '-')
.slice(0, Math.max(1, modelPrefixBudget))
.replace(/[^a-zA-Z0-9]+$/, '');
const hashedLabel = `${prefix}${labelAlias}~${modelPrefix || fallbackPrefix}-${hash}`;
return hashedLabel.length <= MAX_GITHUB_LABEL_LENGTH ? hashedLabel : null;
}

function applyModelLabelPrefix(defaultLabel: string, prefix: string, agentAlias: string, modelId: string): string | null {
if (!defaultLabel.startsWith('llm-')) {
return defaultLabel.length <= MAX_GITHUB_LABEL_LENGTH ? defaultLabel : null;
}

const suffix = defaultLabel.slice('llm-'.length);
if (suffix.includes('~')) {
return buildPrefixedDynamicModelLabel(prefix, agentAlias, modelId);
}

const staticLabel = `${prefix}${suffix}`;
return staticLabel.length <= MAX_GITHUB_LABEL_LENGTH
? staticLabel
: buildPrefixedDynamicModelLabel(prefix, agentAlias, modelId);
}

async function resolveCanonicalModelLabel(
target: string,
modelLabelPattern: string,
correlatedLogger: ReturnType<typeof logger.withCorrelation>,
prNumber: number,
): Promise<string | null> {
try {
const modelLabelRegex = new RegExp(modelLabelPattern);
const targetMatch = modelLabelRegex.exec(target);
const routingTarget = targetMatch?.[1] || target;
const [resolution] = await resolveReviewModels([routingTarget]);
const registry = AgentRegistry.getInstance();
const agent = registry.getAgentByAlias(resolution.agentAlias);
if (!agent) {
correlatedLogger.warn(
{ pullRequestNumber: prNumber, target },
'/use target is unknown, disabled, or unsupported; model label was not changed',
);
return null;
}

const modelInfo = MODEL_INFO_MAP[resolution.model];
const defaultLabel = modelInfo
? buildAgentModelLlmLabel(agent.config.type, agent.config.alias, modelInfo)
: buildDynamicLlmLabel(agent.config.alias, resolution.model);
const { prefix, derived } = modelLabelPrefix(modelLabelPattern);
if (!derived) {
correlatedLogger.warn(
{ pullRequestNumber: prNumber, modelLabelPattern },
'Could not derive label prefix from MODEL_LABEL_PATTERN; /use cannot safely select a model label',
);
return null;
}

const canonicalLabel = applyModelLabelPrefix(defaultLabel, prefix, agent.config.alias, resolution.model);
if (!canonicalLabel) {
correlatedLogger.error(
{ pullRequestNumber: prNumber, modelLabelPattern },
'/use could not build a canonical model label within GitHub\'s label length limit',
);
return null;
}
const canonicalMatch = modelLabelRegex.exec(canonicalLabel);
const canonicalRoutingToken = canonicalMatch?.[1];
if (!canonicalRoutingToken) {
correlatedLogger.error(
{ pullRequestNumber: prNumber, canonicalLabel, modelLabelPattern },
'/use resolved a canonical label that does not match MODEL_LABEL_PATTERN',
);
return null;
}

let routedResolution: Awaited<ReturnType<typeof resolveReviewModels>>[number];
try {
[routedResolution] = await resolveReviewModels([canonicalRoutingToken]);
} catch (error) {
correlatedLogger.error(
{ pullRequestNumber: prNumber, canonicalLabel, canonicalRoutingToken, error: (error as Error).message },
'/use resolved a canonical label that cannot route back to the selected model',
);
return null;
}
if (routedResolution.agentAlias !== resolution.agentAlias || routedResolution.model !== resolution.model) {
correlatedLogger.error(
{
pullRequestNumber: prNumber,
canonicalLabel,
selectedAgentAlias: resolution.agentAlias,
selectedModel: resolution.model,
routedAgentAlias: routedResolution.agentAlias,
routedModel: routedResolution.model,
},
'/use resolved a canonical label that routes to a different model',
);
return null;
}
return canonicalLabel;
} catch (error) {
correlatedLogger.warn(
{ pullRequestNumber: prNumber, target, error: (error as Error).message },
'/use target is unknown, disabled, or unsupported; model label was not changed',
);
return null;
}
}

async function handleUseCommand(opts: UseCommandOptions): Promise<void> {
const { commandMeta, comment, commentAuthor, eventContext, config, correlatedLogger } = opts;
const { prNumber, owner, repo } = eventContext;

if (commandMeta.models.length === 0) {
correlatedLogger.warn({ pullRequestNumber: prNumber, commentId: comment.id, commentAuthor }, '/use command requires a model argument, ignoring');
return;
}

const modelLabelPattern = config.MODEL_LABEL_PATTERN || '^llm-(.+)$';
const canonicalLabel = await resolveCanonicalModelLabel(commandMeta.models[0], modelLabelPattern, correlatedLogger, prNumber);
if (!canonicalLabel) return;

const { prLabels } = await getLivePRBranchAndLabels({ owner, repo, prNumber });
const canonicalLabelIdentity = canonicalLabel.toLowerCase();
const existingModelLabels = prLabels.filter(label => label.name.startsWith('llm-')).map(label => label.name);
const labelsToRemove = existingModelLabels.filter(label => label.toLowerCase() !== canonicalLabelIdentity);
const targetPresent = prLabels.some(label => label.name.toLowerCase() === canonicalLabelIdentity);

if (labelsToRemove.length === 0 && targetPresent) {
correlatedLogger.debug({ pullRequestNumber: prNumber, modelLabel: canonicalLabel }, '/use model label is already active');
return;
}

const labels = [
...prLabels
.filter(label => !label.name.startsWith('llm-') && label.name.toLowerCase() !== canonicalLabelIdentity)
.map(label => label.name),
canonicalLabel,
];

try {
const octokit = await getAuthenticatedOctokit();
await octokit.request('PUT /repos/{owner}/{repo}/issues/{issue_number}/labels', {
owner,
repo,
issue_number: prNumber,
labels,
});
} catch (error) {
correlatedLogger.error(
{ pullRequestNumber: prNumber, modelLabel: canonicalLabel, error: (error as Error).message },
'/use failed to update the PR model label',
);
return;
}

correlatedLogger.info(
{ pullRequestNumber: prNumber, modelLabel: canonicalLabel },
'/use updated the PR model label',
);
}

type SwitchCommandOptions = Omit<SlashCommandHandlerOptions, 'parsedCommand'> & { commandMeta: CommandMeta & { mode: 'switch' } };

async function handleSwitchCommand(opts: SwitchCommandOptions): Promise<void> {
Expand Down
5 changes: 3 additions & 2 deletions packages/core/src/webhook/slashCommandParser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ export interface SwitchCommandMeta {

export interface UseCommandMeta {
mode: 'use';
/** Target model labels for single-run override */
/** Target model label to make active on the PR */
models: string[];
/** Extra instructions from lines below the command */
instructions: string;
Expand Down Expand Up @@ -120,7 +120,8 @@ function normalizeModelLabel(label: string): string {
* For `/fix`: captures everything after `/fix` as instructions.
* For `/merge`: returns a simple merge marker.
* For `/switch`: extracts single model target and optional instructions.
* For `/use`: extracts single model for one-time override and optional instructions.
* For `/use`: extracts the single model label target. Trailing text is parsed for
* compatibility but ignored by the webhook handler.
* For `/ultrafix`: parses positional goal or named key=value arguments.
*/
export function buildCommandMeta(parsed: ParsedSlashCommand): CommandMeta {
Expand Down
2 changes: 1 addition & 1 deletion src/shared/slashCommandsBlock.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ export function buildSlashCommandsBlock(): string {
'| `/review` | Request an AI code review | `/review` or `/review claude-sonnet` |',
'| `/fix` | Implement fixes for issues found by `/review` | `/fix` or `/fix address the null check issue` |',
'| `/switch` | Change the AI model for this PR | `/switch claude-opus` |',
'| `/use` | Override the model for a single follow-up run | `/use claude-sonnet` |',
'| `/use` | Change this PR\'s model label | `/use claude-sonnet` |',
'| `/ultrafix` | Loop review→fix cycles until score goal is met | `/ultrafix` or `/ultrafix goal=8 max=10` |',
'',
'</details>',
Expand Down
Loading
Loading