Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions desktop/docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,23 @@ closed `EngineType` union and narrows external strings with `isEngineType()`.
`engineManagerName()` / `engineTypeFromManagerName()` are the one place that
translation to and from the engine manager's own spelling lives.

### Model downloads

A download is started with `engine:action{pull_model}`, or
`engine:remote-pull-model` for a pinned peer, and stopped with
`engine:cancel-pull` or `engine:remote-cancel-pull`. All of them name their
target in a `model` field: an engine can be downloading several models at once,
and `engine:pull-progress` carries the same field so every frame lands on the
row it belongs to.

Cancellation is a request, not a result. The engine manager stops the transfer
and settles the partial files before answering, and the row clears when the
pull itself settles. A peer is given a much longer budget to answer than the
desktop waits, on purpose — cutting a peer off part-way through a cancel is
worse than waiting for it — so the desktop giving up first means the cancel is
still running, not that it failed. The row stays in Canceling and the cancel
can be issued again.

### Engine settings

Server port, proxy port, and the engine arguments are one authoritative
Expand Down
14 changes: 13 additions & 1 deletion desktop/docs/frontend-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ They do not imply a WebSocket connection. Browser clients are not supported.
- `getInitialState()` returns engine statuses, models, active progress, and available
updates.
- `toggle`, `install`, `uninstall`, and `update` manage engine lifecycle.
- `pullModel`, `loadModel`, `unloadModel`, `deleteModel`, and
- `pullModel`, `cancelModelPull`, `loadModel`, `unloadModel`, `deleteModel`, and
`setModelExpiry` manage models.
- `searchHub(engineType)` returns the curated model catalog for an engine.
- `onStateChanged`, `onProgress`, and `onProgressRemove` expose backend truth.
Expand Down Expand Up @@ -112,6 +112,7 @@ environment assignments pass through without an engine-option catalog.
- `uninstall`;
- `update`;
- `pullModel`;
- `cancelModelPull`;
- `loadModel`;
- `unloadModel`;
- `deleteModel`;
Expand All @@ -121,6 +122,17 @@ Commands return no state. Renderer stores update from
`engines:state-changed`, `engines:progress-changed`, and
`engines:progress-cleared`.

The model-bearing commands — `pullModel`, `cancelModelPull`, `loadModel`,
`unloadModel`, `deleteModel`, and `setModelExpiry` — carry the target in
`model`. It is what lets a node run several downloads at once and have each
one cancelled, and progress-tracked, on its own.

`cancelModelPull` reaches `engine:cancel-pull`, or `engine:remote-cancel-pull`
when `nodeId` names a peer. The download is not cancelled when the command
returns: the row moves to Canceling and clears when the pull itself settles. A
cancel whose request outlives its budget leaves the row in Canceling, because
the backend is still working on it, and stays available to issue again.

### `pairApi.workloads`

- `getInitial()` returns active workloads.
Expand Down
2 changes: 2 additions & 0 deletions desktop/docs/services-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@
| `errors:clear` | notification (we consume) | ✅ yes |
| `errors:report` | notification (we consume) | ✅ yes |
| `engine:action` | request (we call) | ✅ yes |
| `engine:cancel-pull` | request (we call) | ✅ yes |
| `engine:configure-launch` | request (we call) | ⚠️ not called |
| `engine:configured-ports` | request (we call) | ⚠️ not called |
| `engine:describe` | request (we call) | ⚠️ not called |
Expand All @@ -109,6 +110,7 @@
| `engine:prepare-shutdown` | request (we call) | ✅ yes |
| `engine:preview-launch` | request (we call) | ⚠️ not called |
| `engine:remote-apply-settings` | request (we call) | ⚠️ not called |
| `engine:remote-cancel-pull` | request (we call) | ✅ yes |
| `engine:remote-delete-model` | request (we call) | ✅ yes |
| `engine:remote-get-installed` | request (we call) | ✅ yes |
| `engine:remote-get-settings` | request (we call) | ⚠️ not called |
Expand Down
21 changes: 21 additions & 0 deletions desktop/docs/services-backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ reserved for inference clients.
| `engine:ready` / `engine:state-changed` | Update engine facts and models | `engines:state-changed` |
| `engine:settings-changed` | Validate and republish the owning node's settings snapshot | `engines:settings-changed` |
| `engine:install-progress` / `engine:remote-progress` | Update operation progress | engine progress pushes |
| `engine:pull-progress` | Advance the optimistic pull entry the `model` field names | engine progress pushes |
| `errors:update` | Replace the error snapshot | `errors:update` |
| `cluster:invite-received` | Parse inbound invite | `cluster:invite-received` |
| `cluster:invite-canceled` / `cluster:invite-expired` | Prune the canceled or timed-out inbound invite from the authoritative set | `cluster:pending-invites-changed` |
Expand Down Expand Up @@ -153,6 +154,26 @@ Local engine operations include install, start, stop, uninstall, update, port
changes, and model actions. Remote cluster operations use the engine manager's
remote control surface where supported.

### Cancelling a download

`engine:cancel-pull` stops a download on this node and
`engine:remote-cancel-pull` stops one on a pinned peer. Both name their target
with a `model` field, which is what distinguishes them from the engine-wide
commands: an engine may have several downloads in flight, and the frames on
`engine:pull-progress` carry the same field so each one lands on its own row.

Neither returns state. The backend answers only once the transfer has stopped
and its partial files are settled, so the reply is an acknowledgement and the
row clears from the pull's own settling, not from the cancel.

That answer can outlive the desktop's budget. A peer is served by the engine
manager's readiness client, whose response-header budget is far longer than
`MODULAR_CANCEL_PULL_TIMEOUT_MS`, because cutting a peer off mid-cancel is
worse than waiting for it. A timeout therefore means "still cancelling", not
"failed": the row stays in Canceling rather than dropping back to Downloading,
and the cancel can be issued again. Only an explicit rejection restores the
previous status and reports an error.

### Engine settings

Ports and the engine arguments are one authoritative, revisioned record owned by
Expand Down
113 changes: 107 additions & 6 deletions desktop/docs/services-parity.md
Original file line number Diff line number Diff line change
Expand Up @@ -267,15 +267,116 @@ the engine simply shows as stopped and then running.

A local `pull_model` streams live download progress: the engine-manager routes
`engine:action{pull_model}` through its streaming pull path and emits
`engine:pull-progress` (`{ engine, op, stage, percent, message }`) — the local
`engine:pull-progress` (`{ engine, model, op, stage, percent, message }`) — the local
counterpart of `engine:remote-progress`. Personal AI Router consumes it in
`applyLocalEngineProgress` (`modular-supervisor.ts` → `modular-state.ts`),
backfilling the dispatched model (the frame carries none) and advancing the
optimistic pull entry's percent in place, so a local pull shows "Pulling · N%"
keying the frame to its optimistic pull entry by model and advancing that
entry's percent in place, so a local pull shows "Pulling · N%"
to completion just like a remote pull. The awaited action response owns
completion (clearing the entry and refreshing the model list); a CLI-driven pull
(LM Studio) emits a single `pulling` marker and degrades to the indeterminate
spinner.
completion (clearing the entry and refreshing the model list). LM Studio's CLI
progress updates the same percentage display. A pull for an engine that already
has one running is reported as `stage:"queued"` until its turn.

The pull request itself stays pending for the whole download, so it is the
request's own response that reports the outcome — a completion or a
cancellation — while progress arrives out of band on the notification:

```mermaid
sequenceDiagram
participant UI as Model Manager
participant Bridge as Electron bridge
participant Manager as Engine manager
participant Engine as Ollama or LM Studio

UI->>Bridge: Download model
Bridge->>UI: Show optimistic "Pulling…" row
Bridge->>Manager: Start pull (request remains pending)
Manager->>Manager: Record the partial files already on disk
Manager->>Engine: Begin download

loop While downloading
Engine-->>Manager: Download status
Manager-->>Bridge: Progress {engine, model, stage, percent}
Bridge-->>UI: Update the matching model row
end

alt Download completes
Engine-->>Manager: Success
Manager-->>Bridge: Original pull request completes
else User cancels
UI->>Bridge: Cancel download
Bridge->>UI: Show "Canceling…"
Bridge->>Manager: Cancel {engine, model}
Manager->>Engine: Stop transfer
Manager->>Manager: Remove the partial files this pull created
Manager-->>Bridge: Cancellation completes
Manager-->>Bridge: Original pull settles as canceled
end

Bridge->>Manager: Refresh model list
Manager-->>Bridge: Authoritative models
Bridge->>UI: Replace temporary download state
```

`engine:cancel-pull` and `engine:remote-cancel-pull` cancel the selected model
download. The UI keeps "Canceling…" visible until the pull settles, including
when the cancel RPC's own budget elapses first — the backend is still working,
so the row is not dropped back to "Downloading". Ollama closes the pull request
and removes partial blobs named by its progress digests; LM Studio receives
Ctrl+C and a negative answer to its background-download prompt, and after
acknowledgement the partial files carrying the requested quantization are
removed. Completed model files, other quantizations, and unrelated downloads are
retained, and a cancellation the CLI never confirmed deletes nothing. Only a
cancellation someone requested removes files: a pull interrupted by app
shutdown, a dropped remote connection, or the action timeout leaves its partial
data resumable.

Both engines funnel every client on the machine into one cache, so naming a file
is not the same as owning it — Ollama blobs are content-addressed, and the LM
Studio app downloads into the same repository directory. Each pull therefore
records the partial files already present before it starts, and cancelling it
considers only the ones that appeared afterwards. A file that is still growing
once the transfer has stopped is shared with another client even by that
measure, so it survives as well:

```mermaid
flowchart TD
Request["Model pull requested"] --> Tracked["Track by engine + model"]
Tracked --> Busy{"Another PAIR pull active<br/>for this engine?"}

Busy -- No --> Snapshot["Record the partial files<br/>already on disk"]
Busy -- Yes --> Queued["Show as queued"]

Queued --> QueueCancel{"Canceled while queued?"}
QueueCancel -- Yes --> NeverStarted["Remove operation<br/>without starting download"]
QueueCancel -- No --> Wait["Wait for active pull to settle"]
Wait --> Snapshot

Snapshot --> Active["Start download"]
Active --> Cancel{"Cancellation requested?"}
Cancel -- No --> Complete["Download completes normally"]
Cancel -- Yes --> Engine{"Which engine?"}

Engine -- Ollama --> OllamaStop["Close HTTP pull request"]
OllamaStop --> OllamaCandidates["Select partial blobs named<br/>by this pull's digests"]

Engine -- LM Studio --> LMSStop["Send Ctrl+C"]
LMSStop --> LMSPrompt["Answer no to background download"]
LMSPrompt --> LMSCandidates[".part files carrying the<br/>requested quantization"]

OllamaCandidates --> Owned{"Absent from<br/>the snapshot?"}
LMSCandidates --> Owned
Owned -- No --> Preserve["Preserve completed,<br/>shared, and unrelated files"]
Owned -- Yes --> Quiet{"Held still across<br/>repeated checks?"}
Quiet -- No --> Preserve
Quiet -- Yes --> Delete["Remove the partial file"]

Delete --> Settle["Settle pull and refresh models"]
Preserve --> Settle
Complete --> Settle
NeverStarted --> Settle
Settle --> Next["Allow next queued pull to start"]
```

`engine:models` (and the `em` `GET /v1/models` surface) returns the flat model
union, the per-engine breakdown (`modelsByEngine`), and the per-engine set of
Expand Down
10 changes: 10 additions & 0 deletions desktop/src/electron/service-bridge/empty-handlers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,11 @@ function routeEngineManagerCommand(payload: WsInvokeRequest<'engine:command'>):
void supervisor.pullModel(engine, payload.engineType, payload.model)
}
break
case 'cancelModelPull':
if (payload.model) {
void supervisor.cancelModelPull(engine, payload.engineType, payload.model)
}
break
case 'deleteModel':
if (payload.model) {
void supervisor.deleteModel(engine, payload.engineType, payload.model)
Expand Down Expand Up @@ -365,6 +370,11 @@ function routeRemoteEngineCommand(payload: WsInvokeRequest<'engine:command'>): v
void supervisor.pullModelRemote(nodeId, engine, payload.engineType, payload.model)
}
break
case 'cancelModelPull':
if (payload.model) {
void supervisor.cancelModelPull(engine, payload.engineType, payload.model, nodeId)
}
break
case 'uninstall':
case 'update':
refuseRemote(
Expand Down
9 changes: 8 additions & 1 deletion desktop/src/electron/service-bridge/json-rpc-subprocess.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@ interface JsonRpcError {

export class JsonRpcResponseError extends Error {}

/**
* The request's own budget elapsed. The backend has not answered and has not
* failed either — it is still working — so a caller that showed optimistic
* state must decide whether to keep it rather than assume the operation lost.
*/
export class JsonRpcTimeoutError extends Error {}

type JsonRpcId = number | string

interface JsonRpcMessage {
Expand Down Expand Up @@ -148,7 +155,7 @@ export class JsonRpcSubprocess extends EventEmitter<JsonRpcSubprocessEvents> {
? undefined
: setTimeout(() => {
this.pending.delete(id)
reject(new Error(`${this.name} ${method} timed out`))
reject(new JsonRpcTimeoutError(`${this.name} ${method} timed out`))
}, timeoutMs)
this.pending.set(id, { resolve, reject, timeout })
this.write(message).catch(err => {
Expand Down
Loading
Loading