Repository navigation
Conversation
Collaborator
Author
3 tasks done
rominf
force-pushed
the
feat/service-api-docs
branch
from
October 26, 2026 10:17
c3612ca to
54625d3
Compare
rominf
force-pushed
the
feat/serve-speech-to-text
branch
from
October 26, 2026 10:17
2d0d400 to
513c0a2
Compare
Of the servers rocm-cli runs, only vLLM generates API docs; Lemonade and the llama.cpp server behind it serve none, so a user of `rocm serve` gets an endpoint URL and no reference for what it accepts. The background helper `rocm serve` already starts now serves docs for every running model server on 127.0.0.1 (port 11430, `api_docs` in config.json): an OpenAPI description per server built with utoipa from the routes each engine reports, rendered by Swagger UI (vendored, works offline). "Try it out" goes through a same-origin forwarder to the server, so it works on every engine whatever its cross-origin settings. The forwarder bypasses proxies, passes on only Content-Type, Accept and Authorization, and refuses non-loopback Host and cross-site Origin headers. `rocm serve` and `rocm services list` print a `docs` link, but only once the docs answer with that server's description, so a link is never printed for docs that are off, a taken port, or an older helper. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Every served model was assumed to be a chat model: the protocol had no notion of what a model does, readiness was proven with a chat request, and the serve summary always printed /v1/chat/completions. Add a model task (chat or transcription) to catalog recipes, service records and the engine protocol. Every new field defaults to chat, so records, state files and external engine plugins written before this change keep their behaviour, and a plugin that does not advertise tasks is treated as chat-only. - rocm serve refuses a task the selected engine does not advertise before the GPU gate, naming an engine that can serve it. - The Lemonade adapter serves transcription models through its router with an explicit whispercpp backend (ROCm, else Vulkan, never CPU) and proves a test transcription before reporting ready. Lemonade's per-model device label is not trusted: 11.5.1 reports ROCm Whisper as cpu, so the loaded backend is verified instead, as for chat. - Readiness probes send a transcription for a speech-to-text service rather than a chat request, whose 404 would read as serving. - The task survives restarts from both rocm and rocmd. - vLLM advertises chat only and refuses other tasks itself. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Add Whisper-Large-v3-Turbo (alias `whisper`, `whisper-turbo`) and Whisper-Tiny as speech-to-text recipes served by Lemonade. A speech-to-text server cannot answer chat, so the commands that need a chat model now recognise one and say why instead of failing obscurely: - rocm chat --model <speech model> refuses before sending any request, and the built-in assistant selection never picks a speech service. - The Console chat pane skips speech-to-text services, and its well-known-port fallback will not adopt a port one of them holds. - rocm bench load refuses a model the catalog marks as speech-to-text. - rocm services list shows the task and the transcription route for a speech-to-text server, and the assistant's services tool reports the task. Scenarios cover the chat-only engine refusal, the services list entry and its working endpoint, the chat and bench refusals, and (on the Lemonade GPU lanes) a real Whisper serve answering on the printed endpoint. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Describe `rocm serve whisper` and its transcription endpoint in the README and `rocm serve --help`, the engine `tasks` capability and the Lemonade whisper.cpp backend choice in the engine-plugin guide, a manual GPU check for speech-to-text, the focused tests, and the two new modules in the architecture map. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Review of the speech-to-text change found paths that still treated every service as chat: - rocmd recovery rebuilt the record as chat and relaunched it that way; the task is now carried over from the registry like the other fields. - The CLI's own readiness wait after a launch or restart required a llama.cpp backend and sent a chat probe, so a Whisper service ended its launch as `starting`. It now checks for the model loaded through whispercpp on a GPU backend and probes with a transcription. - An unrecognised task value failed the whole capabilities message, record or recipe index. It now degrades only its own entry; a recipe with such a task fails when it is served. - A failed whisper.cpp backend install advised a command that only reinstalls the llama.cpp backend. Also: `rocm services list` keeps `endpoint` as the base URL and adds a speech-to-text endpoint line; the attached serve prints that line too; the chat refusal says to drop --model; wording no longer promises more than is checked (the CPU build is never selected). Adds the missing regression tests (record, argv and recovery threading, readiness) and a scenario for the interactive chat pane skipping a newer speech service. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
A speech-to-text server answers the OpenAI transcription route, not the chat routes, so its API docs listed routes it does not serve and offered no way to try the one it does. The docs now pick the routes from the server's task, from the same per-task list the engines report as supported routes: a speech-to-text server's page lists POST /v1/audio/transcriptions with an audio file upload and the model already filled in, and no chat routes. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
rominf
force-pushed
the
feat/service-api-docs
branch
from
October 26, 2026 11:29
54625d3 to
9ac33b1
Compare
rominf
force-pushed
the
feat/serve-speech-to-text
branch
from
October 26, 2026 11:29
513c0a2 to
0ae3160
Compare
- A task recorded by a newer rocm-cli is refused before anything is stopped on restart, recovery and Lemonade resolve/launch, and engine state with an unparsable task reads as unknown rather than chat. - A speech-to-text readiness probe needs a 200 transcript; a 4xx no longer counts as serving. - Another engine advertising speech-to-text is judged by its model list instead of never becoming ready. - The chat provider no longer lists speech-to-text models, and the assistant's services tool names their task and route. - The task endpoint line has one shared helper. - The e2e transcription mock runs on its own thread, and the steps send their transcription requests from a blocking thread, so harness workers blocked on CLI subprocesses cannot starve either side; the mock rejects anything but a real upload for its model, the GPU serve steps check the docs' routes per task again, and the Vulkan whisper.cpp GPU claim is stated as the known gap it is. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
rominf
force-pushed
the
feat/serve-speech-to-text
branch
from
October 26, 2026 12:01
0ae3160 to
028d964
Compare
rominf
force-pushed
the
feat/service-api-docs
branch
from
November 3, 2026 13:41
9ac33b1 to
a170585
Compare
This was referenced Nov 4, 2026
Merged
rominf
force-pushed
the
feat/service-api-docs
branch
from
November 16, 2026 11:35
981ca91 to
588a3c6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
ROCm CLI can only serve chat models, so a user who wants speech-to-text on their AMD GPU has no
rocm servepath for it, even though Lemonade, which ROCm CLI already runs, ships whisper.cpp with ROCm and Vulkan builds. Serving a Whisper model as if it were a chat model fails in a less obvious way: every chat-shaped path (readiness probing with a chat completion,rocm chat, the Console chat pane,rocm bench load, the API docs) treats the speech server as something it is not.What changes
chat(the default, and what every existing record reads as) ortranscription. A task value written by a newer ROCm CLI reads as unknown rather than as chat. Such a server is never probed or offered as chat, and restarting, recovering or serving it is refused before anything is stopped, naming the newer version as what can serve it. The engine protocol carries the task in resolve/launch requests and in each engine's advertisedtasks, omitted for chat so older engines and CLIs see unchanged messages.rocm serve whisper(Whisper Large-v3-Turbo;whisper-tinyis the smallest) runs through Lemonade's router with thewhispercpprecipe. The backend is named explicitly: ROCm, or Vulkan when Lemonade reports ROCm unsupported. Lemonade's CPU build is never selected, even when installed, androcm serve's GPU gate still applies. Before reporting ready, the engine checks that Lemonade loaded the model on that backend and that a short silent recording transcribes with HTTP 200. vLLM refuses speech models, androcm serve whisper --engine vllmis refused before the GPU gate with the Lemonade command that works./v1/healthshows it loaded on a GPU backend (its/v1/modelslists every downloaded model). It is then probed with a transcription that must answer 200, never with a chat request. Another engine advertising speech-to-text is judged by its/v1/models, as its chat models are.rocm services listprint aspeech-to-text endpointline (from one shared helper) beside the unchanged base URL. Restarts and the helper's recovery relaunch the service with its task.rocm chat --model whisperandrocm bench loadrefuse with the reason.POST /v1/audio/transcriptionswith a file upload and the model already filled in, and no chat routes. Route lists per task live inrocm-engine-protocol(supported_routes), shared by Lemonade and the docs.Verification
serve-26: refusal on vLLM with the working command.serve-28: listed with task and endpoint, and that endpoint reaches the transcription route.chat-10: chat refuses and says why.chat-11: the interactive chat pane ignores a newer speech service (Linux lanes).bench-07: bench refuses.service-docs-05: docs list only the transcription route, and an upload through them is transcribed.service-docs-05fails if the forwarder dropsContent-Type. The mock runs on its own thread, and the steps send their transcription requests from a blocking thread, so harness workers blocked on CLI subprocesses cannot starve either side under full-lane concurrency.serve-27serves Whisper-Tiny on a real GPU, transcribes through the printed endpoint, checks the service list, and opens the printed API docs link (Lemonade GPU lanes,@requires-gpu @requires-engine:lemonade).serve-27on the Lemonade GPU lanes).devicefrom the backend name, so it is not trusted. That makes the loaded backend checkable but not where a Vulkan build actually ran. With no usable Vulkan device, whisper.cpp's Vulkan build may run on the CPU unreported, the same gap the chat path's Vulkan llama.cpp has.docs/engine-plugins.mdstates it; the ROCm backend is not affected.rocm services restart; only the restarted argv is unit-tested.Scope
Kokoro and Moonshine, Lemonade's other speech models, are not offered: they run only on the CPU.
No
rocm transcribecommand; a client sends the request to the printed endpoint (README has acurlexample).If this PR fixes a bug, searched
tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows. (Not a bug fix.)If this PR adds a new subcommand or subsystem, its domain implementation lives in its own file per
docs/architecture.md(the clap declaration and dispatch wiring staying inmain.rs/lib.rsis expected, not a violation). (engines/lemonade/src/speech.rs,crates/rocm-core/src/model_task.rs.)Every new or changed user-facing message was read against the code path that runs after it, and its test asserts the resulting state — not only the wording — per AGENTS.md §3. Covers claims of an outcome, remediation advice naming a command, and promises that something will not happen.