Repository navigation
Conversation
Collaborator
Author
rominf
force-pushed
the
feat/serve-speech-to-text
branch
from
October 26, 2026 10:17
2d0d400 to
513c0a2
Compare
rominf
force-pushed
the
feat/text-to-speech-serving
branch
from
October 26, 2026 10:17
59e098b to
f98ee5b
Compare
rominf
force-pushed
the
feat/serve-speech-to-text
branch
from
October 26, 2026 11:29
513c0a2 to
0ae3160
Compare
rominf
force-pushed
the
feat/text-to-speech-serving
branch
from
October 26, 2026 11:29
f98ee5b to
e09103b
Compare
Add a `speech` model task (`POST /v1/audio/speech`) and an OpenMOSS-TTS recipe (aliases `tts`, `moss-tts`), served by Lemonade's `openmoss` recipe on its ROCm or Vulkan build. Lemonade's Kokoro model is not offered because it runs only on the CPU. The Lemonade speech path serves both speech tasks from one table of recipe, load flag, backend option and weight format, and proves a text-to-speech model with one spoken word, answered with 200, before reporting ready; that smoke test also records the server as verified, so the CLI does not repeat it. A text-to-speech probe has a 30 s budget instead of a chat probe's 8 s. Each task's probe budget is now explicit, failed probes are spaced by at least twice their budget so a slow probe is never due again when it returns, the serve wait caps a probe at its own deadline, and chat commands no longer probe servers that are not chat servers. The API docs of a text-to-speech server list its speech route with an example naming the served model. Audio answers are forwarded with an attachment disposition, so the docs page offers them as a download: without it Swagger UI renders a player that re-requests the route with GET, which plays nothing. serve-30 checks that a chat-only engine refuses a text-to-speech model and names the Lemonade command; a nightly GPU scenario (serve-29) serves the 12.5 GB model, checks the printed endpoint returns audio and the service list names its task, and opens the printed API docs link. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
rominf
force-pushed
the
feat/serve-speech-to-text
branch
from
October 26, 2026 12:01
0ae3160 to
028d964
Compare
rominf
force-pushed
the
feat/text-to-speech-serving
branch
from
October 26, 2026 12:01
e09103b to
5d76e0b
Compare
This was referenced Nov 2, 2026
Merged
2 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
With speech-to-text served (base PR), the other half of a local voice loop is still missing: there is no way to have ROCm CLI serve a text-to-speech model on an AMD GPU. Lemonade ships OpenMOSS-TTS with ROCm and Vulkan builds, so it fits the same serving path. A text-to-speech probe also behaves differently from the others: synthesizing one word takes far longer than a one-token chat answer, which the shared readiness timing did not allow for.
What changes
speechmodel task (POST /v1/audio/speech) and an OpenMOSS-TTS recipe (aliasestts,moss-tts; a 12.5 GB download, about 14 GiB of GPU memory), served by Lemonade'sopenmossrecipe on its ROCm or Vulkan build; it has no CPU build.rocm serve tts --engine vllmis refused with the Lemonade command that works.inference_probe_timeout), replacing a floor hidden inside the request.rocm serve's readiness wait caps a probe at its own deadline.rocm chatnever waits on a speech server.POST /v1/audio/speech, with an example naming the served model and an audio answer.audio/*answers an attachment disposition, and Swagger UI then offers the WAV as a download.GET, which plays nothing.--help, README,docs/engine-plugins.mdanddocs/architecture.mdcover the new task.Verification
serve-30(every lane): a chat-only engine refuses a text-to-speech model and names the Lemonade command that serves it.serve-29(nightly, Lemonade GPU lanes; the model is a 12.5 GB download) runs that command, checks the printed endpoint returns audio and the service list names its task, and opens the printed API docs link.rocm serve tts --engine lemonade, which passes the same capability check;lemonade backends;openmossloader;serve-29covers the serve on the nightly lane.docs/engine-plugins.mdstates it.Scope
Lemonade's Kokoro text-to-speech model is not offered: it runs only on the CPU.
Chat commands judge non-chat servers by their recorded status alone, without contacting them. A speech server whose engine died before its record caught up can still be named in a refusal as if it were running; that costs a misleading name in an error, where probing it would have cost every chat command up to 30 s.
The engine's post-load smoke test is a single attempt within the 30 s budget. A first synthesis slower than that fails the serve rather than being retried.
If this PR fixes a bug, searched
tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows. (Not a bug fix.)If this PR adds a new subcommand or subsystem, its domain implementation lives in its own file per
docs/architecture.md(the clap declaration and dispatch wiring staying inmain.rs/lib.rsis expected, not a violation). (Extendsengines/lemonade/src/speech.rs.)Every new or changed user-facing message was read against the code path that runs after it, and its test asserts the resulting state — not only the wording — per AGENTS.md §3. Covers claims of an outcome, remediation advice naming a command, and promises that something will not happen.