From 7e0d2a545c34e26b5c285f25a744589d82702015 Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 12 Nov 2026 16:37:21 -0500 Subject: [PATCH 1/3] docs: add a quickstart from install to first chat reply Add a Quickstart section to the README and the top of the Getting started page. It walks through rocm examine, rocm install sdk, rocm serve, and rocm chat, with a separate serve and chat command for each engine: the qwen alias is a GGUF model that only Lemonade serves, so Instinct GPUs on vLLM need a safetensors model. Also correct the Configure ROCm section, which said qwen serves out of the box on any GPU. Signed-off-by: pmoutsia_amdeng --- README.md | 75 ++++++++++++++++++++++++++++++- docs/rocm-docs/getting-started.md | 22 +++++++++ 2 files changed, 95 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 7848990ec1..956b66f4fe 100644 --- a/README.md +++ b/README.md @@ -164,6 +164,73 @@ install -m 0755 target/release/rocm target/release/rocmd ~/.local/bin/ See [CONTRIBUTING.md](https://github.com/ROCm/rocm-cli/blob/main/CONTRIBUTING.md) for the full development setup, test commands, and commit-signing requirements. +## Quickstart + +These steps take a computer with a supported AMD GPU from installing ROCm CLI to +a chat reply from a model served on that GPU. + +1. Install ROCm CLI with the command for your platform in + [Installation](#installation). Then open a new terminal and confirm that + `rocm version` prints the CLI version. + +2. Check the GPU, the ROCm runtime, and the serving engine: + + ``` + rocm examine + ``` + + Two lines in the output decide the next steps: + + - `active_runtime_status`: `ready` means a managed ROCm runtime is already + active, and you can skip step 3. + - `effective_default_engine`: `lemonade` or `vllm`. The engine decides which + model you serve in step 4. + +3. Install a managed ROCm runtime: + + ``` + rocm install sdk + ``` + + This downloads TheRock ROCm wheels and a matching PyTorch stack into a + managed environment. Run `rocm examine` again and confirm that + `active_runtime_status` reports `ready`. + +4. Serve a model in the form your engine accepts: + + - `lemonade` (Ryzen AI and Radeon GPUs) serves GGUF models: + + ``` + rocm serve qwen + ``` + + - `vllm` (Instinct GPUs) serves safetensors models: + + ``` + rocm serve Qwen/Qwen2.5-1.5B-Instruct + ``` + + The server runs in the background. When it's ready, `rocm serve` prints a + deployment summary with the endpoint, the service ID, and the commands that + stop the service and show its logs. + +5. Send the model a prompt. Pass the same model name you served: + + ``` + rocm chat --provider local --model qwen --prompt "Explain ROCm in one sentence" + ``` + + On a `vllm` engine, use `--model Qwen/Qwen2.5-1.5B-Instruct`. To open the + interactive chat instead, run `rocm chat` with no options. Chat is a preview + feature. + +6. Stop the server with the `stop` command from the deployment summary, or list + running servers with `rocm services list`. + +For other models and engine options, see +[Configure ROCm and serve a model](#configure-rocm-and-serve-a-model) and +[Model serving](#model-serving). + ## First run Launch ROCm CLI with no arguments: @@ -281,8 +348,12 @@ After the install finishes, serve a model: rocm serve qwen ``` -`qwen` is a built-in alias for a small assistant model that serves out of the -box. You can also serve any compatible Hugging Face model directly. See +`qwen` is a built-in alias for a small GGUF assistant model, served by the +Lemonade engine on Ryzen AI and Radeon GPUs. On Instinct GPUs, which serve +through vLLM, pass a safetensors model instead, such as +`rocm serve Qwen/Qwen2.5-1.5B-Instruct`. + +You can also serve any compatible Hugging Face model directly. See [Model serving](#model-serving) for the GGUF versus safetensors rule, because which form works depends on the engine your GPU selects. diff --git a/docs/rocm-docs/getting-started.md b/docs/rocm-docs/getting-started.md index 46373f7c45..ef93684e6d 100644 --- a/docs/rocm-docs/getting-started.md +++ b/docs/rocm-docs/getting-started.md @@ -6,6 +6,28 @@ SPDX-License-Identifier: MIT # Getting started with ROCm CLI +## Quickstart + + +These steps take a computer with a supported AMD GPU from installing ROCm CLI to +a chat reply from a model served on that GPU. + +1. Install ROCm CLI with the command for your platform in + [Installing ROCm CLI](install/installation.md). Then open a new terminal and + confirm that `rocm version` prints the CLI version. + +```{include} ../../README.md +:start-after: "`rocm version` prints the CLI version." +:end-before: "For other models and engine options, see" +``` + +For other models and engine options, see +[Configure ROCm and serve a model](#configure-rocm-and-serve-a-model) and +[Model serving](commands.md#model-serving). + ## First run ```{include} ../../README.md From 4e076c6bafac91d5ed1079c5e8cc85abc3d5798b Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Thu, 12 Nov 2026 17:07:18 -0500 Subject: [PATCH 2/3] docs: make the quickstart easier to find and follow Name the quickstart on the landing card, show the two `rocm examine` lines the reader should look for, state the supported platforms and the size of the first download up front, show the form of the stop command, and put the CLI demo right after the quickstart. Signed-off-by: pmoutsia_amdeng --- README.md | 29 ++++++++++++++++++++++++----- docs/rocm-docs/getting-started.md | 14 ++++++++++---- docs/rocm-docs/index.md | 2 +- 3 files changed, 35 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 956b66f4fe..11477447ad 100644 --- a/README.md +++ b/README.md @@ -166,8 +166,11 @@ commands, and commit-signing requirements. ## Quickstart -These steps take a computer with a supported AMD GPU from installing ROCm CLI to -a chat reply from a model served on that GPU. +These steps take a computer with a supported AMD GPU, running Linux, WSL2, or +Windows, from installing ROCm CLI to a chat reply from a model served on that +GPU. The first run downloads a lot: `rocm install sdk` fetches a ROCm runtime +and a PyTorch stack, and `rocm serve qwen` fetches a model of about 2.5 GB. Allow +time and disk space. 1. Install ROCm CLI with the command for your platform in [Installation](#installation). Then open a new terminal and confirm that @@ -179,7 +182,17 @@ a chat reply from a model served on that GPU. rocm examine ``` - Two lines in the output decide the next steps: + Two lines in the output decide the next steps. For example, on a Radeon or + Ryzen AI GPU: + + ``` + active_runtime_status: ready + effective_default_engine: lemonade + ``` + + On an Instinct GPU, the second line reads `effective_default_engine: vllm` + instead. vLLM isn't available on native Windows, so Windows always uses + `lemonade`. The lines mean: - `active_runtime_status`: `ready` means a managed ROCm runtime is already active, and you can skip step 3. @@ -224,8 +237,14 @@ a chat reply from a model served on that GPU. interactive chat instead, run `rocm chat` with no options. Chat is a preview feature. -6. Stop the server with the `stop` command from the deployment summary, or list - running servers with `rocm services list`. +6. Stop the server with the `stop` command from the deployment summary. It has + this form: + + ``` + rocm services stop --yes + ``` + + To find the service ID later, run `rocm services list`. For other models and engine options, see [Configure ROCm and serve a model](#configure-rocm-and-serve-a-model) and diff --git a/docs/rocm-docs/getting-started.md b/docs/rocm-docs/getting-started.md index ef93684e6d..365b295535 100644 --- a/docs/rocm-docs/getting-started.md +++ b/docs/rocm-docs/getting-started.md @@ -12,8 +12,11 @@ SPDX-License-Identifier: MIT with the cross-references retargeted to this site. The include between them uses the end of README step 1 and the start of its closing paragraph as anchors; edit both together. --> -These steps take a computer with a supported AMD GPU from installing ROCm CLI to -a chat reply from a model served on that GPU. +These steps take a computer with a supported AMD GPU, running Linux, WSL2, or +Windows, from installing ROCm CLI to a chat reply from a model served on that +GPU. The first run downloads a lot: `rocm install sdk` fetches a ROCm runtime +and a PyTorch stack, and `rocm serve qwen` fetches a model of about 2.5 GB. Allow +time and disk space. 1. Install ROCm CLI with the command for your platform in [Installing ROCm CLI](install/installation.md). Then open a new terminal and @@ -28,6 +31,11 @@ For other models and engine options, see [Configure ROCm and serve a model](#configure-rocm-and-serve-a-model) and [Model serving](commands.md#model-serving). +The following recording shows the CLI inspecting the environment and chatting +with a locally served model: + +![ROCm CLI demo](https://raw.githubusercontent.com/ROCm/rocm-cli/media/cli.gif) + ## First run ```{include} ../../README.md @@ -35,8 +43,6 @@ For other models and engine options, see :end-before: "## Interactive interfaces" ``` -![ROCm CLI demo](https://raw.githubusercontent.com/ROCm/rocm-cli/media/cli.gif) - ## Interactive interfaces ```{include} ../../README.md diff --git a/docs/rocm-docs/index.md b/docs/rocm-docs/index.md index 203072d63d..7c0a9349d8 100644 --- a/docs/rocm-docs/index.md +++ b/docs/rocm-docs/index.md @@ -50,7 +50,7 @@ Check platform support and install the prebuilt binaries. :link: getting-started :link-type: doc -Inspect your environment, configure ROCm, and serve a model. +Quickstart: install ROCm CLI, serve a model, and chat with it. ``` ```{grid-item-card} Look up a command From 7993b262d0ced6ef5c4f955a40f7323f42169acf Mon Sep 17 00:00:00 2001 From: pmoutsia_amdeng Date: Mon, 16 Nov 2026 09:52:25 -0500 Subject: [PATCH 3/3] docs: reword quickstart intro for the first-run download Say "a few GB" for the first model download instead of a figure that isn't sourced in the repo, and cover both engines since the vLLM model is a different size. Reword the intro and add a lead-in before the numbered steps. Signed-off-by: pmoutsia_amdeng --- README.md | 14 ++++++++------ docs/rocm-docs/getting-started.md | 12 +++++++----- 2 files changed, 15 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 11477447ad..d3c45ec460 100644 --- a/README.md +++ b/README.md @@ -166,11 +166,13 @@ commands, and commit-signing requirements. ## Quickstart -These steps take a computer with a supported AMD GPU, running Linux, WSL2, or -Windows, from installing ROCm CLI to a chat reply from a model served on that -GPU. The first run downloads a lot: `rocm install sdk` fetches a ROCm runtime -and a PyTorch stack, and `rocm serve qwen` fetches a model of about 2.5 GB. Allow -time and disk space. +This quickstart takes you from installing ROCm CLI to getting a chat reply from +a model served on your GPU. You need a computer with a supported AMD GPU running +Linux, WSL2, or Windows. The first run involves large downloads: +`rocm install sdk` fetches a ROCm runtime and a PyTorch stack, and `rocm serve` +fetches a model of a few GB. Allow extra time and disk space. + +Follow these steps: 1. Install ROCm CLI with the command for your platform in [Installation](#installation). Then open a new terminal and confirm that @@ -195,7 +197,7 @@ time and disk space. `lemonade`. The lines mean: - `active_runtime_status`: `ready` means a managed ROCm runtime is already - active, and you can skip step 3. + active, so you can skip step 3. - `effective_default_engine`: `lemonade` or `vllm`. The engine decides which model you serve in step 4. diff --git a/docs/rocm-docs/getting-started.md b/docs/rocm-docs/getting-started.md index 365b295535..bf3fea56e4 100644 --- a/docs/rocm-docs/getting-started.md +++ b/docs/rocm-docs/getting-started.md @@ -12,11 +12,13 @@ SPDX-License-Identifier: MIT with the cross-references retargeted to this site. The include between them uses the end of README step 1 and the start of its closing paragraph as anchors; edit both together. --> -These steps take a computer with a supported AMD GPU, running Linux, WSL2, or -Windows, from installing ROCm CLI to a chat reply from a model served on that -GPU. The first run downloads a lot: `rocm install sdk` fetches a ROCm runtime -and a PyTorch stack, and `rocm serve qwen` fetches a model of about 2.5 GB. Allow -time and disk space. +This quickstart takes you from installing ROCm CLI to getting a chat reply from +a model served on your GPU. You need a computer with a supported AMD GPU running +Linux, WSL2, or Windows. The first run involves large downloads: +`rocm install sdk` fetches a ROCm runtime and a PyTorch stack, and `rocm serve` +fetches a model of a few GB. Allow extra time and disk space. + +Follow these steps: 1. Install ROCm CLI with the command for your platform in [Installing ROCm CLI](install/installation.md). Then open a new terminal and