Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 94 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,94 @@ install -m 0755 target/release/rocm target/release/rocmd ~/.local/bin/
See [CONTRIBUTING.md](https://github.com/ROCm/rocm-cli/blob/main/CONTRIBUTING.md) for the full development setup, test
commands, and commit-signing requirements.

## Quickstart

This quickstart takes you from installing ROCm CLI to getting a chat reply from
a model served on your GPU. You need a computer with a supported AMD GPU running
Linux, WSL2, or Windows. The first run involves large downloads:
`rocm install sdk` fetches a ROCm runtime and a PyTorch stack, and `rocm serve`
fetches a model of a few GB. Allow extra time and disk space.

Follow these steps:

1. Install ROCm CLI with the command for your platform in
[Installation](#installation). Then open a new terminal and confirm that
`rocm version` prints the CLI version.

2. Check the GPU, the ROCm runtime, and the serving engine:

```
rocm examine
```

Two lines in the output decide the next steps. For example, on a Radeon or
Ryzen AI GPU:
Comment thread
pmoutsias-amd marked this conversation as resolved.
Resolved

```
active_runtime_status: ready
effective_default_engine: lemonade
```

On an Instinct GPU, the second line reads `effective_default_engine: vllm`
instead. vLLM isn't available on native Windows, so Windows always uses
`lemonade`. The lines mean:
Comment thread
pmoutsias-amd marked this conversation as resolved.
Resolved

- `active_runtime_status`: `ready` means a managed ROCm runtime is already
active, so you can skip step 3.
- `effective_default_engine`: `lemonade` or `vllm`. The engine decides which
model you serve in step 4.
Comment thread
pmoutsias-amd marked this conversation as resolved.
Resolved

3. Install a managed ROCm runtime:

```
rocm install sdk
```

This downloads TheRock ROCm wheels and a matching PyTorch stack into a
managed environment. Run `rocm examine` again and confirm that
`active_runtime_status` reports `ready`.

4. Serve a model in the form your engine accepts:

- `lemonade` (Ryzen AI and Radeon GPUs) serves GGUF models:

```
rocm serve qwen
```

- `vllm` (Instinct GPUs) serves safetensors models:

```
rocm serve Qwen/Qwen2.5-1.5B-Instruct
```

The server runs in the background. When it's ready, `rocm serve` prints a
deployment summary with the endpoint, the service ID, and the commands that
stop the service and show its logs.

5. Send the model a prompt. Pass the same model name you served:

```
rocm chat --provider local --model qwen --prompt "Explain ROCm in one sentence"
```

On a `vllm` engine, use `--model Qwen/Qwen2.5-1.5B-Instruct`. To open the
interactive chat instead, run `rocm chat` with no options. Chat is a preview
feature.

6. Stop the server with the `stop` command from the deployment summary. It has
this form:

```
rocm services stop <service-id> --yes
```

To find the service ID later, run `rocm services list`.

For other models and engine options, see
[Configure ROCm and serve a model](#configure-rocm-and-serve-a-model) and
[Model serving](#model-serving).

## First run

Launch ROCm CLI with no arguments:
Expand Down Expand Up @@ -281,8 +369,12 @@ After the install finishes, serve a model:
rocm serve qwen
```

`qwen` is a built-in alias for a small assistant model that serves out of the
box. You can also serve any compatible Hugging Face model directly. See
`qwen` is a built-in alias for a small GGUF assistant model, served by the
Lemonade engine on Ryzen AI and Radeon GPUs. On Instinct GPUs, which serve
through vLLM, pass a safetensors model instead, such as
`rocm serve Qwen/Qwen2.5-1.5B-Instruct`.

You can also serve any compatible Hugging Face model directly. See
[Model serving](#model-serving) for the GGUF versus safetensors rule, because which
form works depends on the engine your GPU selects.

Expand Down
34 changes: 32 additions & 2 deletions docs/rocm-docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,45 @@ SPDX-License-Identifier: MIT

# Getting started with ROCm CLI

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As suggested in the comment on TOC, two headings are not required here. We can call this doc "Quickstart guide" or something similar.


## Quickstart

<!-- Step 1 and the closing paragraph are deliberate copies of the README text,
with the cross-references retargeted to this site. The include between
them uses the end of README step 1 and the start of its closing paragraph
as anchors; edit both together. -->
This quickstart takes you from installing ROCm CLI to getting a chat reply from
a model served on your GPU. You need a computer with a supported AMD GPU running
Linux, WSL2, or Windows. The first run involves large downloads:
`rocm install sdk` fetches a ROCm runtime and a PyTorch stack, and `rocm serve`
fetches a model of a few GB. Allow extra time and disk space.

Follow these steps:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Follow these steps, starting with install and ending with your first chat reply:

1. Install ROCm CLI with the command for your platform in
[Installing ROCm CLI](install/installation.md). Then open a new terminal and
confirm that `rocm version` prints the CLI version.
Comment on lines +23 to +25

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Install ROCm CLI with the command for your platform in
[Installing ROCm CLI](install/installation.md). Then open a new terminal and
confirm that `rocm version` prints the CLI version.
1. Find the install command for your platform in [Installing ROCm CLI](install/installation.md) and run it. Then open a new terminal and confirm that rocm version prints the CLI version.


```{include} ../../README.md
:start-after: "`rocm version` prints the CLI version."
:end-before: "For other models and engine options, see"
```

For other models and engine options, see
[Configure ROCm and serve a model](#configure-rocm-and-serve-a-model) and
[Model serving](commands.md#model-serving).

The following recording shows the CLI inspecting the environment and chatting
with a locally served model:

![ROCm CLI demo](https://raw.githubusercontent.com/ROCm/rocm-cli/media/cli.gif)

## First run

```{include} ../../README.md
:start-after: "## First run"
:end-before: "## Interactive interfaces"
```

![ROCm CLI demo](https://raw.githubusercontent.com/ROCm/rocm-cli/media/cli.gif)

## Interactive interfaces

```{include} ../../README.md
Expand Down
2 changes: 1 addition & 1 deletion docs/rocm-docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Check platform support and install the prebuilt binaries.
:link: getting-started
:link-type: doc

Inspect your environment, configure ROCm, and serve a model.
Quickstart: install ROCm CLI, serve a model, and chat with it.
```

```{grid-item-card} Look up a command
Expand Down
Loading