From 870c051258cdda3a39057ad785d646fa19ee7053 Mon Sep 17 00:00:00 2001 From: pfm <2710205046@qq.com> Date: Thu, 27 Aug 2026 19:04:03 +0800 Subject: [PATCH] docs: add NVLink pair diagnostic guidance Signed-off-by: pfm <2710205046@qq.com> --- README.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/README.md b/README.md index c7e9c22..b4fe333 100644 --- a/README.md +++ b/README.md @@ -132,6 +132,35 @@ Note that output from different nodes may be interleaved; redirect per-host outp A small buffer (e.g. `-b 1`) is sufficient for quickly verifying connectivity between two GPUs. Use the default 512 MiB buffer (or omit `-b` entirely) when measuring peak bandwidth, as smaller buffers may underreport achievable throughput. +### Diagnosing an NVLink pair + +Use `--pair` with a device-to-device test to isolate one NVLink-connected GPU +pair before comparing it with the full matrix. First identify the pair from the +topology reported by `nvidia-smi topo -m`, then obtain the corresponding UUIDs +with `nvidia-smi -L`. + +```bash +nvidia-smi topo -m +nvidia-smi -L + +./nvbandwidth -t device_to_device_memcpy_read_sm \ + --pair +``` + +Run the same pair in the opposite direction to expose directional asymmetry: + +```bash +./nvbandwidth -t device_to_device_memcpy_read_sm \ + --pair +``` + +The reported GB/s is measured bandwidth for the selected copy pattern, not a +guaranteed NVLink link rate. Compare like for like: keep the testcase, buffer +size, sample count, driver, and GPU clocks unchanged. For a concurrent +two-direction workload, use a `device_to_device_bidirectional_memcpy_*` +testcase; its reported value is per-direction bandwidth, not the sum of both +directions. + ### Single-node pairwise test Pass both UUIDs directly. The matrix output will show only the row/column entries for those two devices; all other pairs are skipped.