Skip to content

tp: keep verbs load failures in hca diagnostics (carried from #636) - #1127

Closed
jayleaton wants to merge 2 commits into
ashhart:mainfrom
jayleaton:carry/tp-verbs-diagnostics
Closed

jayleaton wants to merge 2 commits into
ashhart:mainfrom
jayleaton:carry/tp-verbs-diagnostics

Conversation

@jayleaton

Copy link
Copy Markdown
Contributor

What this changes

tf-tp-test hca keeps the reason libibverbs failed to load, instead of reporting an absent HCA port. A missing library (LibraryUnavailable) or missing symbols (MissingSymbol) is named as such. An empty successful scan still reports NoRoceDevice, and the peer's port exchange still completes.

This is the second of three carries from #636, as asked in its closing comment (#636 (comment)): the verbs-library error retention and diagnostics from @cwschroeder's series (4df28ce, 0a80da0). Both commits keep @cwschroeder as author, with -x provenance and their Reported-by trailers. They cherry-picked onto main unchanged.

 select → selectWithOpen(Verbs.open)
-  open failed → ports = []          → hcaLine: NoRoceDevice
+  open failed → ports = [], Pick.verbs_open_error = e → hcaLine: e (LibraryUnavailable | MissingSymbol)
  • zig/src/fabric/tp/roce_pick.zig: Pick.verbs_open_error, an injectable open for tests, and HcaError widened by the verbs errors. New controls: a load failure is kept while the peer's port exchange completes, and missing symbols are told apart from an empty successful scan.
  • zig/src/fabric/tp/gpu_main.zig: hca names each failure, and adds the TF_TP_ROCE_HCA / NCCL_IB_HCA hint only where it applies.

Receipt

  • Environment: GitHub Actions on this PR, Zig 0.17.0, base d1018b31.
  • Tests: the new roce_pick tests run in zig build test on Linux (test-tp); results are in this PR's checks.
  • Not run: no RDMA hardware run; no hardware claim. No local Zig build.

Checklist

  • Every token still goes through the lane rounds. (Diagnostics only.)
  • No precision traded for speed. No bits change.
  • No GPU kernel added or changed.
  • New tests fail before the change, pass after it. (Not re-run against the old code here; there hcaLine returns NoRoceDevice for a failed load.)
  • Comments and docstrings are one line.
  • No personal data, machine names, internal hosts or local paths. No AI attribution lines.

(cherry picked from commit 569577a)
Reported-by: Christian Schröder <ashhart#636 (comment)>
(cherry picked from commit 4df28ce)
(cherry picked from commit 8f75a6a)
Reported-by: Christian Schröder <ashhart#636 (comment)>
(cherry picked from commit 0a80da0)
@jayleaton
jayleaton marked this pull request as ready for review October 20, 2026 20:04
@ashhart

ashhart commented Oct 21, 2026

Copy link
Copy Markdown
Owner

Thank you for carrying this from #636. It is on main as 9ca00ae, with @cwschroeder as author and you as co-author: tf-tp-test hca now names a libibverbs load failure instead of reporting an absent RoCE port, and keeps an explicit port list when the scan cannot open the library. On our side the diagnostics also cover the release runtime's VerbsUnavailable error, beside your loader cases. It ships with the next release. Closing as landed.

- Synthema, Ash's AI Assistant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants