[Bizuayeu]

What this changes

GLM-5.3-Flash's CUDA ranks other than 0 take rank 0's NCCL unique id over the ranks' liveness lines (cuda/tp_watch.zig, which open before NCCL), as Flash Next's rank 1 takes its id over its link (families/flashnext/cuda_collective.zig) and the fabric's bootstrap broadcasts rank 0's (fabric/tp/tp.zig). A follower that names NCCL_SOCKET_IFNAME also unsets NCCL_COMM_ID before its first NCCL call.

With NCCL_COMM_ID set, NCCL's bootstrap looks on every rank for a local interface in that address's subnet and does not read NCCL_SOCKET_IFNAME (bootstrapNetInit). On hosts joined only by point-to-point links whose addresses are /32s routed over them, as our three GB10s in a switchless ring, only rank 0's host has one, so ranks 1 and 2 stop there (NET/Socket : No usable listening interface found) and TP=3 does not start. We raised this on #636 (comment), planning to start GLM's ranks through the fabric's Session.open; this is a smaller step that keeps GLM's lines and NCCL_COMM_ID and moves only the id onto the lines. Until now we ran TP=3 with a local commit that passes the id through a file, as the GLM lanes tests' runner does (zig/tests/cuda/glm_lanes_tests.zig, openComm).

What stays: rank 0 makes the id from NCCL_COMM_ID, so the root listens where the operator put it, and NCCL_COMM_ID still names the lines' address on every host. A follower without NCCL_SOCKET_IFNAME keeps NCCL_COMM_ID, and the id it now takes from its line is the one it used to make. For a follower that names NCCL_SOCKET_IFNAME, only the bootstrap moves to that interface: NCCL's net interfaces already took NCCL_SOCKET_IFNAME first. An NCCL_COMM_ID line in ~/.nccl.conf or /etc/nccl.conf comes back at NCCL's first call and keeps the old bootstrap.

The id goes in three records of the lines' 64-byte form, each a part number and up to 48 of the id's 128 bytes. A follower waits for all three up to 15 minutes, as for the ranks' ready records.

Receipt

• Environment: this branch (41e64791) on 1.1.2 (4ff78441), Zig 0.17.0, nvcc 13.0 for sm_121, NCCL 2.30.7; GB10 (sm_121) hosts, one GPU each; checkpoint nvidia/GLM-5.3-Flash-NVFP4 at 423acf37; serve per rank with TF_GLM_RANK, TF_GLM_WORLD and NCCL_COMM_ID, --context 40960 --parallel 1, drafts on.
  • TP=3 on three GB10s in a switchless ring (point-to-point links, /32 addresses). Every rank names NCCL_SOCKET_IFNAME, the interface holding its /32 address, before and after.
  • TP=2 on two GB10s over a direct ConnectX-7 link whose addresses share a subnet.
• Before, v1.1.2 at TP=3 on the ring: rank 1 stops in bootstrapNetInit (NET/Socket : No usable listening interface found), and rank 0 stops with it (rank 1 stopped (ncclGetUniqueId)).
• After, this branch at TP=3 on the ring: all three ranks start, with no NCCL warning in any rank's log (the same NCCL log settings as the before).
• Exactness: a counting, a prose and a code prompt, three greedy replies each, the second and third resumed from the prompt cache, give one token_sha per prompt:
  • at TP=3, the same as v1.1.2 run with glm/cuda: load and serve nvidia/GLM-5.3-Flash-FP4-Local as stored #1050 and our local commit that passes the id in a file;
  • at TP=2, with NCCL_SOCKET_IFNAME on both ranks and on neither, the same as our earlier TP=2 runs of v1.1.1 with two local commits (a heat ceiling and the id file) and of v1.1.2 with glm/cuda: load and serve nvidia/GLM-5.3-Flash-FP4-Local as stored #1050 and the id-file commit; both settings start with no NCCL warning.
• Decode speed before and after: n/a. The change runs once at startup, before ncclCommInitRank; rounds and kernels are unchanged. There is no before at TP=3, where 1.1.2 does not start on the ring. After, on the counting prompt above (512 greedy tokens after a 2,066-token prompt, drafts on; not tools/bench_openai.py): 67.3 tok/s at TP=3, acceptance length 4.65 as on the TP=3 reference, and 37.9 to 38.8 tok/s at TP=2 in both settings.
• Prompt speed before and after: n/a, as above.
• Tests run, the no-GPU workflow's steps on Linux x86-64 from git archive trees of this branch and of 1.1.2:
  • zig build test passes 3,444 of 3,460, with 15 skipped and 1 failing: cuda_cost_identity's "stable reads reuse a drafter, while a changed payload at the same path invalidates it". It fails the same way on 1.1.2, where 3,443 of 3,459 pass.
  • test-kernel-contract, test-golden, test-server-security test-pull-security, fuzz-cortheon and zig build pass.
  • python3 -m unittest discover -s tools/zig -p flashnext_batch_serve_test.py runs 8 tests, all OK.
  • zig fmt --check passes on the two files, and tools/zig/lean_check.py finds no new problem against 1.1.2.
• Not run, and why:
  • Metal, which this CUDA-only path does not reach.
  • zig build test on macOS: the new test runs there, but this repository's macOS job runs the app's npm checks only. On Windows, where the new test skips, the zig-windows check passes on this head.
  • Other CUDA GPUs: every GPU here is a GB10.
  • The GPU tests in tf-cuda-test, which built; the serve runs above go through the startup this changes.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. (The change runs once at startup, before ncclCommInitRank.)
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Resumed equals fresh in the decode check above. Drafts and concurrency run through the unchanged rounds and were not run on this branch.)
• No precision traded for speed. If the bits change, the description says which sums change and why. (No sum or kernel changes; the token ids match, above.)
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt. (No kernel changes.)
• Prompt processing is no slower than the last release. (By reasoning: startup only; no round or kernel changes.)
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not. (CUDA GB10 at TP=3 and TP=2, and the host tests on Linux and, through this head's checks, Windows; Metal, macOS and other CUDA GPUs not run.)
• New tests fail before the change, pass after it, and skip cleanly without their dependency. (The new tp_watch test calls the functions this change adds, so it cannot build before it; it skips on Windows, where this head's checks pass. On the GPU, 1.1.2's rank 1 stops on the ring, as the before above shows.)
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[Bizuayeu]

The description now has the GPU receipt: at TP=3 on the ring, 1.1.2's rank 1 stops in bootstrapNetInit and this branch starts with the same token ids as our id-file run; at TP=2, with NCCL_SOCKET_IFNAME on both ranks and on neither, the token ids are unchanged. Ready for review.

[ashhart]

Landed on main in 81b476d..2ff0191, with your record under your authorship and a maintainer fix-up that keeps bounded rank-ID fanout controls. Thank you, @Bizuayeu. On two GB10s at TP=2 (GLM-5.3-Flash NVFP4, alternating base/new/new/base): tokens equal in 52 of 52 comparisons; prefill and decode within ±0.3%; chat with prompt-cache resume qualified with thinking off and on.

- Synthema, Ash's AI Assistant.

[Bizuayeu]

Landed on main in 81b476d..2ff0191, with your record under your authorship and a maintainer fix-up that keeps bounded rank-ID fanout controls.

#1178 (comment)

Thank you, and for the fanout fix-up. With it our ring of /32 links can bootstrap TP=3 on upstream's own code, and our fork's local NCCL id file can go.