build: parallelize gpuagent build with correct -j dependency graph - #92
Merged
Merged
Conversation
The build could not use make -j: object compilation had no dependency edge to the third-party libs or generated protos, so -j raced ahead and compiled gpuagent before headers/protos existed. The inner build was also invoked without -j at all, forcing serial object compilation (~hundreds of .cc x 3 variants), which dominated build time. - inner Makefile: declare generated *.pb.cc as grouped-target (&:) outputs so the %.o:%.cc pattern rule can resolve every generated .o under -j; add order-only edges so objects wait for generated *.pb.h and the lib stamp without forcing recompiles on proto refresh. - top Makefile: pass make -j$(GPUAGENT_JOBS) (default nproc) to the inner build; override with GPUAGENT_JOBS=N. - Makefile.lib: .NOTPARALLEL: keeps the six third-party libs serial across each other (each already runs make -j$(nproc) internally; concurrent libs would oversubscribe and OOM grpc/boost). - ci.yml: add step summary (per-binary size/sha256), upload the 4 binaries and build.log as artifacts (retention 10d, if: always()). Clean -j8 build verified successful end-to-end (all 3 gpuagent variants + gpuctl); object+link phase ~449s serial -> ~108s parallel. Co-Authored-By: Claude <noreply@anthropic.com>
spraveenio
added a commit
to ROCm/device-metrics-exporter
that referenced
this pull request
Aug 28, 2026
…rallelize staging build (#630) * [NO-JIRA] bump gpuagent to ROCm/gpu-agent @1d816bfe2b42, parallelize staging build (#1581) * [NO-JIRA] bump gpuagent to ROCm/gpu-agent main @1d816bfe2b42, parallelize staging build Bumps GPUAGENT_COMMIT to ROCm/gpu-agent#92 (build: parallelize gpuagent build with correct -j dependency graph), which fixes the inner Makefile so object compilation correctly depends on generated protos/third-party libs and can safely build with -j. Wires -j$(nproc) into the gpuagent-build Dockerfile stage's `make -C sw/nic/gpuagent all` so this repo's staging build actually exercises the new parallel graph instead of building serially. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs: record gpuagent staging build validation + timing make gpuagent-build succeeded at the new pin; all four binaries produced. Compile+link step took 1117.3s (~18m37s) under -j8 on an 8-core host. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> (cherry picked from commit 7700bdff53f280ba2bf7ae76ec019b0ee544652d) * Delete docs-internal/knowledge/plans/2026-08-27-gpuagent-bump-1d816bfe2b42-parallel-build.md --------- Co-authored-by: Praveen Kumar Shanmugam <58961022+spraveenio@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
make -j: object compilation had no dependency edge to the third-party libs or generated protos, so-jraced ahead (compiling before headers/protos existed) and the inner build was invoked without-jat all, forcing serial object compilation across ~hundreds of.cc× 3 variants.Makefile: generated*.pb.ccare now declared as grouped-target (&:, GNU Make ≥4.3) outputs so the%.o:%.ccpattern rule resolves every generated.ounder-j; order-only edges make objects wait for generated*.pb.h+ the lib stamp without forcing recompiles on proto refresh.Makefile: passmake -j$(GPUAGENT_JOBS)(defaultnproc, overrideGPUAGENT_JOBS=N) to the inner build.Makefile.lib:.NOTPARALLEL:keeps the six third-party libs serial across each other (each already runsmake -j$(nproc)internally; concurrent libs would oversubscribe and OOM grpc/boost).ci.yml: adds a step summary (per-binary size/sha256) and uploads the 4 binaries +build.logas artifacts (retention 10d,if: always()).Why
Under
-jthe object files were only siblings ofbuild-libs/gen-protosin the prerequisite lists — serial builds ordered them by luck,-jdid not. On a cold tree this failed at link (missinggogo.grpc.pb.o) because the generated.ccwere never declared as buildable targets.Results
-j8build verified successful end-to-end: all 3 gpuagent variants (gpuagent,gpuagent_gim,gpuagent_mock) +gpuctl.Test plan
make gpuagent(RHEL9 builder) is green and produces all 4 binaries.make docker-shell→ cleanmake -j8 -C sw/nic/gpuagent alllinks successfully.🤖 Generated with Claude Code