Skip to content

feat: log available_parallelism() and version, structured, at startup - #765

Merged
brawer merged 1 commit into
mainfrom
log-available-parallelism
Aug 24, 2026
Merged

brawer merged 1 commit into
mainfrom
log-available-parallelism

Conversation

@brawer

@brawer brawer commented Aug 24, 2026 •

Copy link
Copy Markdown
Owner

Why

Every internal thread pool (prune.rs/assemble.rs's own num_workers, Rayon's default global pool for conflate.match) sizes itself from std::thread::available_parallelism() -- which historically ignored Docker/podman's --cpus (a CFS bandwidth quota, not a cpuset) and just reported the host's full core count. Rust's std did fix this for cgroup v2 (rust-lang/rust#89310), and this toolchain (1.98) is well past that fix, but it's exactly the kind of library behavior worth confirming empirically rather than trusting -- this is that confirmation mechanism, for whenever a --cpus sweep (see docs/PRODUCTION.md) actually happens: log the value once, structured, and check it matches N under a real --cpus=N.

What

Extends logging::init()'s existing one-time startup log record rather than adding a second one -- already the natural, only-runs-once home for this kind of environment fact.

While touching this call site: also moves the pipeline version out of the message text and into a structured field, matching this module's own documented convention (logging.rs's own doc comment: "Call sites that have actual structured data to log ... should attach it via the log crate's key-value API instead of interpolating it into the message text") -- the version was doing exactly that before. Keeps the "vX.Y.Z" form (concat!("v", env!("CARGO_PKG_VERSION")), a zero-cost compile-time concatenation of two string literals, not a runtime format!) to match both the previously-logged string and the release tag format (see RELEASING.md) -- not the bare semver pipeline::provenance's tool_component() uses, which is a different field for a different purpose.

Testing

cargo test (253 passed, extended the existing startup-record test to check both new structured fields instead of parsing the old message string)/clippy/fmt clean. Manually ran against the zugerland.osm.pbf fixture and confirmed the real JSON shape:

{"fields":{"available_parallelism":10,"version":"v0.8.2"},...,"message":"pipeline starting up",...}

🤖 Generated with Claude Code

@github-actions github-actions Bot added the enhancement New feature or request label Aug 24, 2026
@brawer
brawer force-pushed the log-available-parallelism branch from dacda93 to 2e3ef39 Compare August 24, 2026 18:30
@brawer brawer changed the title feat: log available_parallelism() once at pipeline startup feat: log available_parallelism() and version, structured, at startup Aug 24, 2026
Every internal thread pool (prune.rs/assemble.rs's own num_workers,
Rayon's default global pool for conflate.match) sizes itself from
std::thread::available_parallelism() -- which historically ignored
Docker/podman's --cpus (a CFS bandwidth quota, not a cpuset) and just
reported the host's full core count. Rust's std did fix this for
cgroup v2 (rust-lang/rust#89310), and this toolchain (1.98) is well
past that fix, but it's exactly the kind of library behavior worth
confirming empirically rather than trusting -- this is that
confirmation mechanism, for whenever a --cpus sweep (see
docs/PRODUCTION.md) actually happens: log the value once, structured,
and check it matches N under a real --cpus=N.

While touching this call site: also moves the pipeline version out of
the message text and into a structured field, matching this module's
own documented convention (logging.rs's own doc comment: "Call sites
that have actual structured data to log ... should attach it via the
log crate's key-value API instead of interpolating it into the
message text") -- the version was doing exactly that before. Keeps the
"vX.Y.Z" form (concat!("v", env!("CARGO_PKG_VERSION")), a zero-cost
compile-time concatenation of two string literals, not a runtime
format!) to match both the previously-logged string and the release
tag format (see RELEASING.md) -- not the bare semver
pipeline::provenance's tool_component() uses, which is a different
field for a different purpose.

Extends logging::init()'s existing one-time startup log record rather
than adding a second one -- already the natural, only-runs-once home
for this kind of environment fact.

Verified: cargo test (253 passed, extended the existing startup-record
test to check both new structured fields instead of parsing the old
message string)/clippy/fmt clean. Manually ran against the
zugerland.osm.pbf fixture and confirmed the real JSON shape:
{"fields":{"available_parallelism":10,"version":"v0.8.2"},...,
"message":"pipeline starting up",...}

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@brawer
brawer force-pushed the log-available-parallelism branch from 2e3ef39 to 1b5323a Compare August 24, 2026 18:47
@brawer
brawer added this pull request to the merge queue Aug 24, 2026
Merged via the queue into main with commit 3cc4108 Aug 24, 2026
6 checks passed
@brawer
brawer deleted the log-available-parallelism branch August 24, 2026 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant