Skip to content

ci: lint and test the rocm e2e-test-hooks build (ROCMAI-479) - #1101

Open
rominf wants to merge 8 commits into
mainfrom
ROCMAI-479-ci-e2e-test-hooks-lint-test
Open

rominf wants to merge 8 commits into
mainfrom
ROCMAI-479-ci-e2e-test-hooks-lint-test

Conversation

@rominf

@rominf rominf commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Why

The rocm unit tests that exist only with the e2e-test-hooks feature (the low-VRAM hook test forced_low_vram_hook_trips_the_serve_plan_warning in gpu_policy.rs, and the system_packages.rs tests that plant a simulated host root) never ran in CI: no step built rocm with the feature on. No clippy run enabled the feature either, so the hooks and their tests were never linted. The only feature-on run in CI was rocm-core's --lib nextest step (cargo nextest run --locked -p rocm-core --lib --features e2e-test-hooks), which runs rocm-core's own library tests and nothing in rocm.

What changes

  • Clippy (e2e-test-hooks) step in the clippy job: cargo clippy --locked --workspace --all-targets --features rocm/e2e-test-hooks -- -D warnings. It is added next to the existing plain workspace pass rather than replacing it: each hook seam has a cfg(not(feature = ...)) fallback that only compiles with the feature off, and a feature-on half (plus the tests behind it) that only compiles with it on. Enabling the feature on rocm turns it on in rocm-core and the engine crates as well, so their seams are linted here too.

  • Test rocm with e2e-test-hooks step in the test job: cargo nextest run --locked -p rocm --features rocm/e2e-test-hooks, gated like the rocm-core step (Rust changed, and -p rocm is in the affected selection or the selection is the whole workspace; the trailing space stops -p rocm-core from matching). It runs the whole rocm suite with the feature on. It uses nextest so that each test is its own process, because the hooks read process-wide variables the feature-only tests set.

  • Contract test ci_lints_and_tests_every_crate_with_e2e_test_hooks_gated_tests in xtask/src/workflow_contract.rs. It walks each workspace crate's module tree the same way check-module-size does (a small #[cfg(test)] helper in module_size.rs exposes that walk, #[path] modules and test-gated mod declarations included). It finds every crate with test code the feature changes: test code holding a cfg(..) or cfg!(..) the feature can turn on or a cfg_attr(..) naming it, test code behind such a gate, or a mod x; behind the feature. It fails when:

    • such a crate has no feature-on nextest step in the test job carrying its selection gate and no continue-on-error;
    • the clippy job lacks either the feature-off or the feature-on workspace clippy run, or one of them is made advisory;
    • a crate with a hook seam is not reached by rocm's e2e-test-hooks feature, so the feature-on clippy step would not lint it;
    • either scan stops finding the crates known to have feature-gated code (rocm, rocm-core, rocm-engine-lemonade for seams; rocm, rocm-core for tests), so the per-crate checks above cannot pass by iterating over nothing. The message names both causes: a broken scan, or a crate whose last such code was removed.

    Steps are matched on their run: value exactly, after removing comments and folding it onto one line, under any YAML block-scalar header (|, >, with any chomping or indentation indicator). An appended || true, a --no-run, a test filter, a narrower target or an echo therefore fails the match, and a step whose name or comment quotes the command does not count.

    The scanner's own tests include a positive-polarity cfg_attr(feature = "e2e-test-hooks", ignore) case alongside the not(..) one, and the module-walk test has a second package that loads the first package's file through #[path] behind a test gate, pinning that each package gets its own file list: that file is test-gated for the package that borrows it and not for its owner. Two seam-scan rows put a cfg and a cfg_attr naming the feature on a file-level inner attribute (#![cfg(..)], #![cfg_attr(..)]), so a scan that read only a file's items would miss them.

    On a pull request the Linux test job runs the contract only when xtask is in the affected selection, but it still runs on every Rust change: the windows-debug job runs cargo test --workspace --all-targets, and the Linux coverage job runs cargo llvm-cov --workspace minus e2e-cucumber.

  • Line-ending independence. The scanner takes each item's tokens from the parsed syntax tree (quote::ToTokens), not by re-lexing the slice of source text the item's span covers. xtask/ has no eol=lf rule in .gitattributes, so a Windows checkout gives it CRLF line endings; a doc comment's span then ends after the \r, and syn strips a byte order mark before recording offsets, so re-lexing by span failed with a LexError on Windows. A regression test runs the scan over CRLF, BOM and CRLF-plus-BOM variants of the same source and expects the same answer. The seam scan reads each file through the same syn::parse_file, via one helper, source_names_feature_in_a_cfg, that the production loop and its tests share, so the two scanners accept the same input. Raw lexing handles CRLF and a byte order mark; the routes differ on a shebang line, which parse_file drops and raw lexing fails on when the line is not valid Rust tokens. No workspace file opens with a shebang or a byte order mark, so this is parity rather than a fix for an observed failure. the_seam_scan_reads_crlf_bom_and_shebang_sources writes LF, CRLF, BOM, BOM+CRLF and shebang variants of a positive and a negative source to real files and runs the production crates_with_hooks_seams over them. quote is a new xtask dev-dependency; it was already in the lockfile through syn.

  • Docs: docs/testing.md lists both clippy runs and the feature-on test steps, says everything the contract checks and which tests those steps do not build (tests gated on the feature being off, and integration tests under tests/), gives the command to run the contract alone, and says which CI jobs run it on a pull request. Its "full local verification" advice now names the feature-on commands alongside cargo test --workspace --all-targets, and it notes that the pre-push hooks build with the feature off. CONTRIBUTING.md says the same next to its list of what prek checks. CONTRIBUTING.md, docs/testing.md, docs/architecture.md, the cargo xtask pre-push-check --help text (which had said CI always checks the whole workspace) and the comments in ci.yml and xtask/src/proptest_regressions_contract.rs say which jobs run the whole workspace's tests on a pull request: windows-debug runs all of them and the Linux coverage job every crate's but e2e-cucumber's, alongside the merge queue. CONTRIBUTING.md, docs/testing.md, ci.yml and the --help text add that this holds for a pull request that changes Rust. The comment on the pre-push banner's CI claim in xtask/src/prepush.rs now separates what the_banner_claim_about_ci_matches_the_workflow checks (the windows-debug job's cargo test --workspace --all-targets, and the test job's --workspace outside pull requests) from the windows-debug step's heavy path filter, which that test does not check. AGENTS.md §8 adds the feature-on clippy line to the minimum quality gate, and rules that the feature-on test steps are not part of that gate: CI's test job runs them, and a change touching code or tests behind e2e-test-hooks runs them by hand. docs/architecture.md updates the workflow_contract.rs entry and notes the test-only walk module_size.rs provides.

There is no Gherkin scenario. This is CI and test plumbing with no change a user of the CLI can observe (AGENTS.md §3).

Verification

The contract test was shown to fail when its target is broken. Each of these mutations of ci.yml or the scanner turned it red:

Mutation Result
Delete the Test rocm with e2e-test-hooks step red
Drop --features from that step red
Change that step's if: gate red
Add continue-on-error, or append || true to the run: red
Delete the Clippy (e2e-test-hooks) step red
Delete the rocm-core feature-on step red
Break individual scanner branches, or the gated-flag fold in the module walk red
Make the seam scan return no crates red: `rocm` is missing from the crates with code behind `e2e-test-hooks`
Put tokens_of back to re-lexing the source slice by span the CRLF/BOM regression test goes red with the same LexError the Windows job raised
Put the seam scan back to raw str::parse red: the shebang rows of the_seam_scan_reads_crlf_bom_and_shebang_sources
Make the seam scan print only the file's items red: the inner-attribute rows

With the low-VRAM hook broken, the new rocm step goes red. A feature-off run of the same suite executes none of the gated tests.

Run locally on Linux:

  • Passed: cargo fmt; cargo test -p xtask (875 tests); both workspace clippy runs (feature off and on); cargo xtask check-module-size, check-architecture-doc and check-crate-edges; cargo nextest run --locked -p rocm --features rocm/e2e-test-hooks; the prek pre-commit and pre-push hooks, lychee included.
  • cargo test --workspace --all-targets failed only on tests that depend on the host, unrelated to this change: two build_script git_watch tests (they fail under global commit signing) and a service_control container-stop test (it fails on a host with a podman-docker shim).
  • Not run to completion: cargo xtask smoke. Its bwrap sandbox step is blocked by the host's user-namespace restrictions.
  • Not run locally: Windows. The CRLF and BOM cases are covered by the regression test on every platform; the Windows CI job is the authoritative run for a real Windows checkout.

Known gaps

  • Windows clippy does not lint feature-on code. The Windows clippy steps build with the feature off, and there is feature-gated code inside cfg(windows) in crates/rocm-core/src/process.rs that no CI clippy step lints. The comment in ci.yml states this.
  • Local pre-push hooks build with the feature off. The feature-on clippy and test steps run in CI, or by hand with the commands in docs/testing.md. AGENTS.md §8 puts the feature-on clippy run in the minimum local gate and leaves the feature-on test steps out of it, asking for them by hand when a change touches code or tests behind the feature.
  • Integration tests are not scanned. The contract walks library, binary, proc-macro and build-script targets, so a feature-gated test under a crate's tests/ directory would not be found. None exists.
  • Exact matching is strict on purpose. Any change to the two pinned run: strings (the rocm test step and the feature-on clippy step) needs a matching edit to the contract test, and removing the last feature-gated code from a crate the sanity lists name needs that crate dropped from the list.

The rocm unit tests that exist only with the e2e-test-hooks feature,
such as forced_low_vram_hook_trips_the_serve_plan_warning, never ran in
CI, and no clippy run enabled the feature, so neither the hooks nor
their tests were linted. Add a nextest step that runs rocm's tests with
the feature on, and a second workspace clippy pass with it on. The
second pass is an addition rather than a replacement because the
cfg(not(feature)) fallbacks only compile with the feature off.

A contract test in xtask requires a feature-on nextest step for every
crate with e2e-test-hooks-gated tests, and both clippy runs, so these
steps cannot silently disappear and a crate that gains its first gated
test needs a step of its own. It matches each step's `run:` value
exactly, so an appended `|| true`, a `--no-run`, a test filter or an
`echo` cannot satisfy it while the tests go unrun. The docs and the
local quality gate list the extra clippy run, and say which tests the
feature-on steps do not build.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor
Security scan found no critical or high findings on lines changed by this pull request

View the security scan run for complete reports.

“New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta.

…o a CRLF checkout does not panic (ROCMAI-479)

The e2e-test-hooks contract test re-lexed a slice of each source file
by an item's span. On a Windows checkout xtask/ has CRLF line endings,
and a doc comment's span ends after the `\r`, so the slice did not
tokenize; a byte order mark shifts the offsets as well, since syn strips
it before recording spans. The test panicked on Windows. Tokens now come
from the parsed node itself through quote's ToTokens, and a regression
test runs the scan over CRLF and BOM variants of the same source.

The step readers recognised a block-scalar header in two places, and
yaml_value_at stripped only `|` and `>-`, so a command under `|-`, `>`
or `|+` did not match. Both now share one recogniser that reads every
chomping and indentation indicator.

The testing and contributing docs now say the pre-push hooks build with
the feature off, describe what the contract flags, and stop counting
the feature-on steps.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
…s (ROCMAI-479)

The contract's per-crate loops pass vacuously if either of its scans
stops finding anything, so it now asserts that both find the crates
known to have code or tests behind e2e-test-hooks. A miss names both
possible causes: a broken scan, or a crate that lost its last such code.

The module-tree walk test gains a second package that loads another
package's file through #[path], pinning that each package gets its own
list, and the scanner test gains a positive-polarity cfg_attr row.

The module doc no longer claims the feature is off in every CI build;
the e2e lanes build binaries with it. The testing docs say what else
the contract checks, how to run it alone, and that it runs on every
Rust change through the windows-debug and coverage jobs even when xtask
is not in the affected selection. The quality gate notes that the
pre-push hooks build with the feature off.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
…uest (ROCMAI-479)

The docs named only the Windows debug job as running the whole
workspace's tests on a pull request, and in places only the merge
queue. The Linux coverage job runs them too, on every Rust change, so a
change confined to one crate still meets the cross-crate contract tests
before merge. Say so in each place that describes the test selection.

State in the AGENTS.md verification matrix that the e2e-test-hooks
feature-on test steps are not part of the minimum local gate, and when
to run them by hand, since the pre-push hooks build with the feature off.

Also repair a doc-comment fragment in module_size.rs that had lost its
subject.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
… and proptest comments (ROCMAI-479)

The previous commit corrected which CI jobs run the whole workspace's
tests on a pull request in the docs, but three code comments still
said the Windows debug job runs on every change, or that only the
merge queue runs the whole workspace. Bring them in line: the Windows
debug job runs on every Rust change, and the Linux coverage job runs
every crate but e2e-cucumber. Qualify the coverage job the same way in
the contributor guide and the module map, which had said it runs the
whole workspace.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
…nd its test check (ROCMAI-479)

The banner comment said its test fails if the CI claim stops being
true, but the test does not look at the windows-debug job's heavy path
filter. State that filter as unchecked context, separate from what the
test guards.

The pre-push-check --help text said CI always checks the whole
workspace, which is false on a pull request: the Linux test job runs
only the affected crates there. Name the jobs that do test the whole
workspace on a pull request, and the merge queue and main.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
…t with a Rust change (ROCMAI-479)

The Windows debug and Linux coverage jobs only run their tests when a pull request changes Rust files, so the help text says so.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
@rominf
rominf marked this pull request as ready for review November 17, 2026 12:07
@rominf
rominf requested a review from a team as a code owner November 17, 2026 12:07
@rominf
rominf requested a review from jussielo-amd November 17, 2026 12:07

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the diff only (no local build or mutation run). I found nothing blocking: the feature-on clippy pass and the gated rocm nextest step match what ROCMAI-479 asks for, and the contract test pins them. CI is green on every required check; CodeQL was still pending when I looked.

A few non-blocking notes:

  • Scope: about 1000 of the 1350 diff lines are the xtask contract scanner, the yaml_value_at refactor and CRLF/BOM handling, plus the coverage-job doc rewrites. I checked the "coverage job runs the whole workspace" claim against origin/main and it holds. Splitting the doc/yaml work out would make this easier to review, but I don't consider it a blocker.
  • BOM in the seam scan: see the inline comment on workflow_contract.rs. The BOM/CRLF regression test only covers has_hooks_only_test_code.
  • CI cost: each test leg now re-runs the whole rocm suite with the feature on (about 1.5 min per leg). Inline note on ci.yml. Fine if intended.
  • Exact-string pinning: the contract pins the two run: strings and one if: expression, so a harmless flag change needs a contract edit. That seems to be the point, so I'm only noting it.
  • Cosmetic: the doc comment on reached_source_files in module_size.rs is missing a blank /// between its two paragraphs.

Known gaps already disclosed in the PR: Windows feature-on clippy, integration tests under tests/, and local pre-push staying feature-off.

This reflects a diff read, not a runtime check.

Comment thread xtask/src/workflow_contract.rs Outdated
files.iter().any(|(file, _)| {
let source = std::fs::read_to_string(file)
.unwrap_or_else(|e| panic!("reading {}: {e}", file.display()));
names_feature_in_a_cfg(&source.parse().expect("source tokenizes"))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

source.parse() re-lexes the raw file with proc_macro2, which is the path the CRLF/BOM comment on tokens_of says is fragile with a leading BOM. The regression test only exercises has_hooks_only_test_code (syn). Could this panic on "source tokenizes" for a BOM-prefixed file on Windows? Consider parsing via syn::parse_file (which strips the BOM) and walking its tokens, or add a BOM/CRLF case for the seam scan. I haven't run this, so it may be a non-issue.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I checked this. The panic did not reproduce: against the locked proc-macro2 1.0.107, raw str::parse lexed LF, CRLF, BOM and BOM+CRLF sources without error, because its from_str_checked strips a leading BOM. The route did still differ from the item scanner's on one input, a shebang line that is not valid Rust tokens: raw lexing fails there, and syn::parse_file drops it.

cf5bb84 routes crates_with_hooks_seams through syn::parse_file, via one helper, source_names_feature_in_a_cfg, that the production loop and its tests share. It adds the_seam_scan_reads_crlf_bom_and_shebang_sources, which writes LF, CRLF, BOM, BOM+CRLF and shebang variants to real files and runs the production function over them; going back to raw parsing turns its shebang rows red. It also adds two inner-attribute rows (#![cfg(..)], #![cfg_attr(..)]) to the seam scan's tests.

Comment thread .github/workflows/ci.yml
needs.changes.outputs.rust == 'true'
&& (steps.sel.outputs.sel == '--workspace'
|| contains(format('{0} ', steps.sel.outputs.sel), '-p rocm '))
run: cargo nextest run --locked -p rocm --features rocm/e2e-test-hooks

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This re-runs the whole rocm suite feature-on in every test leg, which is about 1.5 min extra per leg. Fine if intended; the three feature-only tests could be selected with a filter, at the cost of a looser contract pin.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intended, so no code change. The whole rocm suite runs with the feature on because ungated code also behaves differently in that build (the hooks are compiled in and read environment variables), and that build is what the e2e lanes run. A -E filter would silently drop a test gated on the feature later. The contract pins this exact command so that || true, --no-run or a filter cannot satisfy it. The cost is about 1.5 min per test leg, inside the job's 20-minute timeout.

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — needs work

Full review of the whole change (base f15f9b4bb9c6b50cbebdaa5bafa734502863a22b, head 23514a431313407f3e1e84c02c365c8c0ed60fdc).
Ticket ROCMAI-479 is named in the PR title; not retrieved here (no tracker access from this review). The PR description's detailed account of what changed matches what is actually in the diff, down to file/line level, in everything checked — no scope mismatch found.

Blocking

  • The new production-seam scan panics on a source file that opens with a byte-order mark — the exact hazard this PR hardened the sibling scan against — xtask/src/workflow_contract.rs:10781 (crates_with_hooks_seams), xtask/src/workflow_contract.rs:11095 (test helper mirrors the same gap)
    crates_with_hooks_seams reads a real file via std::fs::read_to_string and feeds it straight to names_feature_in_a_cfg(&source.parse().expect("source tokenizes")), i.e. str::parse::<proc_macro2::TokenStream>(). syn's own documentation states syn::parse_file "discards a leading byte order mark \u{FEFF}" and that plain from_str-style parsing does not; proc-macro2's fallback lexer (is_whitespace in src/parse.rs) only special-cases U+200E/U+200F, not U+FEFF (whose Unicode White_Space property was deliberately removed), so a leading BOM isn't skipped and .expect("source tokenizes") panics instead of returning a value. This PR added crlf_line_endings_and_a_byte_order_mark_do_not_change_the_answer (line 11056) specifically to guard this class of bug, but that test only exercises has_hooks_only_test_code/test_code, which goes through the BOM-safe syn::parse_file + tokens_of/to_token_stream() path. crates_with_hooks_seams — which decides which crates get the new feature-on clippy pass — takes the unsafe raw-parse path instead, and no comment documents this as a deliberate, accepted gap (unlike this PR's other disclosed gaps). It is cheaply testable: the existing CRLF/BOM matrix already covers the sibling function, so extending it to seam is a small addition, not new machinery.
    Confidence 85 · logic · Fix: route crates_with_hooks_seams (and the seam test helper) through the same BOM-safe path as has_hooks_only_test_code — parse with syn::parse_file and use tokens_of/to_token_stream() rather than source.parse::<TokenStream>() directly — and extend the existing CRLF/BOM regression test (or add a sibling) to cover names_feature_in_a_cfg the same way.

Non-blocking

None

Decisions for the author

  • Exact-string pinning of run:/if: values is brittle to re-wrapping — tradeoff
    yaml_value_at's fold always joins with exactly one space regardless of source indentation, so a harmless line-wrap of an if: condition needs a matching edit to selection_gate's literal. Functionally harmless (GitHub Actions expression parsing ignores whitespace) and consistent with the file's pre-existing style; worth confirming this is the intended long-term shape given the scanner already tokenizes elsewhere.
  • Two hardcoded crate-name canary lists (["rocm","rocm-core"] and ["rocm","rocm-core","rocm-engine-lemonade"]) are a self-documented maintenance liability — tradeoff
    Deliberate, and each failure message distinguishes "scanner regressed" from "test code legitimately removed" — reasonable, but confirm it's the intended long-term shape.
  • CI cost: the new Test rocm with e2e-test-hooks step re-runs the whole rocm suite (not only the feature-only tests) on every leg where rocm is in the affected selection — tradeoff, appears intentional (the step's own comment explains why only nextest's per-process isolation makes this safe), not a defect.
  • Windows + e2e-test-hooks is never linted by any CI clippy step — already-disclosed known gap, not retired here since the PR never claimed to close it; confirm it remains acceptable.

Positive signals

  • Token-tree-based scanning (syn/proc_macro2/quote::ToTokens) rather than text/regex structurally rules out false positives from a feature name appearing inside a string literal or comment, with a dedicated test (production_seams_strings_and_other_features_are_not_found) proving it.
  • The -p rocm vs -p rocm-core trailing-space disambiguation is used consistently in both the CI YAML and the scanner, and verified not to cross-fire.
  • reached_source_files's cross-package #[path] fixture is a genuine, non-vacuous regression test for per-package module-walk isolation, traced by hand against a plausible wrong implementation.
  • The new Test rocm with e2e-test-hooks step's own comment candidly documents a subtle, real data race in the low-VRAM hook test design (process-wide env vars read without ScopedTestEnv's lock) and explains why nextest's per-process isolation is what makes it safe, rather than leaving the risk unstated.
  • Several comments explicitly disclose remaining gaps (Windows feature-on clippy, integration tests under tests/, local pre-push staying feature-off) instead of overclaiming coverage.

Deployment notes

None — CI/test plumbing only, no user-observable or production behavior change.

What this covered

Read the full diff (base f15f9b4bb9c6b50cbebdaa5bafa734502863a22b, head 23514a431313407f3e1e84c02c365c8c0ed60fdc) across all 12 changed files, each read completely (not just diff hunks). Two independent background workers covered .github/workflows/ci.yml + xtask/src/workflow_contract.rs, and xtask/src/module_size.rs + xtask/src/main.rs + xtask/Cargo.toml + xtask/src/prepush.rs + xtask/src/proptest_regressions_contract.rs, respectively; the driver directly reviewed AGENTS.md, CONTRIBUTING.md, docs/testing.md, docs/architecture.md and Cargo.lock against the actual CI/scanner code, plus a short ai-instruction-quality sub-check of the 3-line AGENTS.md addition.

No Rust toolchain was available in this sandbox, so verification is by hand-tracing against the actual source and against syn/proc-macro2's own documented behavior, not by compiling or running the test suite. CI's own run on this commit was read from GitHub's check-run API and taken at face value: all required checks completed with success (CodeQL's conclusion is neutral, not a failure).

Design review (Step 7) was folded into the two code-passes' own design-question output rather than run as a separate pass; no question rose to a fundamental approach problem. History/prior-changes passes were not run as fully separate passes beyond what the two workers did incidentally (e.g. tracing the commit that introduced the rocm-core hook step); given the narrow, recent, single-author nature of these files this is not expected to have missed much, but it was not exhaustive.

Procedural note: per this task's required pre-check, the driver (not the fanned-out workers) had to read one existing submitted review's full text before starting, to check whether it already carried this review's own completion marker. That review raised the same byte-order-mark/seam-scan concern as a non-blocking aside; it is reported here as Blocking because it was independently re-derived and verified against the actual code and against syn/proc-macro2's documented behavior rather than taken on that review's word, and because it meets this review's own objective bar for new behavior with no test covering it — not because of where it was first noticed.

Not reconciled against the PR's other discussion/comments beyond what is noted above.


🤖 Automated by an AI agent

Comment thread xtask/src/workflow_contract.rs Outdated
files.iter().any(|(file, _)| {
let source = std::fs::read_to_string(file)
.unwrap_or_else(|e| panic!("reading {}: {e}", file.display()));
names_feature_in_a_cfg(&source.parse().expect("source tokenizes"))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: this production-seam scan panics on a source file that opens with a byte-order mark — the exact hazard this PR hardened the sibling scan against.

crates_with_hooks_seams reads a real file with std::fs::read_to_string and feeds it straight to names_feature_in_a_cfg(&source.parse().expect("source tokenizes")), i.e. str::parse::<proc_macro2::TokenStream>(). syn's own docs state syn::parse_file "discards a leading byte order mark \u{FEFF}" and that plain from_str-style parsing does not; proc-macro2's fallback lexer (is_whitespace in src/parse.rs) only special-cases U+200E/U+200F, not U+FEFF (whose Unicode White_Space property was deliberately removed), so a BOM at the start of the string isn't skipped and .expect("source tokenizes") panics instead of returning a value.

This PR added crlf_line_endings_and_a_byte_order_mark_do_not_change_the_answer (line 11056) specifically to guard this class of bug, but that test only exercises has_hooks_only_test_code/test_code, which goes through the BOM-safe syn::parse_file + tokens_of/to_token_stream() path. crates_with_hooks_seams — which decides which crates get the new feature-on clippy pass — takes the unsafe raw-parse path instead, and no comment documents this as a deliberate, accepted gap.

Suggested fix: route this (and the seam test helper at line 11095, which mirrors the same gap) through syn::parse_file + tokens_of like has_hooks_only_test_code does, and extend the existing CRLF/BOM regression test (or add a sibling) to cover names_feature_in_a_cfg the same way.

Confidence 85 (traced against the actual code plus syn/proc-macro2's own documented behavior; no Rust toolchain was available to execute this).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cf5bb84, though not for the reason given. The BOM panic did not reproduce: proc-macro2 1.0.107 (the locked version) strips a leading BOM in from_str_checked before lexing, and CRLF lexes as whitespace, so raw str::parse handled LF, CRLF, BOM and BOM+CRLF sources. The two routes did differ on a shebang line that is not valid Rust tokens, which raw lexing fails on and syn::parse_file drops.

The seam scan now reads every file through syn::parse_file, as the item scanner does, via one helper, source_names_feature_in_a_cfg, shared by crates_with_hooks_seams and its tests. The sibling test you asked for is the_seam_scan_reads_crlf_bom_and_shebang_sources: it writes LF, CRLF, BOM, BOM+CRLF and shebang variants of a positive and a negative source to real files and runs the production crates_with_hooks_seams over them. Going back to raw parsing turns its shebang rows red. Two inner-attribute rows (#![cfg(..)], #![cfg_attr(..)]) now also pin that the scan reads the file's own attributes, not only its items.

Comment thread xtask/src/workflow_contract.rs Outdated

#[test]
fn the_seam_scan_sees_either_polarity_and_nothing_else() {
let seam = |source: &str| names_feature_in_a_cfg(&source.parse().expect("tokenizes"));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same gap as the inline comment on crates_with_hooks_seams (line 10781): this test helper (let seam = |source: &str| names_feature_in_a_cfg(&source.parse().expect("tokenizes"))) mirrors the production code's unsafe raw str::parse::<TokenStream>() path rather than the BOM-safe syn::parse_file path has_hooks_only_test_code/test_code use. The CRLF/BOM regression test just above (crlf_line_endings_and_a_byte_order_mark_do_not_change_the_answer, line 11056) never exercises seam/names_feature_in_a_cfg, so this gap is untested rather than merely unfixed.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cf5bb84. The seam helper now calls source_names_feature_in_a_cfg, the same function the production loop in crates_with_hooks_seams calls, so the helper and production share one parse route (syn::parse_file). The new the_seam_scan_reads_crlf_bom_and_shebang_sources drives the production function itself on real files with LF, CRLF, BOM, BOM+CRLF and shebang variants, so the scan's file handling is no longer covered only through the helper.

…its inner-attribute path (ROCMAI-479)

The e2e-test-hooks contract has two scanners. The item scanner already
took its tokens from syn::parse_file; the seam scan lexed each file's raw
text instead. Route the seam scan through the same parse, through one
helper that both the production loop and its tests call, so the two
scanners accept the same input. This is parity, not a fix for an
observed failure: no workspace file opens with a shebang or a byte order
mark.

A reported panic on a byte-order-mark-prefixed file did not reproduce:
proc-macro2 1.0.107 strips a BOM before lexing, and CRLF lexes as
whitespace. The difference between the two routes is a shebang line,
which parse_file drops and raw lexing fails on when the line is not
valid Rust tokens.

New coverage:
- A test writes LF, CRLF, BOM, BOM+CRLF and shebang variants of a
  positive and a negative source to real files and runs the production
  crates_with_hooks_seams over them. Raw lexing inside either the loop
  or the helper turns it red, through the shebang rows.
- Two rows put a cfg and a cfg_attr naming the feature on a file-level
  inner attribute, so a scan that printed only the file's items, or
  dropped its inner attributes, now fails.

Also make two doc comments say what the code does. tokens_of describes
what it emits for an item and for a syn::File. reached_source_files
gains the blank line between its paragraphs and says it lists
test-gated files and packages under tests/, unlike the size count.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
@rominf

rominf commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for both reviews. Answering the review bodies here; the inline threads have the detail.

  • Seam scan and a byte order mark (blocking): answered on the inline threads. The panic did not reproduce against the locked proc-macro2, which strips a leading BOM. The seam scan now reads files through syn::parse_file via one helper shared with its tests, and a new test runs crates_with_hooks_seams on LF, CRLF, BOM, BOM+CRLF and shebang files. Both are in cf5bb84.
  • Blank /// in reached_source_files: fixed in cf5bb84.
  • Exact-string pinning of run: and if:: deliberate. It is what makes || true, --no-run and a test filter fail the contract. The cost is that re-wrapping the if: needs a matching contract edit, which I accept.
  • The two canary crate lists: deliberate. They stop the per-crate loops from passing over an empty set, and each failure message names both causes: a broken scan, or a crate whose last such code was removed.
  • CI cost of the feature-on rocm step: intended; reasons on the ci.yml thread.
  • Windows feature-on clippy: stays a disclosed gap, recorded as a follow-up.
  • Splitting the PR: the doc and workflow changes are what the contract test checks, so I'm keeping them as one change.

@rominf
rominf requested a review from juhovainio November 17, 2026 17:28

@juhovainio juhovainio left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — no blocking findings

Full review of the whole change (base f15f9b4bb9c6b50cbebdaa5bafa734502863a22b, head cf5bb84ab6ee36215f9941be560027c89e58910d).
Ticket ROCMAI-479 is named in the PR title; not retrieved here (public repo, no tracker access from this review). The PR description's account of what changed matches the diff.

Blocking

None

Non-blocking

None

Decisions for the author

  • Exact-string pinning of run:/if: values is brittle to re-wrapping — tradeoff. A harmless line-wrap of the pinned if:/run: strings needs a matching contract edit. Already acknowledged by the author as deliberate (it's what makes || true, --no-run and a filter fail the contract); worth keeping in mind as a maintenance cost.
  • Two hardcoded crate-name canary lists (rocm/rocm-core for tests, plus rocm-engine-lemonade for seams) — tradeoff. Verified these match the actual crates carrying feature = "e2e-test-hooks" cfgs today (git grep confirms exactly those three). Each failure message distinguishes "scanner regressed" from "code legitimately removed" — reasonable, confirm it stays the intended shape.
  • rocm_hooks_feature_entries() would mis-fire on Cargo's weak-dependency (dep?/feature) syntax — non-blocking-improvement. The manifest doesn't use that syntax today, so this isn't a live bug, just a latent gap if the feature list's syntax ever changes.
  • CI cost: the new Test rocm with e2e-test-hooks step re-runs the whole rocm suite (not only the feature-gated tests) on every leg where rocm is in the affected selection — tradeoff, intentional per the step's own comment and the author's response to the earlier review thread.
  • Windows clippy does not lint feature-on code — already-disclosed known gap, not closed by this PR and not claimed to be.

Positive signals

  • The byte-order-mark/CRLF panic risk flagged on an earlier commit of this PR (crates_with_hooks_seams raw-parsing file content instead of going through the BOM-safe syn::parse_file path) is fixed in the single commit landed since that review (cf5bb84a): the function now calls a shared helper, source_names_feature_in_a_cfg, that parses via syn::parse_file, and a new regression test (the_seam_scan_reads_crlf_bom_and_shebang_sources) drives the real production function over real LF/CRLF/BOM/BOM+CRLF/shebang files. Independently re-verified (not taken on the author's word): no raw-token-parse call site remains in the file, and the test genuinely exercises the production path rather than a stand-in.
  • The test-only module-tree walk added to module_size.rs (ReachedPackage/reached_source_files_in) literally reuses the same Walk/visit() machinery as the production check-module-size path rather than re-implementing it, so the two scans cannot silently drift apart.
  • Several comments candidly disclose remaining gaps (Windows feature-on clippy, integration tests under tests/, local pre-push staying feature-off) instead of overclaiming coverage.
  • Doc updates across AGENTS.md, CONTRIBUTING.md, docs/testing.md and docs/architecture.md were cross-checked against the actual ci.yml and coverage.rs behavior and found factually accurate, including the more subtle claims about which CI jobs run the whole workspace's tests on a pull request.

Deployment notes

None — CI/test plumbing only, no user-observable or production behavior change.

What this covered

Read the full diff (base f15f9b4bb9c6b50cbebdaa5bafa734502863a22b, head cf5bb84ab6ee36215f9941be560027c89e58910d) across all 12 changed files. Two independent background workers covered .github/workflows/ci.yml + xtask/src/workflow_contract.rs, and xtask/src/module_size.rs + xtask/src/main.rs + xtask/src/prepush.rs + xtask/src/proptest_regressions_contract.rs + xtask/Cargo.toml + Cargo.lock, respectively, each reading every file in scope completely. The driver directly reviewed AGENTS.md, CONTRIBUTING.md, docs/testing.md and docs/architecture.md against the actual CI/scanner code, and read the single fix commit (cf5bb84a) in full.

No Rust toolchain was available in this sandbox, so verification is by hand-tracing against the actual source rather than compiling or running the test suite. CI's own run on this commit was read from GitHub's check-run API after it finished: all 41 check runs completed, all success or neutral (CodeQL's conclusion is neutral, not a failure) — no failing or pending required check.

Design review (Step 7) turned up no fundamental approach problem; the open design questions (exact-string pinning, hardcoded canary lists, CI cost) are tradeoffs the author had already made deliberately and explained in the existing review thread, which this review independently agrees are reasonable rather than importing unverified. History/prior-changes passes: workflow_contract.rs's own git log shows only unrelated main-branch commits, nothing specific to this feature's prior iterations beyond this PR's own review thread.

Reconciled against the PR's own description and the existing review threads/comments (one human reviewer, one prior automated review that went CHANGES_REQUESTED on an earlier commit): the earlier review's one blocking finding was independently re-derived and re-verified against the current head's actual code rather than taken on anyone's word, and found fixed.


🤖 Automated by an AI agent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants