Skip to content

fix(cargo-anvil): scope coverage reports to affected packages - #190

Merged
martin-kolinek merged 7 commits into
mainfrom
fix/anvil-coverage-report-scope
Sep 18, 2026
Merged

martin-kolinek merged 7 commits into
mainfrom
fix/anvil-coverage-report-scope

Conversation

@martin-kolinek

Copy link
Copy Markdown
Collaborator

🤖 Scope generated LCOV exports to the same affected-package set used for test execution and cargo-coverage-gate.

Problem

anvil-llvm-cov runs tests for impacted packages but currently invokes cargo llvm-cov report without those package selectors. The exported LCOV therefore includes instrumented, unselected workspace dependencies whose own tests did not run. cargo-coverage-gate filters them back out, while Codecov and ADO ingest the unfiltered report and can report misleading project and patch coverage failures.

This was observed in microsoft/oxidizer#767: every affected package passed the local 100% gate, while Codecov reported 97.8% project coverage from the same run.

Change

  • pass the affected package arguments to each cargo llvm-cov report invocation
  • preserve --workspace behavior for unscoped runs
  • retain the Windows error-206 response-file fallback
  • document that exported reports share the gate's package scope
  • add a recipe contract test and refresh generated backend snapshots

Validation

  • targeted coverage-scope recipe contract
  • all 49 recipe contract tests with one test thread
  • generated backend snapshot tests
  • cargo-anvil Clippy with all targets/features
  • workspace formatting check

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 17, 2026 14:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The checked-in recipe and lock are stale, and workspace-mode report behavior lacks contract coverage.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Scopes LCOV exports to the affected packages used for testing and coverage gating.

Changes:

  • Pass package selectors to cargo llvm-cov report.
  • Preserve workspace behavior and Windows fallback.
  • Add contract coverage, update snapshots, and document the scope.
File summaries
File Summary
crates/cargo-anvil/tests/snapshots/snapshots__local_only.snap Refreshes local generated output.
crates/cargo-anvil/tests/snapshots/snapshots__github_backend.snap Refreshes GitHub generated output.
crates/cargo-anvil/tests/snapshots/snapshots__ado_backend.snap Refreshes ADO generated output.
crates/cargo-anvil/tests/recipe_contracts.rs Adds affected-package report-scope coverage.
crates/cargo-anvil/templates/justfiles/anvil/checks/llvm-cov.just Scopes reports; critical: regenerate the checked-in recipe and lock, and moderate: add workspace-mode coverage.
crates/cargo-anvil/docs/design/checks.md Documents scoped coverage reports.
Review details
  • Files reviewed: 6/6 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/llvm-cov.just
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/llvm-cov.just
@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Sep 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.6%. Comparing base (1910173) to head (f34a0a3).

❌ Your project status has failed because the head coverage (97.6%) is below the target coverage (100.0%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@          Coverage Diff          @@
##            main    #190   +/-   ##
=====================================
  Coverage   97.6%   97.6%           
=====================================
  Files        304     304           
  Lines      69683   69683           
=====================================
+ Hits       68016   68018    +2     
+ Misses      1667    1665    -2     
Flag Coverage Δ
linux 97.5% <ø> (+<0.1%) ⬆️
linux-arm 97.5% <ø> (ø)
scheduled ?
windows 97.7% <ø> (+<0.1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 17, 2026 15:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No unresolved blocking issues are identified, and the changes include contract tests and generated updates.

Review details
  • Files reviewed: 7/8 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 17, 2026 15:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Revert the version bump or complete the repository release process, including changelog updates.

Get a fresh assessment by requesting another Copilot review.

Review details
  • Files reviewed: 9/11 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread crates/cargo-anvil/Cargo.toml Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 17, 2026 16:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No unresolved review comments remain, and the submitted validation covers the changes.

Review details
  • Files reviewed: 10/12 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 17, 2026 16:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No unresolved review comments remain, and the supplied assessments indicate the changes are ready.

Review details
  • Files reviewed: 10/12 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@github-actions

github-actions Bot commented Sep 17, 2026

Copy link
Copy Markdown

⚠️ SemVer check advisory

Inconclusive comparisons

cargo semver-checks could not complete the following comparisons. These failures are informational because an unbuildable baseline is not evidence of a breaking API change.

cargo-aprz-lib (exit 101)

     Cloning 1910173d2c5eb596341960371a2ca25957292c4d
    Building cargo-aprz-lib v1.1.2 (current)
error: running cargo-doc on crate 'cargo-aprz-lib' failed with output:
-----
   Compiling proc-macro2 v1.0.107
   Compiling quote v1.0.47
   Compiling unicode-ident v1.0.26
   Compiling libc v0.2.189
    Checking memchr v2.8.3
    Checking cfg-if v1.0.5
    Checking smallvec v1.16.1
    Checking bytes v1.12.1
    Checking once_cell v1.21.4
    Checking regex-syntax v0.8.11
   Compiling thiserror v2.0.20
   Compiling syn v3.0.6
    Checking stable_deref_trait v1.2.1
   Compiling crc32fast v1.5.2
    Checking scopeguard v1.2.0
   Compiling crossbeam-utils v0.8.23
    Checking regex-automata v0.4.18
   Compiling parking_lot_core v0.9.12
    Checking zlib-rs v0.6.8
   Compiling getrandom v0.4.3
    Checking gix-trace v0.1.21
    Checking bitflags v2.13.2
    Checking fastrand v2.5.0
    Checking lock_api v0.4.14
   Compiling serde_core v1.0.229
    Checking tinyvec v1.13.3
    Checking parking_lot v0.12.5
    Checking bstr v1.13.1
    Checking unicode-normalization v0.1.25
    Checking itoa v1.0.18
    Checking gix-utils v0.3.6
    Checking gix-validate v0.11.4
    Checking crossbeam-channel v0.5.17
    Checking same-file v1.0.6
    Checking walkdir v2.5.0
    Checking allocator-api2 v0.2.21
    Checking foldhash v0.2.0
    Checking equivalent v1.0.2
    Checking byteorder v1.5.0
   Compiling find-msvc-tools v0.1.12
   Compiling shlex v2.0.1
    Checking prodash v31.0.0
   Compiling jobserver v0.1.35
    Checking gix-error v0.2.5
   Compiling cc v1.4.6
   Compiling version_check v0.9.5
   Compiling pkg-config v0.3.34
   Compiling generic-array v0.14.7
    Checking hashbrown v0.17.1
    Checking typenum v1.20.1
   Compiling heapless v0.8.0
    Checking hash32 v0.3.1
   Compiling thiserror-impl v2.0.20
    Checking faster-hex v0.10.0
    Checking cpufeatures v0.2.17
   Compiling serde v1.0.229
   Compiling serde_derive v1.0.229
    Checking pin-project-lite v0.2.17
    Checking crypto-common v0.1.7
    Checking block-buffer v0.10.4
    Checking jiff-core v0.1.1
    Checking digest v0.10.7
    Checking sha1 v0.10.7
    Checking sha1-checked v0.10.0
   Compiling synstructure v0.14.0
    Checking gix-path v0.12.6
    Checking gix-features v0.48.1
    Checking jiff v0.2.37
   Compiling rustix v1.1.5
    Checking gix-hash v0.25.1
    Checking linux-raw-sys v0.12.1
   Compiling zerofrom-derive v0.1.8
    Checking zerofrom v0.1.8
   Compiling yoke-derive v0.8.3
    Checking gix-date v0.15.6
   Compiling autocfg v1.5.1
    Checking gix-actor v0.41.2
    Checking gix-hashtable v0.15.2
    Checking futures-core v0.3.34
    Checking gix-object v0.61.0
   Compiling zerovec-derive v0.11.6
    Checking yoke v0.8.3
    Checking errno v0.3.14
   Compiling tokio-macros v2.7.2
    Checking signal-hook-registry v1.4.8
   Compiling displaydoc v0.2.7
    Checking memmap2 v0.9.11
    Checking mio v1.2.3
    Checking socket2 v0.6.5
   Compiling cmake v0.1.58
   Compiling dunce v1.0.5
    Checking tokio v1.53.1
   Compiling fs_extra v1.3.0
    Checking zerovec v0.11.8
   Compiling rustversion v1.0.23
   Compiling aws-lc-sys v0.45.0
    Checking futures-sink v0.3.34
    Checking tinystr v0.8.4
    Checking tempfile v3.27.0
    Checking gix-fs v0.21.2
    Checking gix-chunk v0.7.3
    Checking percent-encoding v2.3.2
    Checking writeable v0.6.4
   Compiling vcpkg v0.2.15
    Checking litemap v0.8.3
    Checking icu_locale_core v2.3.0
   Compiling libz-sys v1.1.29
    Checking gix-tempfile v23.0.2
    Checking potential_utf v0.1.6
    Checking zerotrie v0.2.5
    Checking gix-quote v0.7.2
    Checking tracing-core v0.1.36
    Checking nonempty v0.12.0
    Checking zeroize v1.9.0
    Checking utf8_iter v1.0.4
   Compiling aws-lc-rs v1.18.1
    Checking slab v0.4.12
   Compiling icu_normalizer_data v2.3.0
   Compiling icu_properties_data v2.3.0
    Checking simd-adler32 v0.3.10
    Checking tracing v0.1.44
    Checking icu_collections v2.3.0
    Checking icu_provider v2.3.1
    Checking http v1.5.0
    Checking adler2 v2.0.1
    Checking fnv v1.0.7
    Checking miniz_oxide v0.9.1
    Checking rustls-pki-types v1.15.1
    Checking gix-commitgraph v0.37.1
    Checking gix-glob v0.26.1
    Checking indexmap v2.14.2
    Checking futures-task v0.3.34
    Checking log v0.4.34
    Checking futures-io v0.3.34
    Checking futures-util v0.3.34
    Checking gix-revwalk v0.32.0
    Checking tokio-util v0.7.19
    Checking http-body v1.1.0
    Checking icu_properties v2.3.0
    Checking icu_normalizer v2.3.0
    Checking flate2 v1.1.10
    Checking gix-lock v23.0.1
   Compiling num-traits v0.2.19
   Compiling syn v2.0.119
    Checking untrusted v0.9.0
   Compiling zmij v1.0.23
   Compiling rustls v0.23.45
   Compiling httparse v1.10.1
    Checking idna_adapter v1.2.2
   Compiling encoding_rs v0.8.41
    Checking futures-channel v0.3.34
    Checking static_assertions v1.1.0
    Checking unicode-bom v2.0.3
    Checking try-lock v0.2.5
    Checking tower-service v0.3.3
    Checking subtle v2.6.1
    Checking tower-layer v0.3.3
    Checking atomic-waker v1.1.2
    Checking h2 v0.4.19
    Checking want v0.3.1
    Checking idna v1.1.0
    Checking form_urlencoded v1.2.2
    Checking gix-config-value v0.18.1
    Checking core_detect v1.0.0
   Compiling multiversion_no_op v1.0.0
    Checking shell-words v1.1.1
    Checking simdutf8 v0.1.5
   Compiling serde_json v1.0.151
    Checking compression-core v0.4.33
    Checking compression-codecs v0.4.43
    Checking hyper v1.11.1
    Checking gix-command v0.9.2
    Checking url v2.5.8
    Checking kstring v2.0.4
    Checking sync_wrapper v1.0.2
    Checking gix-sec v0.14.2
    Checking openssl-probe v0.2.1
    Checking base64 v0.22.1
    Checking ipnet v2.12.2
    Checking rustls-native-certs v0.8.4
    Checking tower v0.5.3
    Checking gix-attributes v0.33.2
    Checking hyper-util v0.1.20
    Checking async-compression v0.4.48
    Checking http-body-util v0.1.5
   Compiling heck v0.5.0
   Compiling crossbeam-epoch v0.9.21
    Checking tower-http v0.6.11
    Checking gix-ref v0.64.0
    Checking gix-url v0.36.2
    Checking gix-packetline v0.21.5
    Checking filetime v0.2.29
   Compiling crossbeam-deque v0.8.8
    Checking base64 v0.23.1
    Checking mime v0.3.17
    Checking either v1.18.0
    Checking gix-prompt v0.15.1
    Checking gix-traverse v0.58.0
    Checking gix-revision v0.46.0
   Compiling zstd-sys v2.1.0+zstd.1.5.7
    Checking gix-bitmap v0.3.3
    Checking hashbrown v0.16.1
    Checking fast_time v0.1.32
    Checking arrayvec v0.7.8
   Compiling rayon-core v1.13.0
   Compiling camino v1.2.6
    Checking uluru v3.1.0
    Checking nm_impl v0.1.50
    Checking gix-refspec v0.42.0
    Checking gix-index v0.52.0
    Checking clru v0.6.3
    Checking gix-credentials v0.38.2
    Checking gix-ignore v0.21.1
   Compiling memoffset v0.9.1
    Checking semver v1.0.28
   Compiling cfg_aliases v0.2.2
   Compiling thiserror v1.0.69
    Checking winnow v1.0.4
    Checking utf8parse v0.2.2
    Checking anstyle-parse v1.0.0
    Checking toml_parser v1.1.3+spec-1.1.0
   Compiling nix v0.30.1
    Checking gix-worktree v0.53.0
    Checking gix-pack v0.71.0
    Checking nm v0.1.50
    Checking gix-config v0.57.0
    Checking gix-filter v0.31.0
    Checking gix-pathspec v0.18.1
   Compiling maybe-async v0.2.11
   Compiling thiserror-impl v1.0.69
    Checking gix-shallow v0.12.1
    Checking gix-negotiate v0.32.0
    Checking arc-swap v1.9.2
   Compiling fs-err v3.3.1
    Checking smol_str v0.3.6
    Checking toml_datetime v1.1.1+spec-1.1.0
    Checking serde_spanned v1.1.1
    Checking io-close v0.3.7
    Checking colorchoice v1.0.5
    Checking bumpalo v3.20.3
    Checking anstyle v1.0.14
    Checking is_terminal_polyfill v1.70.2
   Compiling typeid v1.0.3
    Checking unicode-properties v0.1.4
   Compiling portable-atomic v1.15.0
    Checking bit-vec v0.8.0
    Checking anstyle-query v1.1.5
    Checking toml_writer v1.1.2+spec-1.1.0
   Compiling zstd-safe v8.0.0
    Checking toml v1.1.6+spec-1.1.0
    Checking anstream v1.0.0
    Checking bit-set v0.8.0
    Checking ra-ap-rustc_lexer v0.160.0
error[E0080]: evaluation panicked: unicode-properties and unicode-ident must use the same Unicode version, `unicode_properties::UNICODE_VERSION` and `unicode_ident::UNICODE_VERSION` are different.
  --> /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ra-ap-rustc_lexer-0.160.0/src/lib.rs:47:9
   |
47 | /         panic!(
48 | |             "unicode-properties and unicode-ident must use the same Unicode version, \
49 | |             `unicode_properties::UNICODE_VERSION` and `unicode_ident::UNICODE_VERSION` are \
50 | |             different."
51 | |         );
   | |_________^ evaluation of `_` failed here

For more information about this error, try `rustc --explain E0080`.
error: could not compile `ra-ap-rustc_lexer` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...

-----

error: failed to build rustdoc for crate cargo-aprz-lib v1.1.2
note: this is usually due to a compilation error in the crate,
      and is unlikely to be a bug in cargo-semver-checks
note: the following command can be used to reproduce the error:
      cargo new --lib example &&
          cd example &&
          echo '[workspace]' >> Cargo.toml &&
          cargo add --path /home/runner/work/ox-tools/ox-tools/crates/cargo-aprz-lib --features internals &&
          cargo check &&
          cargo doc

error: aborting due to failure to build rustdoc for crate cargo-aprz-lib v1.1.2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 17, 2026 16:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The contract test does not verify repeated package selectors, leaving a regression gap.

Review details

Suppressed comments (1)

crates/cargo-anvil/tests/recipe_contracts.rs:2332

  • This contract covers only one scoped --package pair. The impact format allows repeated selectors (--package A@v --package B@v), so a regression that drops or mis-splats subsequent package arguments would still pass while the production coverage set contains multiple affected packages. Add a multi-package case and assert that both report invocations preserve every selector.
        ("--package measured@0.1.0", "llvm-cov report --package measured@0.1.0 --lcov", true),
  • Files reviewed: 10/12 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 18, 2026 09:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No unresolved blocking issues were identified.

Review details
  • Files reviewed: 10/12 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@martin-kolinek
martin-kolinek merged commit 2dfdaab into main Sep 18, 2026
29 checks passed
@martin-kolinek
martin-kolinek deleted the fix/anvil-coverage-report-scope branch September 18, 2026 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants