Skip to content

test_runner: match coverage globs against cwd-relative paths only - #65940

Open
NAVEENKUMARKR777 wants to merge 1 commit into
nodejs:mainfrom
NAVEENKUMARKR777:fix-test-coverage-ancestor-test-dir
Open

test_runner: match coverage globs against cwd-relative paths only#65940
NAVEENKUMARKR777 wants to merge 1 commit into
nodejs:mainfrom
NAVEENKUMARKR777:fix-test-coverage-ancestor-test-dir

Conversation

@NAVEENKUMARKR777

Copy link
Copy Markdown

Summary

--test-coverage-exclude and --test-coverage-include β€” including the built-in default pattern used to drop test files from coverage reports β€” were matched against both the cwd-relative path and the absolute filesystem path of every candidate file.

For a relative-style glob such as the default **/{test,test/**/*,test-*,*[._-]test}.{js,mjs,cjs}, matching against the absolute path lets a leading ** cross into directory segments that have nothing to do with the project: a container WORKDIR of /test, a home directory literally named test, a CI checkout path with a test segment, and so on. Any project living under such a path had every one of its files spuriously match the default exclude glob and silently vanish from the coverage report, even though --experimental-test-coverage reported 100% coverage of nothing.

This is #58654, open since June 2025 and confirmed by a maintainer. A previous fix attempt (#62362) stalled and auto-closed because it dropped absolute-path matching entirely, which broke the (intentional, if under-justified) ability to pass an absolute --test-coverage-include/--test-coverage-exclude pattern β€” see the discussion in #53553.

Why this wasn't caught by the existing tests

This repo's own coverage-default-exclusion tests run from test/.tmp.N, i.e. from inside this repository's own test/ directory β€” which should be exactly the scenario that triggers the bug. It doesn't, purely by coincidence: the absolute-path matcher uses dot: false (added in 22e99dc / #63401, to fix a related-but-different dotfile-matching issue), and the .tmp.N segment is dot-prefixed, which blocks the buggy glob from crossing it. That masked the bug in-repo while leaving it fully reproducible for any real project whose path doesn't happen to contain a dotfile segment between the ancestor test directory and the source files (i.e. almost everyone hitting this in the wild). The new regression test is deliberately set up under a directory outside of test/.tmp.N so it actually exercises the bug β€” I verified it fails without the fix and passes with it.

Fix

Only match a glob against the absolute path when the glob pattern is itself an absolute path (e.g. an explicit /abs/path/** passed to --test-coverage-include). Every relative-style pattern, including all of the built-in default exclude patterns, is now evaluated exclusively against each file's path relative to the current working directory, which is what those patterns were actually designed to describe. This keeps the ability to use an absolute include/exclude pattern intact while eliminating the accidental ancestor-path leak.

doc/api/cli.md is updated to describe the corrected matching semantics.

Test plan

  • Reproduced the bug against a from-scratch build, confirmed the fix resolves it
  • Temporarily reverted just the lib/internal/test_runner/coverage.js change and confirmed the new regression test fails (proving it actually catches the bug), then restored the fix and confirmed it passes
  • python3 tools/test.py parallel/test-runner-coverage-default-exclusion parallel/test-runner-coverage parallel/test-runner-coverage-thresholds parallel/test-runner-coverage-source-map parallel/test-runner-run-coverage parallel/test-config-file parallel/test-runner-flag-propagation β€” all pass
  • Full test-runner parallel suite (83 files) and test-runner output-snapshot suite (109 files) β€” all pass, no regressions
  • Confirmed the dotfile-exclusion behavior from Default behavior of --test-coverage-exclude globbing does NOT apply to "dotfiles" like .mytest.cjs (common globbing oversight)Β #63397/test_runner: match dotfiles in default coverage excludeΒ #63401 still works correctly
  • eslint clean on all changed files

Fixes: #58654

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Welcome to Node.js, and thank you for your first contribution!

Before review, please take a moment to read:

Please make sure every commit is signed off. For a first pull request, GitHub Actions require collaborator approval and Jenkins CI must be started by a collaborator or triager, so an initial wait is normal.

Caution

AgentScan found account activity patterns that may be consistent with automation. This is a heuristic, not proof that this pull request was opened by an agent or violates policy. AI-assisted contributions are permitted, but automated tooling must not open pull requests without advance approval, and contributors must personally understand, test, verify, and take responsibility for every submitted change. See the AgentScan analysis, AI use policy, and automation policy for additional context.

--test-coverage-exclude and --test-coverage-include (including the
default exclude pattern used to drop test files from coverage
reports) were matched against both the cwd-relative path and the
absolute filesystem path of every candidate file. For a
relative-style glob such as the default
`**/{test,test/**/*,test-*,*[._-]test}.{js,mjs,cjs}`, matching
against the absolute path lets a leading `**` cross into directory
segments that have nothing to do with the project: a container
`WORKDIR` of `/test`, a home directory literally named `test`, a CI
checkout path with a `test` segment, and so on. Any project living
under such a path had every one of its files spuriously match the
default exclude glob and silently vanish from the coverage report,
even though `--experimental-test-coverage` reported 100% coverage of
nothing.

This was previously papered over in Node's own test suite by an
unrelated detail: the absolute-path matcher uses `dot: false`, and
this repository's own test tmp dir (test/.tmp.N) has a dot-prefixed
segment that happens to block the buggy match from crossing it. That
made the bug unobservable through the existing coverage-default-
exclusion fixtures despite them running from a directory nested
under this repo's own `test/` folder, and meant the dotfile-handling
fix in 22e99dc addressed a related but different problem without
touching this one.

Fix this by only matching a glob against the absolute path when the
glob pattern itself is an absolute path (e.g. an explicit
`/abs/path/**` passed to --test-coverage-include). Every
relative-style pattern, including all of the built-in default
exclude patterns, is now evaluated exclusively against each file's
path relative to the current working directory, which is what every
default pattern was actually designed to describe.

The new regression test is deliberately set up under a fresh
directory outside of test/.tmp.N, since that directory's dot-prefixed
segment is exactly what prevented the existing fixtures from
catching this bug.

Fixes: nodejs#58654
Signed-off-by: NAVEENKUMARKR777 <naveenkumarkr555@gmail.com>
@NAVEENKUMARKR777
NAVEENKUMARKR777 force-pushed the fix-test-coverage-ancestor-test-dir branch from 62d4c5e to e16e12e Compare September 9, 2026 15:17
@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 3 lines in your changes missing coverage. Please review.
βœ… Project coverage is 90.18%. Comparing base (fd6682c) to head (e16e12e).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/test_runner/coverage.js 80.00% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #65940      +/-   ##
==========================================
+ Coverage   90.16%   90.18%   +0.01%     
==========================================
  Files         771      771              
  Lines      265445   265454       +9     
  Branches    50455    50463       +8     
==========================================
+ Hits       239329   239389      +60     
+ Misses      17056    17002      -54     
- Partials     9060     9063       +3     
Files with missing lines Coverage Ξ”
lib/internal/test_runner/coverage.js 60.39% <80.00%> (+0.06%) ⬆️

... and 34 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agentscan:automated-account needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

--experimental-test-coverage empty since 23.5.0

2 participants