Skip to content

e2e: collect coverage data from e2e test runs - #792

Open
klihub wants to merge 3 commits into
e2e-test-fixesfrom
e2e-coverage
Open

klihub wants to merge 3 commits into
e2e-test-fixesfrom
e2e-coverage

Conversation

@klihub

@klihub klihub commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Make test/e2e/run_tests.sh collect go coverage data of the plugins the tests
exercise, and report per-plugin and total coverage at the end of a run.

  • make e2e-tests builds the resource-manager-based plugins with go build -cover (make COVER=1 build images).
  • With the test APIs enabled a plugin serves its coverage data over the
    instrumentation HTTP server, so a plugin which never exits gracefully is
    covered too. What is served and what a plugin dumps at exit merge as they are.
  • test/e2e/report-coverage.sh merges what a run collected into a profile, an
    HTML report and summary.json. reset_coverage=1 discards earlier data first.

Verified on a full run of both suites: 55/55 PASS, balloons 78.6%,
topology-aware 68.0%, 59% of all instrumented packages.

Port forwarding to the plugin needs the vm-wait-pod-regexp fix from the PR below.

@klihub
klihub added this pull request to stack #796 September 16, 2026 11:44
Add a coverage package which serves the coverage data of a plugin built
with go build -cover over the instrumentation HTTP server: the meta-data,
a snapshot of the counters, and the ID of the binary tying the two
together. The ID is the one the go runtime names the files it writes to
$GOCOVERDIR with, dug out of the meta-data header, so that data served
over HTTP and data dumped at exit merge as they are. This is how we get
the coverage of a plugin which is still running, or which never gets to
exit gracefully.

Serve it from the resource manager, so from the nri-resource-policy-*
plugins, and only with the test APIs enabled. runtime/coverage works in
a binary built with -cover and never in a test binary, so the tests
build a helper to check that what we serve is what go tool covdata
expects.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
COVER=1 builds the resource-manager-based plugins with coverage
instrumentation. Instrument our own packages only, and with atomic
counters: the plugins are concurrent, and clearing the counters at
runtime does not work without them.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
Collect the coverage data of the plugins the tests exercise, and report
the coverage of each plugin and the total at the end of a run. The tests
clear the counters when they launch a plugin and ask it for a dump once
done. make e2e-tests builds the plugins with COVER=1 so that a run has
something to collect; collecting never fails a test.

go tool covdata percent cannot report per plugin, and prints a package
which has no statements without a percentage and without a line break,
running the next package into it, so calculate the numbers from the
profile instead, weighted by statements the way go tool cover does.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
@klihub klihub changed the title e2e coverage e2e: collect coverage data from e2e test runs Sep 16, 2026

@askervin askervin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants