Skip to content

Instrument first-party libraries in coverage reports - #8322

Draft
Amaury Chamayou (achamayou) wants to merge 1 commit into
mainfrom
achamayou-coverage-instrumentation
Draft

Instrument first-party libraries in coverage reports#8322
Amaury Chamayou (achamayou) wants to merge 1 commit into
mainfrom
achamayou-coverage-instrumentation

Conversation

@achamayou

Copy link
Copy Markdown
Member

Summary

Fix the missing native implementation coverage exposed by Coverage run 34252103362. Application and test translation units were instrumented, but separately compiled first-party libraries were not.

  • Instrument all nine first-party C++ implementation libraries (ccf, ccf_launcher, ccf_js, ccf_kv, ccf_endpoints, ccf_tasks, ccf_pal, ccf_threading, and ccfcrypto) when COVERAGE=ON: all 80 implementation translation units now receive counters and coverage mapping.
  • Propagate the coverage runtime link requirement to consumers of instrumented static libraries, including uninstrumented and installed consumers. Leave COVERAGE=OFF, third-party code and Rust instrumentation unchanged.
  • Keep the generated inventory at the same 69 linked binaries. Do not add static archives as duplicate or unlinked report inputs.
  • Export machine-readable per-file LLVM counts alongside HTML, retaining the existing exclusions and source-path mapping. Add framework/sample/other totals with an explicit denominator-change warning.
  • Enable a missing/zero-hit guard for 12 representative implementation files and headers in coverage CI. Run the six standard-library reporting regression tests in that report-generation step and document local usage. Preserve the coverage-report-html artifact name and its root index.html.

Validation performed

All development, builds, tests and coverage commands ran in Ubuntu-26.04 WSL, using Clang/LLVM 21.1.8 and a session-isolated native build based on e4cc4fb1b4f3a0524f69e803c67d2d6eb6aca8dc, configured with Debug, COVERAGE=ON and LONG_TESTS=ON. Builds used -j1; no production behavior or test assertions were changed.

  • Compile-database comparison: 0/80 to 80/80 first-party implementation units instrumented; all 21 third-party units remain uninstrumented; generated binary inventory unchanged.
  • Built all nine implementation libraries and the required application/test consumers. A small CMake fixture exercised coverage OFF/ON, uninstrumented consumers and installed-library runtime-link propagation.
  • Six targeted unit tests passed: base64_test, crypto_test, kv_test, task_system_test, js_test, and jwt_auth_test. All six reporting regressions passed. Targeted formatting, shell, ASCII, copyright/comment, whitespace and existing test-bucket checks passed.
  • All 18 original logging group/protocol pairs passed serially: the original nine groups for both e2e_logging and e2e_logging_http2, using the existing CR_FILTER, -j1 and unchanged 360-second timeout and assertions. The runner verified the complete original inventory and exactly one executed group per result; none was skipped.

Successful selected-test report, not a full-workflow baseline

This report uses only the six passing unit profiles plus 46 successful serial logging-node profiles, with a fixed inventory of ten already-built binaries. Failed canonical and partition profiles are excluded. All 12 sentinels have genuine nonzero node-runtime hits, and the instrumentation guard passes.

Scope Covered lines Covered branches
Framework 35,706/62,980 (56.69%) 6,073/16,801 (36.15%)
Samples 3,327/4,206 (79.10%) 246/450 (54.67%)
Combined 39,033/67,186 (58.10%) 6,319/17,251 (36.63%)

These percentages are not compared with the old report's denominator: previously invisible implementation code is now included.

Remaining limitations / related work

  • The canonical concurrent two-test logging executions did not pass. They encountered timeouts and HTTP/2 requests to cached former primaries after leadership loss. Archived node logs establish those step-downs; the same original groups subsequently passed in isolation. Serial results are not a passing canonical execution or a full Coverage/CI baseline.
  • The complete existing Coverage selection and a full CI baseline have not been run. This PR remains draft.
  • A separate, independent partition-workflow change adds the existing partition selection; this PR does not alter partition selection. Its local reconfiguration scenario failed in both the full partition run and an isolated run. Those failures and profiles are not presented as passing evidence here, and no passing incremental partition-coverage gain is claimed.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant