Skip to content

Union code coverage across all six pipelines#3716

Open
aaronburtle wants to merge 15 commits into
mainfrom
dev/aaronburtle/combined-coverage-union
Open

Union code coverage across all six pipelines#3716
aaronburtle wants to merge 15 commits into
mainfrom
dev/aaronburtle/combined-coverage-union

Conversation

@aaronburtle

Copy link
Copy Markdown
Contributor

Why make this change?

Closes #3715

What is this change?

  • Adds a best-effort Total Code Coverage job to the existing unit-test
    pipeline (unittest-pipelines.yml) that surfaces the true combined coverage
    across all six test pipelines, instead of the misleadingly low per-pipeline
    numbers.
    • DAB's tests are split across six independent ADO pipelines (unit + MsSql,
      PostgreSql, MySql, DwSql, CosmosDb). Each publishes its own coverage tab and
      nothing merges them, so every pipeline in isolation reads much lower than the union.
  • How it works:
    • The build job publishes this run's unit cobertura as a coverage-unit
      pipeline artifact.
    • The new total_code_coverage job (dependsOn: build) downloads that plus the
      latest available cobertura from the five DB pipelines
      (DownloadPipelineArtifact@2, definition IDs DwSql=1, MsSql=2, MySql=3,
      PgSql=4, Cosmos=6).
    • scripts/merge-coverage.ps1 unions all reports and emits one combined
      Cobertura.xml, which PublishCodeCoverageResults@2 publishes.
  • Design notes:
    • Rolling dashboard, not a synchronized per-PR gate. DB coverage is pulled
      with runVersion: latest, so it may reflect an older commit ADO offers no
      primitive to block on all six sibling runs from inside one of them. A true
      per-PR guarantee would require consolidating all suites into one pipeline with
      a dependsOn-all aggregation job. This is not necessary for tracking code coverage.
    • Best-effort: continueOnError on the job and every step, so it can never
      fail or block unit CI. If aggregation fails, the build job's own unit-only
      coverage remains as the tab fallback.
    • The merge keys on fully-qualified class name + line (agent-path independent)
      so points union correctly across Windows/Linux agents, and excludes *.Tests
      assemblies.

How was this tested?

  • Integration Tests
  • Unit Tests

Validated scripts/merge-coverage.ps1 locally against real cobertura from all six
pipelines (combined = 76.77% branch / 83.33% line) and against synthetic reports
covering cross-agent path union, branch-condition max-merge, test-assembly
exclusion, and well-formed Cobertura output. Pipeline changes are CI-only.

Sample Request(s)

  • N/A — CI/pipeline change only; no REST, GraphQL, or CLI surface is affected.
  • Local usage of the merge script:
    ./scripts/merge-coverage.ps1 -Path <folder-of-cobertura> -OutFile combined/Cobertura.xml
    

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a “Total Code Coverage” aggregation mechanism to provide a more accurate combined (union) coverage number across DAB’s six independent Azure DevOps pipelines by downloading each pipeline’s Cobertura output and merging them into a single report.

Changes:

  • Introduces scripts/merge-coverage.ps1, a dependency-free Cobertura union merger that can also emit a merged Cobertura XML for publishing.
  • Updates the unit-test pipeline to publish its coverage as a pipeline artifact and adds a new best-effort job that downloads latest DB-pipeline coverage and publishes a combined coverage report.
  • Extends the shared build/unit-test template to stage and publish the unit coverage artifact used by the aggregation job.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
scripts/merge-coverage.ps1 Adds a PowerShell Cobertura union-merger (line + branch) and optional merged XML emitter.
.pipelines/unittest-pipelines.yml Converts the unit pipeline to multi-job and adds a best-effort “Total Code Coverage” aggregation job.
.pipelines/templates/build-pipelines.yml Publishes unit Cobertura files as a pipeline artifact (coverage-unit) for later aggregation.

Comment thread .pipelines/unittest-pipelines.yml
Comment thread .pipelines/templates/mssql-test-steps.yml
Comment thread .pipelines/unittest-pipelines.yml Outdated
Comment thread .pipelines/unittest-pipelines.yml Outdated
Comment thread .pipelines/unittest-pipelines.yml Outdated
Comment thread scripts/merge-coverage.ps1
Comment thread scripts/merge-coverage.ps1 Outdated
@github-project-automation github-project-automation Bot moved this from Todo to Review In Progress in Data API builder Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Test related issue.

Projects

Status: Review In Progress

Development

Successfully merging this pull request may close these issues.

Aggregate code coverage reports to identity total code coverage

5 participants