Summary
Wire up Codecov Test Analytics so failed tests (and flaky tests) surface in Codecov PR comments and the Tests tab, separate from our existing coverage upload.
We already upload coverage (coverage.lcov via codecov/codecov-action@v5 + CODECOV_TOKEN). Test Analytics needs a second upload: JUnit XML test results with report-type: test_results.
What we get
- Failed test reporting on PRs — failed tests + stack traces in the Codecov PR comment
- Flaky test callouts — free for public/OSS repos
- Tests tab — per-test duration, failure rates, slow tests (results retained ~60 days)
How it works
swift test → JUnit XML → Codecov (report-type: test_results) → PR comment + Tests tab
Codecov only supports JUnit XML for test results today.
Implementation sketch
1. Emit JUnit from swift test
Today (.github/workflows/ci.yml):
- name: Test
run: swift test --enable-code-coverage
Add xunit output, e.g.:
swift test --enable-code-coverage --xunit-output junit.xml
(--xunit-output is built into SwiftPM; filename should match *junit.xml or be passed explicitly to the uploader.)
2. Upload test results (even when tests fail)
Use the current codecov/codecov-action@v5 with report-type: test_results. The dedicated codecov/test-results-action is deprecated in favor of this.
Critical: if the Test step fails, later steps are skipped by default — so failed-test reporting never gets the file. Upload with if: ${{ !cancelled() }} (and ideally do the same for coverage export/upload so failures still report coverage when possible).
Sketch:
- name: Upload test results to Codecov
if: ${{ !cancelled() }}
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
report-type: test_results
files: junit.xml
fail_ci_if_error: false
Keep the existing coverage upload as a separate step (no report-type, or coverage files only).
3. Verify
- CI log shows successful test-results upload
- Codecov PR comment acknowledges test results received
- Intentionally fail a test once and confirm failed tests + stack traces appear in the PR comment
- Tests tab populates on the default branch after a few runs
Notes / caveats
- Two independent uploads: coverage (
lcov) vs test results (junit) — both use the same token
- XCTest xunit detail can be thinner than some frameworks; still enough for names + pass/fail analytics. Swift Testing often has richer
<failure> messages if we mix runners later
- Not a CI gate — analytics only; the job still fails when tests fail
- No change needed to
codecov.yml for basic setup (flags optional if we ever matrix test envs)
References
Acceptance criteria
Summary
Wire up Codecov Test Analytics so failed tests (and flaky tests) surface in Codecov PR comments and the Tests tab, separate from our existing coverage upload.
We already upload coverage (
coverage.lcovviacodecov/codecov-action@v5+CODECOV_TOKEN). Test Analytics needs a second upload: JUnit XML test results withreport-type: test_results.What we get
How it works
Codecov only supports JUnit XML for test results today.
Implementation sketch
1. Emit JUnit from
swift testToday (
.github/workflows/ci.yml):Add xunit output, e.g.:
swift test --enable-code-coverage --xunit-output junit.xml(
--xunit-outputis built into SwiftPM; filename should match*junit.xmlor be passed explicitly to the uploader.)2. Upload test results (even when tests fail)
Use the current
codecov/codecov-action@v5withreport-type: test_results. The dedicatedcodecov/test-results-actionis deprecated in favor of this.Critical: if the Test step fails, later steps are skipped by default — so failed-test reporting never gets the file. Upload with
if: ${{ !cancelled() }}(and ideally do the same for coverage export/upload so failures still report coverage when possible).Sketch:
Keep the existing coverage upload as a separate step (no
report-type, or coverage files only).3. Verify
Notes / caveats
lcov) vs test results (junit) — both use the same token<failure>messages if we mix runners latercodecov.ymlfor basic setup (flags optional if we ever matrix test envs)References
report-type: test_results).github/workflows/ci.yml(build-testjob)Acceptance criteria
report-type: test_results