Skip to content

test: Count outcomes from the notes #1693

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
chfast merged 1 commit into from
Sep 2, 2026
+6 −21
Merged

test: Count outcomes from the notes #1693

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 6 additions & 21 deletions test/utils/test_driver.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,6 @@ int run_tests(std::span<const TestCase> cases, std::ostream& out, const RunOptio
out << "collected " << cases.size() << (cases.size() == 1 ? " test\n\n" : " tests\n\n");

std::vector<Note> notes;
size_t passed = 0;
size_t failed = 0;
size_t skipped = 0;
Progress row{out, cases.size()};

for (const auto& test : cases)
Expand All @@ -97,7 +94,6 @@ int run_tests(std::span<const TestCase> cases, std::ostream& out, const RunOptio
auto outcome = Outcome::passed;
std::string reason;
std::string exception_reason;
std::string skip_reason;
if (!options.progress)
out << test.name << '\n'; // The only thing naming what the test prints next.
out << std::flush;
Expand All @@ -108,7 +104,7 @@ int run_tests(std::span<const TestCase> cases, std::ostream& out, const RunOptio
catch (const UnsupportedTestFeature& ex)
{
outcome = Outcome::skipped;
skip_reason = ex.what();
reason = ex.what();
}
catch (const std::exception& ex)
{
Expand All @@ -133,29 +129,18 @@ int run_tests(std::span<const TestCase> cases, std::ostream& out, const RunOptio
std::move(exception_reason) :
failures.front().what;
}
else if (outcome == Outcome::skipped)
{
reason = std::move(skip_reason);
}
if (outcome != Outcome::passed)
notes.push_back({outcome, test.name, std::move(reason), std::move(failures)});

switch (outcome)
{
case Outcome::passed:
++passed;
break;
case Outcome::failed:
++failed;
break;
case Outcome::skipped:
++skipped;
break;
}
if (options.progress)
row.advance(outcome);
}

// Every test which did not pass left exactly one note, so the counts follow from them.
const auto failed = std::ranges::count(notes, Outcome::failed, &Note::outcome);
const auto skipped = std::ranges::count(notes, Outcome::skipped, &Note::outcome);
const auto passed = cases.size() - notes.size();

if (failed != 0)
{
out << '\n';
Expand Down