feat: align health score v2 with community feedback commitments IN-1226 #4460
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| DESCRIPTION > | ||
| - `health_score_v2_raw_inputs_snapshot_ds` captures raw measured inputs (not computed scores) for | ||
| Health Score v2 at periodic intervals for validation and historical analysis. | ||
| - Populated by `health_score_v2_raw_inputs_snapshot.pipe` (append-mode, monthly cadence). | ||
| - Each row is keyed by `repoUrl + snapshotDate + methodologyVersion` and captures all the atomic | ||
| measurement signals used to compute the three health score categories: | ||
| - Maintainer signals: bus factor (curated maintainer count), observed review/merge actors, | ||
| organization diversity (distinct org count), PR/issue response times (medians). | ||
| - Security signals: open CVE counts (critical/high/moderate), OpenSSF Scorecard score, security | ||
| practices flags (policy/branch-protection state), vulnerable dependencies count. | ||
| - Development signals: release recency/cadence, commit counts (6m windows), issue counts/closure | ||
| times, PR counts/merge times. | ||
| - Responsiveness/lifecycle signals: unanswered aged issue/PR counts and 18mo opened totals | ||
| (the per-repo unanswered count/ratio the compliance spec item 6 requires, and the historical | ||
| input the abandoned-threshold-sweep validation analysis runs against). Definitions mirror | ||
| health_score_v2_lifecycle.pipe: open, no non-author response, aged 90+ days, opened within | ||
| the 18mo window. | ||
| - Does NOT include computed category/overall scores — those are derived from these raw inputs via | ||
| the scoring logic in the respective pipes. This datasource is for tracing where scores come from. | ||
| - snapshotId = `toStartOfInterval(now(), INTERVAL 1 day)` at pipe execution time. | ||
| - Partitioned by year(snapshotDate) and month(snapshotDate) for efficient time-based queries. | ||
| - TTL set to 24 months (enough for trend analysis and validation studies). | ||
| - ReplacingMergeTree keyed on (snapshotDate, repoUrl, methodologyVersion): a same-day manual | ||
| re-run of the copy converges to one row per key after background merges instead of | ||
| accumulating duplicates (UNIQUE_KEY is not a thing in Tinybird — attempted and reverted in | ||
| cdf6cf263). Validation queries must still read with FINAL (or dedupe via argMax/LIMIT BY) | ||
| since replacement is asynchronous. | ||
|
|
||
| SCHEMA > | ||
| `repoUrl` String, | ||
| `snapshotDate` DateTime, | ||
| `methodologyVersion` String, | ||
| `busFactorCuratedCount` Nullable(UInt64), | ||
| `busFactorObservedActorsCount` Nullable(UInt64), | ||
| `orgDiversityCount` Nullable(UInt64), | ||
| `medianPrResponseSeconds` Nullable(Float64), | ||
| `medianIssueResponseSeconds` Nullable(Float64), | ||
| `openCriticalVulns` Nullable(UInt64), | ||
| `openHighVulns` Nullable(UInt64), | ||
| `openModerateVulns` Nullable(UInt64), | ||
| `scorecardScore` Nullable(String), | ||
| `securityPolicyEnabled` Nullable(UInt8), | ||
| `branchProtectionEnabled` Nullable(UInt8), | ||
| `branchProtectionRequiredReviews` Nullable(Int32), | ||
| `branchProtectionRequiresStatusChecks` Nullable(UInt8), | ||
| `branchProtectionAllowsForcePush` Nullable(UInt8), | ||
| `vulnerableDeps` Nullable(UInt64), | ||
| `daysSinceLatestRelease` Nullable(Int64), | ||
| `daysBetweenRecentReleases` Nullable(Int64), | ||
| `commitsLast6m` Nullable(UInt64), | ||
| `commitsPrior6m` Nullable(UInt64), | ||
| `lastCommitAt` Nullable(DateTime64(3)), | ||
| `issuesClosedLast12m` Nullable(UInt64), | ||
| `issuesOpenedLast12m` Nullable(UInt64), | ||
| `medianIssueCloseSeconds` Nullable(Float64), | ||
| `prsMergedLast12m` Nullable(UInt64), | ||
| `prsClosedUnmergedLast12m` Nullable(UInt64), | ||
| `medianPrMergeSeconds` Nullable(Float64), | ||
| `excluded` Nullable(UInt8), | ||
| `trackedPackageCount` Nullable(UInt64), | ||
| `unansweredIssuesAged90d` Nullable(UInt64), | ||
| `unansweredPrsAged90d` Nullable(UInt64), | ||
| `issuesOpenedLast18m` Nullable(UInt64), | ||
| `prsOpenedLast18m` Nullable(UInt64) | ||
|
|
||
| ENGINE ReplacingMergeTree | ||
| ENGINE_PARTITION_KEY toYYYYMM(snapshotDate) | ||
| ENGINE_SORTING_KEY (snapshotDate, repoUrl, methodologyVersion) | ||
| ENGINE_TTL snapshotDate + INTERVAL 24 MONTH |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,7 @@ | ||
| DESCRIPTION > | ||
| Per-repo Lifecycle state, computed via the spec's decision tree (first match wins): | ||
| archived (repos.archived) > abandoned (an issue or PR open and unanswered for 90+ days AND no | ||
| commits in 12mo) > inert (no commits in 18mo AND no issue/PR activity in 18mo) > | ||
| declining (commits down >50% vs prior 6mo AND issues opened up vs prior 6mo) > stable (release | ||
| within 12mo, <50 open issues, no open critical vulns, commits down >50% vs prior 6mo) > active | ||
| (fallback) — but NULL overrides all of the above when there is no usable signal at all (see | ||
|
|
@@ -27,13 +28,51 @@ DESCRIPTION > | |
| commitsLast6m=0 (down from 1895 prior), because it still has 3 real PRs in the 18mo window — | ||
| real activity, just not enough to clear any of the other branches' thresholds. That's a | ||
| legitimate 'active' classification under the existing spec, not the bug being fixed here. | ||
| - `r.lastCommitAt IS NULL` added to the NULL-guard condition (2026-08-11, per Joana's PR | ||
| review): the guard as written (zero commits in both 6mo windows + zero issue/PR activity in | ||
| 18mo) was a strict superset of the `inert` branch's own condition (no commits in 18mo + zero | ||
| issue/PR activity), since zero commits across the trailing 12mo already implies | ||
| `lastCommitAt` is either NULL or older than 18mo. That made the guard fire first on every repo | ||
| that could otherwise have matched `inert`, so `inert` was dead code — never actually emitted. | ||
| Restricting the guard to the `lastCommitAt IS NULL` case (matching the blocknetdx/blocknet | ||
| example this branch was originally written for) lets repos with a known, stale last-commit | ||
| timestamp fall through into the real decision tree, where `inert` can now actually fire. | ||
|
|
||
| NODE health_score_v2_lifecycle_calc | ||
| DESCRIPTION > | ||
| - Precedence (first match wins): archived > abandoned > inert > declining > stable > active. | ||
| - Threshold note (provisional, pending validation analysis): abandoned requires at least one | ||
| issue OR PR that has sat unanswered for 90+ days, plus a 12-month no-commit gate. The age | ||
| gate is a single threshold, NOT a closed 90-180 band: a band would let a repo exit | ||
| 'abandoned' once its ignored issues age past the upper bound (they still block 'inert' via | ||
| issuesInWindow18m, so the repo would flip back to 'active'). The spec's "~90-180 days" is | ||
| the tuning range for this one threshold — the abandoned-threshold-sweep validation analysis | ||
| picks the final value. The no-commit gate requires BOTH commit signals to be silent: zero | ||
| authored-commit activity in both trailing 6mo windows (covers repos where lastCommitAt is | ||
| NULL/stale from the upstream `repos` data gap but real commits exist in activityRelations) | ||
| AND repos.lastCommitAt NULL or older than 12mo (covers the reverse gap). Both signals are | ||
| last-commit-by-anyone, not maintainer-specific activity; whether maintainer review/comment | ||
| activity should also block 'abandoned' is an open question on the PR. | ||
| - Inert: lastCommitAt older than 18mo AND zero issues/PRs opened in the 18mo window. | ||
| lastCommitAt IS NULL deliberately does NOT qualify: a NULL-with-no-signal repo is already | ||
| caught by the NULL guard above, and a NULL-with-recent-commits repo (upstream data gap in | ||
| `repos`) must not be mislabeled inert just because one timestamp field is missing. | ||
| - Unanswered issue: opened in the 18mo window, no non-author comment, still open, and has been | ||
| open 90+ days. | ||
| - Unanswered PR: opened in the 18mo window, no reviewer activity (reviewedAt IS NULL), still | ||
| open (closedAt IS NULL), and has been open 90+ days. Mirrors the issue-side signal using | ||
| pull_requests_analyzed.reviewedAt, which is already the first-response timestamp for PRs | ||
| (first review / changes-requested / Gerrit patchset approval) — no new datasource field needed. | ||
| - Declining: commit count down >50% vs prior 6mo AND issues opened up vs prior 6mo. | ||
| - Stable: release within 12mo, <50 open issues, no open critical vulns, AND commits down | ||
| >50% vs prior 6mo. | ||
|
|
||
| SQL > | ||
| SELECT | ||
| r.url AS repoUrl, | ||
| if( | ||
| r.archived != 1 | ||
| AND r.lastCommitAt IS NULL | ||
| AND coalesce(c.commitsLast6m, 0) = 0 | ||
| AND coalesce(c.commitsPrior6m, 0) = 0 | ||
| AND coalesce(w.issuesInWindow18m, 0) + coalesce(p.prsInWindow18m, 0) = 0, | ||
|
|
@@ -42,9 +81,14 @@ SQL > | |
| multiIf( | ||
| r.archived = 1, | ||
| 'archived', | ||
| (coalesce(w.unansweredAged90d, 0) > 0 OR coalesce(pw.unansweredAged90d, 0) > 0) | ||
| AND coalesce(c.commitsLast6m, 0) = 0 | ||
| AND coalesce(c.commitsPrior6m, 0) = 0 | ||
| AND (r.lastCommitAt IS NULL OR r.lastCommitAt < now() - INTERVAL 12 MONTH), | ||
|
cursor[bot] marked this conversation as resolved.
|
||
| 'abandoned', | ||
|
cursor[bot] marked this conversation as resolved.
|
||
| r.lastCommitAt < now() - INTERVAL 18 MONTH | ||
| AND coalesce(w.issuesInWindow18m, 0) + coalesce(p.prsInWindow18m, 0) = 0, | ||
| 'inert', | ||
|
gaspergrom marked this conversation as resolved.
cursor[bot] marked this conversation as resolved.
Contributor
There was a problem hiding this comment. CRITICAL — The outer no-signal guard fires before is a strict superset of the Any repo qualifying for Fix options:
|
||
| coalesce(c.commitsLast6m, 0) < coalesce(c.commitsPrior6m, 0) * 0.5 | ||
| AND coalesce(w.issuesOpenedLast6m, 0) > coalesce(w.issuesOpenedPrior6m, 0), | ||
| 'declining', | ||
|
|
@@ -56,7 +100,8 @@ SQL > | |
| 'active' | ||
| ) | ||
| ) | ||
| ) AS lifecycleLabelV2, | ||
| '2.0.0' AS methodologyVersion | ||
| FROM | ||
| ( | ||
| SELECT base.url AS url, base.archived AS archived, rc.lastCommitAt AS lastCommitAt | ||
|
|
@@ -95,7 +140,13 @@ SQL > | |
| countIf( | ||
| openedAt <= now() - INTERVAL 6 MONTH AND openedAt > now() - INTERVAL 12 MONTH | ||
| ) AS issuesOpenedPrior6m, | ||
| countIf(openedAt > now() - INTERVAL 18 MONTH) AS issuesInWindow18m, | ||
| countIf( | ||
| openedAt > now() - INTERVAL 18 MONTH | ||
| AND respondedInSeconds IS NULL | ||
| AND closedAt IS NULL | ||
| AND openedAt <= now() - INTERVAL 90 DAY | ||
| ) AS unansweredAged90d | ||
| FROM issues_analyzed | ||
| GROUP BY channel | ||
| ) AS w | ||
|
|
@@ -107,6 +158,21 @@ SQL > | |
| GROUP BY channel | ||
| ) AS p | ||
| ON p.repoUrl = r.url | ||
| LEFT JOIN | ||
| ( | ||
| SELECT | ||
| channel AS repoUrl, | ||
| countIf( | ||
| openedAt > now() - INTERVAL 18 MONTH | ||
| AND reviewedAt IS NULL | ||
| AND approvedAt IS NULL | ||
| AND closedAt IS NULL | ||
| AND openedAt <= now() - INTERVAL 90 DAY | ||
| ) AS unansweredAged90d | ||
| FROM pull_requests_analyzed | ||
| GROUP BY channel | ||
| ) AS pw | ||
| ON pw.repoUrl = r.url | ||
| LEFT JOIN | ||
| ( | ||
| SELECT repoUrl, countIf(status = 'OPEN' AND severity = 'CRITICAL') AS openCriticals | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.