Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
e2fe74a
fix(chip): give the CHIP filter the contamination estimate
GeiserX Nov 20, 2026
e927d0e
test(chip): check that every CHIP path hands FilterMutectCalls the co…
GeiserX Nov 20, 2026
395bef2
fix(chip): estimate contamination from every common site, not only th…
GeiserX Nov 20, 2026
3ae8ecc
fix(carrier): keep the low-confidence mark and the zygosity on every …
GeiserX Nov 20, 2026
f1d31c8
docs(carrier): say what each carrier row carries and when a row is no…
GeiserX Nov 20, 2026
070d779
fix(chip): warn when the contamination estimate runs without gnomAD, …
GeiserX Nov 20, 2026
0da2276
test(chip): verify the module input order for common sites
GeiserX Nov 21, 2026
799d247
fix(carrier): keep showing rows with no unchanged copy in both report…
GeiserX Nov 21, 2026
e2737d8
test(personal-data): allow the synthetic LPA genotypes of the all rep…
GeiserX Nov 21, 2026
f631b3c
test(parity): fix the time of the report corpus, so its goldens hold …
GeiserX Nov 21, 2026
6785e78
Merge remote-tracking branch 'origin/main' into pgp/i411-carrier-conf…
GeiserX Nov 21, 2026
6b93bf4
Merge the T11 batch tip into #559
GeiserX Nov 21, 2026
6059508
fix(sf): cap a secondary-findings pair that rests on a low-confidence…
GeiserX Nov 21, 2026
fa5f6e6
fix(live): flag the carrier headline of og live serve for hom or hemi…
GeiserX Nov 21, 2026
4df2db0
test(parity): whole-report goldens for the carrier cases
GeiserX Nov 21, 2026
9a27ad7
fix(nextflow): stop the clinical filter crashing on REVEL and AlphaMi…
GeiserX Nov 21, 2026
12341ab
fix(carrier): give the carrier list and the SF card one answer beside…
GeiserX Nov 21, 2026
a857c8c
test(report): check the reason the reports give for an unknown SV sup…
GeiserX Nov 21, 2026
f91075b
fix(report): give the right reason when step 22's consensus cannot te…
GeiserX Nov 21, 2026
44d3398
ci(personal-data): allow the corpus's made-up genotypes in report-ren…
GeiserX Nov 21, 2026
966083d
test(e2e): compress the score-tier tables with the bcftools image, no…
GeiserX Nov 21, 2026
f059dc3
fix(report): count an SV row without a support key as not known
GeiserX Nov 21, 2026
87f194a
fix(carrier): let only confident calls give the phase of a two-copy pair
GeiserX Nov 21, 2026
6ec4394
docs(carrier): say where the carrier list and the SF card agree
GeiserX Nov 21, 2026
327a7c3
fix(live): give the live site's secondary-findings rows the consensus…
GeiserX Nov 21, 2026
ff501c3
fix(carrier): let no weak call make an allele firm or give it a phase
GeiserX Nov 21, 2026
ae476db
docs(carrier): name the weak HIGH-impact call where the two lists differ
GeiserX Nov 21, 2026
4299bb3
docs(carrier): name the real split between the two lists, not allele …
GeiserX Nov 21, 2026
366365f
fix(carrier): let a phased combined allele lend no phase to its weak …
GeiserX Nov 21, 2026
8c2adf8
Merge remote-tracking branch 'origin/main' into pgp/i411-carrier-conf…
GeiserX Nov 21, 2026
a1b27b3
test(parity): regenerate 38 report-render goldens on main's SV suppor…
GeiserX Nov 21, 2026
e5ae524
Merge main into the T15 batch tip
GeiserX Nov 21, 2026
f7cb2a1
Merge #559 i411-carrier-confidence into the T15 batch tip
GeiserX Nov 21, 2026
8a882d8
Merge #570 i411-nf-vcfanno-ops into the T15 batch tip
GeiserX Nov 21, 2026
59df890
Merge #556 i411-chip-contamination into the T15 batch tip
GeiserX Nov 21, 2026
7a6a81d
ci(e2e): weigh prs-3-panel-measure at 2900 s only on a run that is no…
GeiserX Nov 21, 2026
8befa62
docs(prs): say the panel measure adds 35 to 80 minutes, as its case says
GeiserX Nov 21, 2026
f657812
test(parity): regenerate report-render goldens for #559 with #571
GeiserX Nov 21, 2026
cc4b811
docs(carrier): name the deletion-straddle shape where the SF card is …
GeiserX Nov 21, 2026
23eb52a
ci(e2e): say the panel measure time applies only on a run that is not…
GeiserX Nov 21, 2026
e117c5c
fix(setup): fail validation when the required og executable is missing
GeiserX Nov 22, 2026
3150d30
build(containers): run OwnGenome tasks in a pinned stock image
GeiserX Nov 22, 2026
0c09b8a
fix(setup): validate both og builds when installed
GeiserX Nov 22, 2026
37c19c7
test(containers): check the pinned og runtime in migrated steps
GeiserX Nov 22, 2026
29b57ff
test(setup): use the og runtime fixture for genetic maps
GeiserX Nov 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/container-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -181,9 +181,12 @@ jobs:
fi
echo "SMOKE_WORK=${work}" >> "$GITHUB_ENV"

# The LOCITYPER_IMAGE row runs /pgp-bin/og locityper report in the image.
# Both rows run og inside the tested image.
- name: Build og as scripts/setup.sh --og does
if: matrix.var == 'LOCITYPER_IMAGE'
if: matrix.var == 'OG_IMAGE' || matrix.var == 'LOCITYPER_IMAGE'
env:
OG_ARCHES: x86_64
OG_DARWIN_ARCHES: ''
run: |
scripts/setup.sh --og
rust/og.sh check --required
Expand Down
15 changes: 8 additions & 7 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -112,19 +112,20 @@ env:
# ports a tool adds it here for its pull request.
# str-motif left on purpose: og keeps pathogenic and unknown minority
# motif reads as uncertain_motif, which the Python did not (claim sv-28).
OG_SHADOW: setup-mask-fasta dev-contract pgx-cyp2d6-bed pgx-cyp2d6-measure pgx-cyp2d6-check str-catalog cdna-check comphet-phase
OG_SHADOW: dev-contract pgx-cyp2d6-bed pgx-cyp2d6-measure pgx-cyp2d6-check cdna-check comphet-phase
# The shard legs besides the base leg. A full run uses all of them; a run
# of picked cases runs fewer when they fit (scripts/ci/e2e-shards.sh count). The cases took 230
# minutes in one job on 2026-11-02 (run 41390434697).
E2E_SHARDS: 6
# Times that replace scripts/ci/e2e-case-times.tsv's for this run
# (scripts/ci/e2e-shards.sh). prs-3-panel-measure measures the 1000 Genomes
# panel only on a run dispatched on a branch other than main (measure_on in
# its file), where it took 2684 to 3452 s (median 2900, runs 44488552622,
# 44616213787 and 44790003535 on the geiresx fork); elsewhere it takes 0 s,
# as the table says. Without this its shard got a full share of other cases
# on top and finished last. Set here, so every job of a run splits alike.
E2E_SECONDS: ${{ github.event_name == 'workflow_dispatch' && github.ref != 'refs/heads/main' && 'prs-3-panel-measure=2900' || '' }}
# panel only on a run dispatched on a branch other than main and not traced
# (measure_on in its file), where it took 2684 to 3452 s (median 2900, runs
# 44488552622, 44616213787 and 44790003535 on the geiresx fork); elsewhere it
# takes 0 s, as the table says. Without this its shard got a full share of
# other cases on top and finished last. Set here, so every job of a run
# splits alike.
E2E_SECONDS: ${{ github.event_name == 'workflow_dispatch' && github.ref != 'refs/heads/main' && !inputs.trace && 'prs-3-panel-measure=2900' || '' }}

concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref }}
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions assets/output_contract.tsv
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,7 @@ param viral_ref - - - - experimental - -
param chip_gff - - - - experimental - -
param chip_germline_resource - - - - experimental - -
param chip_pon - - - - experimental - -
param chip_common_sites - - - - experimental - -
param parascopy_population - - - - experimental - -
param parascopy_depth_bed - - - - experimental - -
param paraphase_regions - - - - experimental - -
Expand Down
15 changes: 8 additions & 7 deletions bin/live_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ def a_clinical_filter(S, s):
if sf.get("unavailable"):
# Not assessed is never zero findings: the report's sentence instead of the counts.
return {"facts": facts, "lines": rr.sf_apart_lines(sf), "attention": True}
rows = [[r[0], r[1], r[2], r[3]] for r in rr.sf_rows(sf)] if sf else []
rows = [[r[0], r[1], r[2], r[3]] for r in rr.sf_rows(sf, _data(S, "annotsv").get("consensus"))] if sf else []
high = f"{rr.high_rare(sf)}{rr.high_apart(sf)}" if sf.get("high_impact_checked") else "not checked"
facts += [[f"{sf.get('version') or 'ACMG SF'}: ClinVar P/LP in its genes", len(sf.get("clinvar_hits") or [])],
["High-impact rare variants in its genes", high]]
Expand Down Expand Up @@ -250,7 +250,7 @@ def a_structural(S, s):
["Consensus SVs (2+ callers)", sv.get("consensus", ".")],
["AnnotSV class 4 and 5", av.get("count", ".")]]
rows = [[x.get("type"), x.get("sv"), x.get("gt"), x.get("length"), x.get("acmg_class"),
rr.annotsv_genes_text(x) if x.get("genes") is not None else ".", rr.annotsv_support_text(x),
rr.annotsv_genes_text(x) if x.get("genes") is not None else ".", rr.annotsv_support_text(x, av.get("consensus")),
"; ".join(rr.annotsv_flags(x)) if "exon_edge" in x else "."]
for x in av.get("rows") or []]
return {"facts": facts, "table": table(["Type", "Where", "Genotype", "Length", "Class", "Genes", "Support",
Expand Down Expand Up @@ -374,7 +374,7 @@ def a_carrier(S, s):
if d.get("unavailable"):
return {"lines": rr.carrier_lines(d), "attention": True}
rows = [[c.get("gene"), c.get("variant"), c.get("genotype"), c.get("significance"), c.get("stars"),
c.get("inheritance"), c.get("condition")] for c in d.get("carriers") or []]
c.get("inheritance"), c.get("condition")] for c in (d.get("carriers") or []) + (d.get("hom_or_hemizygous") or [])]
two = (d.get("two_hits") or {}).get("genes") or []
return {"facts": [["Genes on the carrier list", d.get("genes_on_list")], ["Carrier findings", len(rows)],
["Genes with two hits", ", ".join(g if isinstance(g, str) else g.get("gene", ".") for g in two)
Expand Down Expand Up @@ -833,14 +833,14 @@ def gene(x):
return g if isinstance(g, str) else None
sf_genes = {gene(x) for x in _items(sf.get("clinvar_hits")) if isinstance(x, dict)}
car = _data(S, "carrier")
carrier_genes = {gene(c) for c in _items(car.get("carriers")) if isinstance(c, dict)}
carrier_genes = {gene(c) for c in _items(car.get("carriers")) + _items(car.get("hom_or_hemizygous")) if isinstance(c, dict)}
th = car.get("two_hits") if isinstance(car.get("two_hits"), dict) else {}
two_hit = {gene(g) for g in _items(th.get("genes"))} - {None}
# 1. ACMG secondary findings: the gene's own reporting rule already said "report". The report's own
# rows (rr.sf_rows): one it cannot read costs this rule, never the rest.
sf_list = []
with _Row(out, "clinical-filter", "act"):
sf_list = rr.sf_rows(sf) if sf and not sf.get("unavailable") else []
sf_list = rr.sf_rows(sf, _data(S, "annotsv").get("consensus")) if sf and not sf.get("unavailable") else []
for r in sf_list:
with _Row(out, "clinical-filter", "act"):
out.append(_f("act", "clinical-filter", ("sf", r[0], r[1]), f"Secondary finding in {r[0]}",
Expand Down Expand Up @@ -889,7 +889,7 @@ def gene(x):
f"A {x.get('type') or 'structural change'} of {abs(int(num(x.get('length')) or 0)):,} bases that "
f"AnnotSV ranks class {x.get('acmg_class')}. Short reads can misplace large changes: confirm it.",
gene=(x.get("genes") or [None])[0], detail=x.get("sv"),
evidence=[["Class", x.get("acmg_class")], ["Callers", rr.annotsv_support_text(x)],
evidence=[["Class", x.get("acmg_class")], ["Callers", rr.annotsv_support_text(x, _data(S, "annotsv").get("consensus"))],
["Genotype", x.get("gt")]], viewer=str(x.get("sv") or "").replace(" ", "") or None))
# 6. Repeat expansions outside Stranger's "normal", or in an intermediate zone.
for l in _items(_data(S, "expansions").get("loci")):
Expand Down Expand Up @@ -943,7 +943,8 @@ def gene(x):
"A ranking to bring to a clinical geneticist, not a diagnosis.", gene=x.get("gene"),
evidence=[["Score", x.get("exomiser_score")], ["Inheritance", x.get("moi")]]))
# 7. Carriers: one changed copy in a gene of the carrier list, and the couple screen.
for c in _items(car.get("carriers")):
# A row with no unchanged copy (hom_or_hemizygous) takes the branch below that says it is not a carrier.
for c in _items(car.get("carriers")) + _items(car.get("hom_or_hemizygous")):
with _Row(out, "carrier-list", "act"):
if c.get("gene") in two_hit:
continue
Expand Down
2 changes: 1 addition & 1 deletion bin/owngenome_live.py
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@ def _h_clinvar(d, summary):

def _h_carrier(d, summary):
lines = _rr().carrier_lines(d)
return lines[0], bool(d.get("carriers"))
return lines[0], bool(d.get("carriers") or d.get("hom_or_hemizygous"))


def _h_couple(d, summary):
Expand Down
25 changes: 16 additions & 9 deletions bin/render_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -836,9 +836,9 @@ def comphet_phase_html(d):
"those calls are listed apart with the reason and are not counted here.")


def annotsv_support_text(x):
def annotsv_support_text(x, consensus=None):
if x["support"] is None:
return "not known (no step 22 consensus)"
return "not known (step 22's consensus cannot tell)" if consensus else "not known (no step 22 consensus)"
if x["support"] == 1:
return f"single caller ({x['caller']})"
return f"{x['support']} callers ({', '.join(x['support_callers'])})"
Expand Down Expand Up @@ -874,7 +874,7 @@ def annotsv_lines(d):
"""Step 05's class 4-5 lines, the same in both reports."""
head = f"Class 4-5 SVs: {d['count']}"
if d["count"]:
head += (f" (single caller: {d['single_caller'] if d['consensus'] else 'not known'}; deletions on exon edges: "
head += (f" (single caller: {str(d['single_caller']) + (f', not known: {m}' if (m := sum(x.get('support') is None for x in d.get('rows') or [])) else '') if d['consensus'] else 'not known'}; deletions on exon edges: "
f"{d['exon_edge'] if d['exon_edge_checked'] else 'not checked'}; common: {d['common']})")
lines = [head]
if d.get("downgraded"):
Expand Down Expand Up @@ -1343,7 +1343,7 @@ def head(key, title=None):
w(f" {line}")
for x in d["rows"]:
w(f" {x['type']:<4} {x['sv']} class {x['acmg_class']} {annotsv_genes_text(x)} "
f"support: {annotsv_support_text(x)}")
f"support: {annotsv_support_text(x, d['consensus'])}")
for f in annotsv_flags(x):
w(f" {f}")
if d.get("downgraded"):
Expand Down Expand Up @@ -1553,7 +1553,7 @@ def head(key, title=None):
f"{len(sf['clinvar_hits'])} ClinVar P/LP, {high_rare(sf)} rare HIGH impact" + high_apart(sf)
+ (f", {len(sf['structural'])} class 4/5 SV(s)" if sf.get("structural_checked") else "")
+ (f", {len(sf['not_reported'])} not reported by the gene's rule" if sf.get("not_reported") else ""))
for g, v, z, ev in sf_rows(sf):
for g, v, z, ev in sf_rows(sf, (S.get("annotsv", MISSING).get("data") or {}).get("consensus")):
w(f" {g:<8} {v} {z} {ev}")
for line in sf_apart_lines(sf):
w(f" {line}")
Expand Down Expand Up @@ -1991,7 +1991,7 @@ def paralogs_unchecked(S):
"change in the same gene (step 39). A hypomorph or pseudodeficiency allele is set aside.")


def sf_rows(sf):
def sf_rows(sf, consensus=None):
"""The secondary findings as (gene, variant, genotype, evidence) rows,
the same in both reports: ClinVar hits, rare HIGH-impact records, then
class 4 and 5 SVs."""
Expand All @@ -2004,7 +2004,7 @@ def extra(x):
for x in sf["high_impact"]]
def support(x):
if x["support"] is None:
return "caller support not known (no step 22 consensus)"
return "caller support not known (step 22's consensus cannot tell)" if consensus else "caller support not known (no step 22 consensus)"
return f"single caller ({x['caller']})" if x["support"] == 1 else f"{x['support']} callers"
rows += [(x["gene"], x["variant"], ".", f"AnnotSV class {x['acmg_class']}, {support(x)}"
+ (f"; through {x['via']}" if x.get("via") else "") + extra(x)) for x in sf.get("structural", [])]
Expand All @@ -2025,13 +2025,20 @@ def carrier_lines(c):
"""The sample's own carrier list (no partner), the same in both reports."""
if c.get("unavailable"):
return [f"Carrier list: not made, {c['unavailable']}"]
both = c.get("hom_or_hemizygous") or [] # no unchanged copy: listed apart, not counted as carriers
lines = [f"Carrier list ({c['genes_on_list']} genes): {len(c['carriers'])} finding(s)"
+ (f", {len(both)} with no unchanged copy" if both else "")
+ (f", {len(c['set_aside'])} set aside" if c["set_aside"] else "")]
def hemi(x):
return f" ({x['hemizygous']})" if x.get("hemizygous") else ""
for x in c["carriers"]:
lines.append(f"{x['gene']} ({x['condition']}; {x['inheritance']}): {x['variant']} {x['genotype']}{hemi(x)}, "
f"ClinVar {x['significance']}")
for x in both:
lines.append(f"{x['gene']} ({x['condition']}; {x['inheritance']}): {x['variant']} {x['genotype']}{hemi(x)}, "
f"ClinVar {x['significance']}: "
+ ("two changed copies" if x["genotype"] == "hom" else "the only copy changed")
+ ", not the pattern of a carrier")
for x in c["set_aside"]:
lines.append(f"Set aside: {x['gene']} {x['variant']} {x['genotype']}{hemi(x)}: {x['name']} is a {x['kind']} "
"allele, not a carrier finding on its own")
Expand Down Expand Up @@ -2843,7 +2850,7 @@ def a(key, lines):
if av.get("rows"):
body.append(f" <table><tr><th>SV</th><th>Class</th><th>Genes</th><th>Callers</th><th>Checks</th>{VIEWER_TH}</tr>"
+ "".join(f"<tr><td>{E(x['type'] + ' ' + x['sv'])}</td><td>{E(x['acmg_class'])}</td>"
f"<td>{E(annotsv_genes_text(x))}</td><td>{E(annotsv_support_text(x))}</td>"
f"<td>{E(annotsv_genes_text(x))}</td><td>{E(annotsv_support_text(x, av['consensus']))}</td>"
f"<td>{E('; '.join(annotsv_flags(x)) or 'none')}</td>{viewer_td_of(x['sv'])}</tr>"
for x in av["rows"])
+ "</table>")
Expand Down Expand Up @@ -3062,7 +3069,7 @@ def a(key, lines):
if sf and (sf["clinvar_hits"] or sf["high_impact"] or sf.get("structural") or sf.get("not_reported")
or sf.get("unavailable")):
rows = "".join(f"<tr><td>{E(g)}</td><td>{E(v)}</td><td>{E(z)}</td><td>{E(ev)}</td>{viewer_td_of(v)}</tr>\n"
for g, v, z, ev in sf_rows(sf))
for g, v, z, ev in sf_rows(sf, (S.get("annotsv", MISSING).get("data") or {}).get("consensus")))
a("clinical", card(S["clinical"], f"Secondary-Findings Genes ({sf['version']})",
[" <table>", f" <tr><th>Gene</th><th>Variant</th><th>Genotype</th><th>Evidence</th>{VIEWER_TH}</tr>",
rows.rstrip("\n"), " </table>"]
Expand Down
Loading
Loading