Skip to content

feat(ancestry): estimate admixture fractions in og with honest bootstrap ranges, dropping ADMIXTURE - #577

Open
geiresx wants to merge 47 commits into
GeiserX:mainfrom
geiresx:pgp/anc-project
Open

geiresx wants to merge 47 commits into
GeiserX:mainfrom
geiresx:pgp/anc-project

Conversation

@geiresx

@geiresx geiresx commented Nov 22, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #571: merge that first. This branch then takes main with a merge commit.

The problem

Step 26b ran ADMIXTURE to turn a person's genotypes into ancestry fractions. Three things were wrong with that:

  • The ranges were too narrow. ADMIXTURE's bootstrap kept every group's allele frequencies fixed, so the range left out how well the reference panel knows each group. A group of ten reference people got the same range as a group of five hundred.
  • A short run gave no warning. ADMIXTURE stops on a change in log-likelihood. Nothing told you whether the answer was really the best one.
  • It was a third-party binary in a container, driven by a shell script that og ancestry-fine sample-bed wrote. That kept a non-Rust step in the 1.0.0 pipeline.

The fix

og ancestry-fine project now finds the fractions inside og.

  • Each group's frequencies are fixed, so the log-likelihood of the fractions is concave and has one maximum. og finds it with the EM update plus SQUAREM steps, and keeps a step only when the likelihood does not drop. It stops when no fraction moves by more than 1e-9.
  • It then checks the KKT conditions of that maximum. If they fail, the step stops with an error instead of printing a wrong answer.
  • Each bootstrap replicate draws 5 cM blocks of SNVs with replacement. Blocks are cut per chromosome, so no block spans two chromosomes. The same replicate also draws each group's reference people again from model/ref.geno and counts the frequencies again. This is the uncertainty ADMIXTURE left out.
  • One seed gives byte-identical output for any thread count. Each replicate has its own random stream.
  • The output keeps the .Q and boot.Q layouts under work/fractions/, plus a <sample>_<level>.fit.tsv per level: SNVs, passes, log-likelihood, the KKT numbers and the bootstrap settings.
  • A model made before ref.geno existed stops the bootstrap with one line: run setup.sh --ancestry-fine-panel again. Setup refits it in seconds.
  • In Nextflow, ANCESTRY_FINE_FRACTIONS replaces ANCESTRY_FINE_PREPARE and ANCESTRY_FINE_ADMIXTURE. og ancestry-fine sample-bed, its generated script and its parity cases are gone.
  • og ancestry-fine report takes --fractions DIR. Its sentence now names the projection.

Decisions:

  • report --admixture DIR stays as an alias of --fractions DIR, so a 1.0.0 command line keeps working.
  • ADMIXTURE_IMAGE stays pinned in versions.env. Its removal belongs to the retire package.
  • docs/26b-ancestry-fine.md, rust/README.md and docs/rust.md still describe ADMIXTURE and sample-bed. They belong to the ancestry docs package.

Files outside the package's list, each needed for the change to build or stay true:

  • rust/crates/og-ancestry/src/lib.rs: drops the sample-bed module and its dispatch.
  • rust/og/src/main.rs: the usage and help lines.
  • rust/crates/og-admix/src/lib.rs: module docs.
  • workflows/clinical.nf: wires the new process.
  • scripts/ci/gen-containers-config.sh: the process table that generates conf/containers.config (regenerated, --check passes for all 127 processes).
  • tests/shellcheck-baseline.txt: the deleted script's line.
  • tests/parity/golden/ancestry-fine/report*/: new goldens, because the fractions sentence changed on purpose.
  • tests/fixtures/live/demo-001.weblog.jsonl: the demo weblog, generated from assets/steps.tsv. tests/demo/make_demo_events.py wrote it again, and only the 26b process set changed.

How it was checked

Unit tests, on a test machine:

  • og-admix: 38 tests pass, og-ancestry: 79, og port_scaffold: 3. cargo fmt --check and cargo clippy -p og-admix -p og-ancestry -p og --tests -D warnings are clean.
  • recovers_planted_fractions: planted fractions on a simulated 3-group panel of 5,000 SNVs come back within 0.01.
  • a_truncated_run_fails_the_kkt_check: a run cut to 3 passes is refused.
  • a_nonfinite_fit_is_refused: a frequency of 1e-310 overflows the EM step into NaN, which used to pass as converged. The projection now stops with an error. The test failed before that fix (it got Ok with NaN fractions).
  • no_block_spans_two_chromosomes: blocks on a 2-chromosome map.
  • the_command_projects_and_bootstraps: the same seed gives the same boot.Q bytes on 1 and 4 threads, and another seed differs.
  • drawing_the_people_again_widens_a_small_group: with 10-person groups, the range is wider when the people are drawn again than with fixed frequencies.

Red runs, on a test machine:

  • cargo test -p og-admix --features planted-red fails recovers_planted_fractions (two frequency columns swapped) and no_block_spans_two_chromosomes (blocks cut by SNV count).
  • og-ancestry has no such feature, so its two reds were made by hand and then reverted. Fixed frequencies plus one random stream per thread made drawing_the_people_again_widens_a_small_group fail, and the_command_projects_and_bootstraps fail on bytes that differ between thread counts.
  • The e2e case now checks each level's fit.tsv for 100 replicates with reference resampling. Its red is the batch E2E run.

Parity with ADMIXTURE 1.3.0 (admixture -P, the pinned image, run one last time) on the same .P and genotypes:

set fits max |dq| L(og) − L(ADMIXTURE) pass (|dq| ≤ 0.005, L(og) ≥ L(ADM) − 1e-6|L|)
20 simulated genomes, 3 groups, 3,000 SNVs 20 0.00002 ≥ 0 on every fit 20 of 20
three real genomes, the full panel (110k SNVs, 5 and 17 groups) 6 under 0.0001 > 0 on every fit 6 of 6

The one simulated difference of 0.00002 is a genome with a zero fraction: ADMIXTURE keeps every fraction at 1e-5 or more. The real-genome values stay off this PR.

On the full panel (111,071 SNVs, 2,868 reference people), with a model fitted by og ancestry-fine freqs, the 100-replicate bootstrap with reference resampling finished on all three genomes in about 4 minutes each, at 4 threads and under 250 MB. Each run had 719 blocks per replicate. On the model installed before ref.geno, the bootstrap stopped with the one-line message.

Fast checks on the fork, head dfdd462:

E2E cases for the batch: ancestry-fine-grained-1 and gvcf-pass-gate.

Beads: lin-49, lin-52, D5, D14, D1

FilterMutectCalls in step 49 ran without a contamination table, so another
person's DNA in the sample, whose alleles sit at a low VAF, could pass as a
small clone. With step 29's common-sites VCF installed, the step and the
Nextflow CHIP_MUTECT2 now run GetPileupSummaries inside the CHIP exons and
CalculateContamination, and pass --contamination-table and
--tumor-segmentation to FilterMutectCalls. New optional parameter
--chip_common_sites; run-all.sh passes it when the VCF and its .tbi are
installed. Without the VCF the log says there is no estimate.

Bead: trait-62
…ntamination table

The fake-docker case runs step 49 without the common-sites VCF, with it, and
with it but no .tbi; renders CHIP_MUTECT2's script block with a fake gatk with
and without --chip_common_sites; and checks that run-all.sh passes the VCF to
the pipeline only with its .tbi.

Bead: trait-62
…e CHIP exons

The CHIP coding exons hold 8 of the 59,295 sites of the common-sites VCF,
too few for CalculateContamination to estimate anything. Contamination is a
property of the sample, so GetPileupSummaries now reads every common site of
the VCF in both the step and CHIP_MUTECT2. The fake-docker case checks the
table first, so main fails it on the --contamination-table check.

Bead: trait-62
…carrier row

The carrier list parsed each hit with the default call floor (GQ 30,
DP 10) and dropped the floor the hits file states, and it wrote no call
quality at all, so a weak call read like a sure one. Each row now has
zygosity, gq, dp and low_confidence, judged against the file's own
##PGP_CALL_FLOOR, as the ClinVar section judges the same hit.

A homozygous or hemizygous P/LP row has no unchanged copy, so it moves
from carriers to a new list, hom_or_hemizygous. Two different
heterozygous hits in one recessive gene get pair and phase from the
haplotype rule of the secondary-findings card (two hits, possible two
hits, or one copy in cis). A pair resting on a low-confidence call, and
a deletion plus a low-confidence call (two_hits), is at most possible.

The schema describes the carrier section; the summary-collect goldens
change only in the carrier section. og-render's hemizygous tests read
the row from its new list.

Bead: clin-15
Bead: clin-19
…or not at all

FilterMutectCalls weighs each allele by its population frequency, which
Mutect2 takes from gnomAD's germline resource. On public data (HG002 with
HG005's reads mixed in) the estimate removed 31 of 35 PASS calls with
gnomAD and none without it. The step now warns in that case, and main.nf
warns on the console without --chip_common_sites and with it but without
--chip_germline_resource, since a task's stderr is not shown. The docs say
the estimate needs gnomAD too and give the measurement.

Bead: trait-62
Check the positional binding of the common-sites VCF and index. List the optional VCF in the Nextflow guide.

Bead: trait-62
…s and on the live page

Moving homozygous and hemizygous P/LP rows from `carriers` to
`hom_or_hemizygous` left every consumer reading only `carriers`. The live
page lost its act finding for such a row ("two changed copies", "the only
copy changed"), and a gene left with a homozygous row fell to a plain
ClinVar review finding. Both reports dropped the row from the carrier
block.

The live page (finding rows, the carrier panel and its headline) now
reads both lists. Both report renderers (og-render and render_report.py,
still step 24's) print each such row on its own line in the carrier
block, "two changed copies" or "the only copy changed", "not the pattern
of a carrier", and the block's first line counts it apart from the
carriers. The report-render parity cases whose summary holds such rows
get goldens for the new lines.

The two CFTR pair tests now assert their row count, so they cannot pass
with no row.

Bead: clin-15
Bead: clin-19
…ort goldens

The report-render goldens of the all and all-history cases print the
LPA section of the synthetic corpus summary, whose two risk-allele lines
read as an rsID with a genotype. They are invented test data: allow
those two lines in those goldens only.

Bead: clin-15
…on every run

A collected summary carries the minute og summary collect ran
(generated_utc), and both reports print it. The report-render goldens of
the cases with a row in hom_or_hemizygous therefore failed on the next
day's run. report-corpus.sh now writes one fixed time into every
collected summary (sed on that one line, so a summary that holds NaN or
Infinity on purpose stays as it is), and the goldens are written again
from og run inside the checkout, as CI runs it: the all and all-nf
summaries now read their repeat tables, as on CI. The pinned oracle
matches the old og on every other output (299 of 301; the other 2 are
the committed viral golden), and no new golden is stale.

The all and all-history text goldens also print the corpus's invented
repeat sizes: allow those two lines too.

Bead: clin-15
# Conflicts:
#	CHANGELOG.md
… call

The SF card called two_hits(), so a class 4/5 deletion plus a P/LP call
below step 06's call floor read 'biallelic' there, while the carrier list
of the same report read it as possibly biallelic (low-confidence call).
Two phased hets of which one is a low-confidence call also read
'biallelic' with no mark.

Each counted allele now carries its low_confidence mark. The deletion rule
goes through two_hits_with, and a pair whose proof needs a low-confidence
call reads 'possibly biallelic (low-confidence call)'. A confident
homozygous call or a confident pair stays 'biallelic'.

Bead: clin-19
…zygous rows

The Python server flags a sample whose carrier list holds only homozygous
or hemizygous rows (a man with only a GLA row). og live serve read only
'carriers', so it gave such a sample no attention flag.

Bead: clin-15
The report-render cases now compare whole reports. The 24 cases whose
summary holds a homozygous or hemizygous carrier row get whole-report
goldens; each differs from the oracle only in the carrier block. The
docs and the changelog say what the SF card now does with a pair on a
low-confidence call.

Bead: clin-15, clin-19
…ssense scores written as text

Bead: nf-vcfanno-ops
… a weak call

The two cards capped different pairs when a gene held a low-confidence
call. The carrier list capped a pair whenever any of its hets was weak,
so a confident phased pair beside a weak third het read "possible two
hits (low-confidence call)". The secondary-findings card capped only a
pair whose proof needed the weak call. Commit 6059508 also said a
confident pair stays biallelic beside a low-confidence call. That was
false for a confident deletion pair: the card gave it the low-confidence
word, while the same row's two-hit line said biallelic. And a combined
allele lost its confidence when any one of its calls was weak.

Both cards now count a gene's hits twice, with every call and with the
confident calls alone:

- A pair the confident calls prove is "two hits" on the list and
  "biallelic" on the card, whatever weak call sits beside it.
- On the card, a confident deletion pair that is biallelic or possibly
  biallelic is used before the low-confidence cap.
- A pair that only a weak call proves gets the low-confidence word.
- Hits of unknown phase are "possible two hits" and "possibly
  biallelic" when two of them are confident. Two confident unphased hets
  plus a weak homozygous call now read "possibly biallelic" on the card,
  as the carrier list already said.
- An allele counts the copies its confident calls give (a count, not a
  flag), so a combined allele that one confident call proves stays firm.
  The carrier list counts a combined allele the same way.

No reported or not-reported decision moves; only `pair` and `phase`
change. A new test runs MUTYH, which is on both lists, through both
cards over 25 shapes of hits and requires the same answer from both.

Bead: clin-19
…port

With step 22's consensus set and an SV whose support is null, both
reports must say the consensus cannot tell, never that it did not run,
in the AnnotSV rows and in the secondary-findings SV rows, and the
AnnotSV count line must count the unknown rows. Without the consensus
the old text stays. Red on the code before the fix: the header lacks
"not known: 1" and the row says "no step 22 consensus".

Bead: sv-01
…ll an SV's support

og summary collect writes an SV's support as null when step 22's
consensus ran but cannot tell (an SV_VCF run's rows, a row with no ID,
an ID in two events, a vote with no readable ID). Both renderers still
said "not known (no step 22 consensus)" for every null, and the AnnotSV
count line read "single caller: 0".

The AnnotSV section's consensus flag now picks the reason in the AnnotSV
rows and the secondary-findings SV rows, and the count line adds
", not known: M" when the consensus ran. bin/render_report.py and
bin/live_results.py change the same lines for the live site.

The pinned Python oracle prints the old reason, so the 50 report-render
cases that show it get whole-report goldens. report-corpus.sh pins each
collected summary's generated_utc so those goldens stay stable (the same
hunk as the carrier-confidence lane).

Bead: sv-01
…der goldens

The whole-report goldens of the c-full, all and a-/s- report-render
cases print the made-up lipoprotein(a) genotypes and repeat sizes of the
summary-collect corpus. The scan read them as personal data. Allow
those four lines under tests/parity/golden/report-render only.

Bead: sv-01
…t the 9 GB GATK one

GATK_IMAGE is pulled lazily, so the shard that ran this case pulled about
9 GB to bgzip five small tables. The bcftools image is cached and ships
bgzip and tabix.
The new count line read each row's support with d["support"] (Python)
and at("support") (og-render). A live-site summary whose rows have no
support key then raised, and the live site dropped the structural card:
tests/test_owngenome_live_family.py counted 0 flagged cards where it
expects 1. Both now count a missing key as not known, the same as null.

Bead: sv-01
Two confident hets of unknown phase beside a weak het in trans with one
of them read "possible two hits (low-confidence call)" on the carrier
list but "possibly biallelic" on the secondary-findings card. The card
checked two confident copies of unknown phase before the low cap; the
list capped first. The list now takes the same step in the same order.

A combined allele that is confident only through a folded part kept the
GT and phase set of its own weak call, so both cards could call a pair
biallelic that the confident calls do not prove. Each allele is now
counted with the GT and phase set of the confident call that makes it
confident, on both cards. On the card's deletion path a weak combined
allele with a confident part is no longer marked low when the
confident calls give its copies, and it pairs with the deletion by
that confident call's phase.

New shapes in the agreement test: R1, R1b, R1c (the list now says
possible two hits), R2 and R3d (the card now says possibly biallelic,
with its two-hit line), R3 (both now say possibly), and R6 and R7,
which keep the low word on both cards.

Bead: clin-19
The docs and the changelog said a gene on both lists gets the same
answer. That holds for a recessive gene the SF card counts by two
copies, and only at the gene level: on the carrier list the answer is
the best of `pair`, the `two_hits` verdict and the homozygous rows, and
`pair` counts heterozygous hits only. Say so, and name the new rule for
two confident copies of unknown phase.

Bead: clin-19
… reason too

bin/live_results.py called sf_rows without the AnnotSV consensus in the
clinical-filter table and in the findings list, so those rows still said
"no step 22 consensus" after step 22 ran, next to AnnotSV rows that said
it cannot tell. Both now pass the consensus, as the AnnotSV rows do.
The parity README names the new golden cases and the generated_utc pin,
and the two repeat-size allow lines cover only the text goldens.

Bead: sv-01
A HIGH-impact record of the clinical VCF for the same variant made a weak
call count as confident on the secondary-findings card, because the card
read only step 06's low_confidence mark. With the record's phase set it
proved a pair biallelic from a weak phase alone, and beside a deletion it
cleared the low-confidence word. The card now also treats a call whose
HIGH-impact record has weak_call as low confidence.

The carrier list paired a weak combined allele with a deletion by the weak
row's own GT and phase set, while the card used its confident part. The
list now pairs it the same way: by the first confident row of the same
allele with as many copies.

The card dropped the phase of a combined allele whose confident call gives
fewer copies than the weak one, although that phase came from the
confident call. Every small allele now counts with the copies, GT and phase
set of its confident call.

The agreement test gains eight shapes (H1, H2, H3p, H5, X4, X4b, X9b,
X10); each fails on the previous head.

Part of GeiserX#411 (clin-19).
A HIGH-impact record is weak on a skewed allele balance too, not only below
the clinical call floor. Such a call is now low confidence on the
secondary-findings card and confident on the carrier list, even when both
files state the same floor. The changelog and the results page said the
two lists agree whenever the floors match; they now name this case.

Part of GeiserX#411 (clin-19).
…balance

The docs said a HIGH-impact call weak on allele balance is low confidence
on the card only. It never lowers a hit whose ClinVar row is confident, so
that split does not exist. The real one is the HIGH-impact call of a part
of a combined allele, which makes the allele confident on the card while
the list marks it low confidence.

Bead: clin-19
…part

The carrier list rescued a weak part by its confident combined allele in
every case. The card counts a part through its combined allele only when
that allele is unphased and the part is its first part present, so a
phased combined allele inside a deletion made its weak part just outside
it read biallelic on the list. The SF card's deletion pair on a weak
HIGH-impact call also blamed the call floor; it now names both kinds of
low-confidence call.

Bead: clin-19
…t text

Main now prints the SV caller support as main's code words it ("caller
support not known (no step 22 consensus)" and the like), so the 38 report
goldens this PR adds no longer matched og. Each file is og's own output
from the merged tree; oracle-diff summary-collect report-render agrees on
all 1097 items, and no summary-collect golden moved.

Bead: clin-19
# Conflicts:
#	CHANGELOG.md
#	tests/parity/golden/report-render/all-history/all-history.txt
#	tests/parity/golden/report-render/all/all.txt
#	tests/parity/golden/report-render/cb-full/cb-full.html
#	tests/parity/golden/report-render/cb-full/cb-full.txt
#	tests/parity/golden/report-render/cb-quoted-dosage/cb-quoted-dosage.html
#	tests/parity/golden/report-render/cb-quoted-dosage/cb-quoted-dosage.txt
#	tests/parity/golden/report-render/cb-quoted-interval/cb-quoted-interval.html
#	tests/parity/golden/report-render/cb-quoted-interval/cb-quoted-interval.txt
#	tests/parity/golden/report-render/cb-quoted-smn/cb-quoted-smn.html
#	tests/parity/golden/report-render/cb-quoted-smn/cb-quoted-smn.txt
#	tests/parity/golden/report-render/cb-unicode-contig/cb-unicode-contig.html
#	tests/parity/golden/report-render/cb-unicode-contig/cb-unicode-contig.txt
#	tests/parity/golden/report-render/m-clinical-escape/m-clinical-escape.html
#	tests/parity/golden/report-render/m-clinical-escape/m-clinical-escape.txt
#	tests/parity/golden/report-render/m-clinical-stale/m-clinical-stale.html
#	tests/parity/golden/report-render/m-clinical-stale/m-clinical-stale.txt
#	tests/parity/golden/report-render/m-clinvar-freq-note/m-clinvar-freq-note.html
#	tests/parity/golden/report-render/m-clinvar-freq-note/m-clinvar-freq-note.txt
#	tests/parity/golden/report-render/m-clinvar-many/m-clinvar-many.html
#	tests/parity/golden/report-render/m-clinvar-many/m-clinvar-many.txt
#	tests/parity/golden/report-render/m-clinvar-none/m-clinvar-none.html
#	tests/parity/golden/report-render/m-clinvar-none/m-clinvar-none.txt
#	tests/parity/golden/report-render/m-couple-empty/m-couple-empty.html
#	tests/parity/golden/report-render/m-couple-empty/m-couple-empty.txt
#	tests/parity/golden/report-render/m-couple/m-couple.html
#	tests/parity/golden/report-render/m-couple/m-couple.txt
#	tests/parity/golden/report-render/m-cpsr-none/m-cpsr-none.html
#	tests/parity/golden/report-render/m-cpsr-none/m-cpsr-none.txt
#	tests/parity/golden/report-render/m-cpsr-noown/m-cpsr-noown.html
#	tests/parity/golden/report-render/m-cpsr-noown/m-cpsr-noown.txt
#	tests/parity/golden/report-render/m-sf-notes/m-sf-notes.html
#	tests/parity/golden/report-render/m-sf-notes/m-sf-notes.txt
#	tests/parity/golden/report-render/m-since-clinical/m-since-clinical.html
#	tests/parity/golden/report-render/m-since-clinical/m-since-clinical.txt
#	tests/parity/golden/report-render/m-slivar-phase/m-slivar-phase.html
#	tests/parity/golden/report-render/m-slivar-phase/m-slivar-phase.txt
#	tests/parity/golden/report-render/m-slivar-unphased/m-slivar-unphased.html
#	tests/parity/golden/report-render/m-slivar-unphased/m-slivar-unphased.txt
…t traced

A traced run skips the panel measure (measure_on in its file, GeiserX#572), so its
shard should not be planned around 2900 seconds of work that never runs.
…eiserX#571

GeiserX#559 (carrier confidence) and GeiserX#571 (SV support reason) each added the same
38 report-render goldens, each written by its own og. On the merged tree og
prints both changes, so the 38 files are og's output on this tip: every line
of each comes from one of the two sides, and oracle-diff now agrees on all
1097 summary-collect and report-render items (38 differed before).
…ed against the optimum

With the groups' allele frequencies fixed, the log-likelihood of a
person's fractions is concave on the simplex, so the maximum is unique.
og-admix now finds it with the EM update and SQUAREM steps that never
lower the likelihood, stops when no fraction moves by more than 1e-9,
and checks the KKT conditions. A run that stops short is an error, not
an answer. The module also cuts 5 cM blocks per chromosome and draws
the SNV weights and the reference people for a bootstrap replicate.

Bead: lin-49
Bead: D5
…panel bootstrap

og ancestry-fine project reads the person's genotypes at the model's
SNVs, projects them on each level's group frequencies, and writes the
.Q and boot.Q files report reads, plus a fit.tsv per level. Each
bootstrap replicate draws 5 cM blocks within chromosomes and draws each
group's reference people again from ref.geno, so the range includes how
well the panel knows each group. One seed gives the same bytes for any
thread count. A model without ref.geno refuses the bootstrap with one
line that says to run setup again.

report takes --fractions DIR and keeps --admixture DIR as an alias; its
sentence now names the projection.

Bead: lin-49
Bead: lin-52
Bead: D14
Step 26b and the Nextflow module now call og ancestry-fine project.
ANCESTRY_FINE_FRACTIONS replaces ANCESTRY_FINE_PREPARE and
ANCESTRY_FINE_ADMIXTURE, work/fractions replaces work/admixture, and
og ancestry-fine sample-bed with its generated ADMIXTURE script is
gone, with its parity cases. The e2e case checks the fit.tsv of each
level.

Bead: lin-49
Bead: D1
@geiresx
geiresx requested a review from GeiserX as a code owner November 22, 2026 00:54
@coderabbitai

coderabbitai Bot commented Nov 22, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 184 files, which is 84 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

CodeRabbit couldn't verify your organization's usage billing state, so no usage-priced review was started. Retry after the billing service recovers.

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 9dd5dda and dfdd462.

Files ignored due to path filters: 13

  • assets/output_contract.tsv is excluded by !**/*.tsv
  • assets/steps.tsv is excluded by !**/*.tsv
  • scripts/ci/e2e-case-paths.tsv is excluded by !**/*.tsv
  • scripts/ci/e2e-case-times.tsv is excluded by !**/*.tsv
  • tests/parity/cases.tsv is excluded by !**/*.tsv
  • tests/parity/gen/report-corpus.sh is excluded by !**/gen/**
  • tests/parity/inputs/ancestry-fine/sb-gz/continent.P is excluded by !**/*.p
  • tests/parity/inputs/ancestry-fine/sb-gz/g.vcf.gz is excluded by !**/*.gz
  • tests/parity/inputs/ancestry-fine/sb-min-sites/continent.P is excluded by !**/*.p
  • tests/parity/inputs/ancestry-fine/sb-p-rows/continent.P is excluded by !**/*.p
  • tests/parity/inputs/ancestry-fine/sb-p-rows/region.P is excluded by !**/*.p
  • tests/parity/inputs/ancestry-fine/sb/continent.P is excluded by !**/*.p
  • tests/parity/inputs/ancestry-fine/sb/region.P is excluded by !**/*.p
Files selected for processing: 184

  • .github/workflows/e2e.yml
  • CHANGELOG.md
  • bin/live_results.py
  • bin/owngenome_live.py
  • bin/render_report.py
  • conf/containers.config
  • docs/05-annotsv.md
  • docs/25-prs.md
  • docs/49-chip.md
  • docs/nextflow.md
  • docs/parameters.md
  • docs/results/secondary-findings.md
  • docs/testing.md
  • main.nf
  • modules/local/ancestry_fine/main.nf
  • modules/local/chip/main.nf
  • modules/local/vcfanno/main.nf
  • nextflow.config
  • nextflow_schema.json
  • rust/crates/og-admix/src/lib.rs
  • rust/crates/og-admix/src/project.rs
  • rust/crates/og-ancestry/src/lib.rs
  • rust/crates/og-ancestry/src/project.rs
  • rust/crates/og-ancestry/src/report.rs
  • rust/crates/og-ancestry/src/run_admixture.sh.in
  • rust/crates/og-ancestry/src/sample_bed.rs
  • rust/crates/og-live/src/state.rs
  • rust/crates/og-render/src/sections/acmg_sf.rs
  • rust/crates/og-render/src/sections/annotsv.rs
  • rust/crates/og-render/src/sections/carrier.rs
  • rust/crates/og-render/tests/clinical.rs
  • rust/crates/og-render/tests/i411_sv_support_text.rs
  • rust/crates/og-summary/src/collect/acmg.rs
  • rust/crates/og-summary/src/collect/carrier.rs
  • rust/crates/og-summary/src/collect/two_hits.rs
  • rust/crates/og-summary/tests/i411_carrier_confidence.rs
  • rust/og/src/main.rs
  • scripts/26b-ancestry-fine.sh
  • scripts/49-chip.sh
  • scripts/ci/check-personal-data.sh
  • scripts/ci/gen-containers-config.sh
  • scripts/run-all.sh
  • tests/e2e/ancestry-fine-grained-1.sh
  • tests/e2e/i411-nf-vcfanno-ops-1-score-tiers.sh
  • tests/fake-docker/i411-chip-contamination.sh
  • tests/fixtures/live/demo-001.weblog.jsonl
  • tests/parity/README.md
  • tests/parity/golden/ancestry-fine/report-no-painting/S_ancestry_fine.txt
  • tests/parity/golden/ancestry-fine/report-no-painting/stdout
  • tests/parity/golden/ancestry-fine/report/out/S_ancestry_fine.txt
  • tests/parity/golden/ancestry-fine/report/stdout
  • tests/parity/golden/ancestry-fine/sample-bed-gz/run_admixture.sh
  • tests/parity/golden/ancestry-fine/sample-bed-p-missing/stderr
  • tests/parity/golden/ancestry-fine/sample-bed-p-rows/adm/S_continent.3.P.in.absent
  • tests/parity/golden/ancestry-fine/sample-bed-p-rows/adm/S_continent.bed.absent
  • tests/parity/golden/ancestry-fine/sample-bed-p-rows/adm/S_continent.bim.absent
  • tests/parity/golden/ancestry-fine/sample-bed-p-rows/adm/S_continent.fam.absent
  • tests/parity/golden/ancestry-fine/sample-bed/adm/run_admixture.sh
  • tests/parity/golden/report-render/a-escape/a-escape.html
  • tests/parity/golden/report-render/a-escape/a-escape.txt
  • tests/parity/golden/report-render/a-failed/a-failed.html
  • tests/parity/golden/report-render/a-failed/a-failed.txt
  • tests/parity/golden/report-render/a-full/a-full.html
  • tests/parity/golden/report-render/a-full/a-full.txt
  • tests/parity/golden/report-render/a-missing/a-missing.html
  • tests/parity/golden/report-render/a-missing/a-missing.txt
  • tests/parity/golden/report-render/a-no-painting/a-no-painting.html
  • tests/parity/golden/report-render/a-no-painting/a-no-painting.txt
  • tests/parity/golden/report-render/a-numbers/a-numbers.txt
  • tests/parity/golden/report-render/a-painting/a-painting.html
  • tests/parity/golden/report-render/a-painting/a-painting.txt
  • tests/parity/golden/report-render/a-partial/a-partial.html
  • tests/parity/golden/report-render/a-partial/a-partial.txt
  • tests/parity/golden/report-render/a-prs-raw/a-prs-raw.html
  • tests/parity/golden/report-render/a-prs-raw/a-prs-raw.txt
  • tests/parity/golden/report-render/a-prs-sex/a-prs-sex.html
  • tests/parity/golden/report-render/a-prs-sex/a-prs-sex.txt
  • tests/parity/golden/report-render/a-since/a-since.html
  • tests/parity/golden/report-render/a-since/a-since.txt
  • tests/parity/golden/report-render/a-stale/a-stale.html
  • tests/parity/golden/report-render/a-stale/a-stale.txt
  • tests/parity/golden/report-render/a-unreadable/a-unreadable.html
  • tests/parity/golden/report-render/a-unreadable/a-unreadable.txt
  • tests/parity/golden/report-render/a-y-calls/a-y-calls.html
  • tests/parity/golden/report-render/a-y-calls/a-y-calls.txt
  • tests/parity/golden/report-render/all-history/all-history.txt
  • tests/parity/golden/report-render/all-nf/all-nf.html
  • tests/parity/golden/report-render/all-nf/all-nf.txt
  • tests/parity/golden/report-render/all/all.txt
  • tests/parity/golden/report-render/c-float--inf/c-float--inf.txt
  • tests/parity/golden/report-render/c-float-inf/c-float-inf.txt
  • tests/parity/golden/report-render/c-float-nan/c-float-nan.txt
  • tests/parity/golden/report-render/c-full/c-full.txt
  • tests/parity/golden/report-render/cb-full/cb-full.html
  • tests/parity/golden/report-render/cb-full/cb-full.txt
  • tests/parity/golden/report-render/cb-nf/cb-nf.html
  • tests/parity/golden/report-render/cb-nf/cb-nf.txt
  • tests/parity/golden/report-render/cb-nfbad/cb-nfbad.html
  • tests/parity/golden/report-render/cb-nfbad/cb-nfbad.txt
  • tests/parity/golden/report-render/cb-no-assets/cb-no-assets.html
  • tests/parity/golden/report-render/cb-no-assets/cb-no-assets.txt
  • tests/parity/golden/report-render/cb-notbi/cb-notbi.html
  • tests/parity/golden/report-render/cb-notbi/cb-notbi.txt
  • tests/parity/golden/report-render/cb-quoted-dosage/cb-quoted-dosage.html
  • tests/parity/golden/report-render/cb-quoted-dosage/cb-quoted-dosage.txt
  • tests/parity/golden/report-render/cb-quoted-interval/cb-quoted-interval.html
  • tests/parity/golden/report-render/cb-quoted-interval/cb-quoted-interval.txt
  • tests/parity/golden/report-render/cb-quoted-smn/cb-quoted-smn.html
  • tests/parity/golden/report-render/cb-quoted-smn/cb-quoted-smn.txt
  • tests/parity/golden/report-render/cb-unicode-contig/cb-unicode-contig.html
  • tests/parity/golden/report-render/cb-unicode-contig/cb-unicode-contig.txt
  • tests/parity/golden/report-render/m-clinical-escape/m-clinical-escape.html
  • tests/parity/golden/report-render/m-clinical-escape/m-clinical-escape.txt
  • tests/parity/golden/report-render/m-clinical-failed/m-clinical-failed.html
  • tests/parity/golden/report-render/m-clinical-failed/m-clinical-failed.txt
  • tests/parity/golden/report-render/m-clinical-stale/m-clinical-stale.html
  • tests/parity/golden/report-render/m-clinical-stale/m-clinical-stale.txt
  • tests/parity/golden/report-render/m-clinical-unreadable/m-clinical-unreadable.html
  • tests/parity/golden/report-render/m-clinical-unreadable/m-clinical-unreadable.txt
  • tests/parity/golden/report-render/m-clinvar-freq-note/m-clinvar-freq-note.html
  • tests/parity/golden/report-render/m-clinvar-freq-note/m-clinvar-freq-note.txt
  • tests/parity/golden/report-render/m-clinvar-many/m-clinvar-many.html
  • tests/parity/golden/report-render/m-clinvar-many/m-clinvar-many.txt
  • tests/parity/golden/report-render/m-clinvar-none/m-clinvar-none.html
  • tests/parity/golden/report-render/m-clinvar-none/m-clinvar-none.txt
  • tests/parity/golden/report-render/m-couple-empty/m-couple-empty.html
  • tests/parity/golden/report-render/m-couple-empty/m-couple-empty.txt
  • tests/parity/golden/report-render/m-couple/m-couple.html
  • tests/parity/golden/report-render/m-couple/m-couple.txt
  • tests/parity/golden/report-render/m-cpsr-none/m-cpsr-none.html
  • tests/parity/golden/report-render/m-cpsr-none/m-cpsr-none.txt
  • tests/parity/golden/report-render/m-cpsr-noown/m-cpsr-noown.html
  • tests/parity/golden/report-render/m-cpsr-noown/m-cpsr-noown.txt
  • tests/parity/golden/report-render/m-sf-notes/m-sf-notes.html
  • tests/parity/golden/report-render/m-sf-notes/m-sf-notes.txt
  • tests/parity/golden/report-render/m-since-clinical/m-since-clinical.html
  • tests/parity/golden/report-render/m-since-clinical/m-since-clinical.txt
  • tests/parity/golden/report-render/m-slivar-phase/m-slivar-phase.html
  • tests/parity/golden/report-render/m-slivar-phase/m-slivar-phase.txt
  • tests/parity/golden/report-render/m-slivar-unphased/m-slivar-unphased.html
  • tests/parity/golden/report-render/m-slivar-unphased/m-slivar-unphased.txt
  • tests/parity/golden/report-render/s-consumer-reasons/s-consumer-reasons.html
  • tests/parity/golden/report-render/s-consumer-reasons/s-consumer-reasons.txt
  • tests/parity/golden/report-render/s-consumer-stale/s-consumer-stale.html
  • tests/parity/golden/report-render/s-consumer-stale/s-consumer-stale.txt
  • tests/parity/golden/report-render/s-sexloss-chrx/s-sexloss-chrx.html
  • tests/parity/golden/report-render/s-sexloss-chrx/s-sexloss-chrx.txt
  • tests/parity/golden/report-render/s-sexloss-detected/s-sexloss-detected.html
  • tests/parity/golden/report-render/s-sexloss-detected/s-sexloss-detected.txt
  • tests/parity/golden/report-render/s-sexloss-failed/s-sexloss-failed.html
  • tests/parity/golden/report-render/s-sexloss-failed/s-sexloss-failed.txt
  • tests/parity/golden/report-render/s-sexloss-gain-stale/s-sexloss-gain-stale.html
  • tests/parity/golden/report-render/s-sexloss-gain-stale/s-sexloss-gain-stale.txt
  • tests/parity/golden/report-render/s-sexloss-not-estimated/s-sexloss-not-estimated.html
  • tests/parity/golden/report-render/s-sexloss-not-estimated/s-sexloss-not-estimated.txt
  • tests/parity/golden/report-render/s-sexloss-odd/s-sexloss-odd.html
  • tests/parity/golden/report-render/s-sexloss-odd/s-sexloss-odd.txt
  • tests/parity/golden/report-render/s-sv-stale/s-sv-stale.html
  • tests/parity/golden/report-render/s-sv-stale/s-sv-stale.txt
  • tests/parity/golden/summary-collect/all-history/all-history.json
  • tests/parity/golden/summary-collect/all-nf/all-nf.json
  • tests/parity/golden/summary-collect/all/all.json
  • tests/parity/golden/summary-collect/cb-full/cb-full.json
  • tests/parity/golden/summary-collect/cb-nf/cb-nf.json
  • tests/parity/golden/summary-collect/cb-nfbad/cb-nfbad.json
  • tests/parity/golden/summary-collect/cb-notbi/cb-notbi.json
  • tests/parity/golden/summary-collect/cb-quoted-dosage/cb-quoted-dosage.json
  • tests/parity/golden/summary-collect/cb-quoted-interval/cb-quoted-interval.json
  • tests/parity/golden/summary-collect/cb-quoted-smn/cb-quoted-smn.json
  • tests/parity/golden/summary-collect/cb-unicode-contig/cb-unicode-contig.json
  • tests/parity/inputs/ancestry-fine/sb-gz/ref.bim
  • tests/parity/inputs/ancestry-fine/sb-min-sites/g.vcf
  • tests/parity/inputs/ancestry-fine/sb-min-sites/ref.bim
  • tests/parity/inputs/ancestry-fine/sb-p-missing/g.vcf
  • tests/parity/inputs/ancestry-fine/sb-p-missing/ref.bim
  • tests/parity/inputs/ancestry-fine/sb-p-rows/g.vcf
  • tests/parity/inputs/ancestry-fine/sb-p-rows/ref.bim
  • tests/parity/inputs/ancestry-fine/sb/g.vcf
  • tests/parity/inputs/ancestry-fine/sb/ref.bim
  • tests/schema/summary.schema.json
  • tests/shellcheck-baseline.txt
  • tests/test_ancestry_fine_report.py
  • tests/test_vcfanno_toml.sh
  • workflows/clinical.nf

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@geiresx

geiresx commented Nov 22, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Action not completed

Review skipped: 183 files exceed the limit of 100.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

The new fit.tsv check passes an awk program through check, so
ShellCheck cannot see it is awk and reports SC2016. Disable it on that
line with the reason, as the script does elsewhere, instead of growing
the baseline.

Bead: lin-52
A frequency such as 1e-310 is strictly between 0 and 1, so the problem
accepts it, but 2 / 1e-310 overflows. The EM step then turns the
fractions into NaN, f64::max drops the NaN, the step size reads 0 and
the run passed as converged with NaN fractions and log-likelihood.
project now refuses a fit whose fractions, log-likelihood or KKT
measures are not finite. The new test failed before the change.

Bead: lin-49
The module doc said only a record with the .bim's two alleles decides
a SNV. The code, like the sample-bed command it replaces, lets the last
record with a called GT decide, and leaves the SNV out when that record
holds another allele. The doc now states the rule the code follows.

Bead: lin-52
The demo weblog is generated from assets/steps.tsv, which now lists
ANCESTRY_FINE_FRACTIONS in place of ANCESTRY_FINE_PREPARE and
ANCESTRY_FINE_ADMIXTURE, so test_live_demo.py's check that the fixture
is what make_demo_events.py writes failed. This is the generator's
output; only that process set changed, and the later hashes and times
shift with it.

Bead: lin-49

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants