feat(liturgy): add semantic alignment audit - #162
Conversation
Context: Existing corpus gates checked reconstruction and index ranges but could not expose unauthored many-to-one semantic precision. Changes: Add renderer-compatible source tokenization, rejecting layered-target validation, exhaustive registered-route auditing, and a CLI ledger. Impact: Invalid analysis targets fail loudly and every dangerous unreviewed group is route-visible without auto-curating sacred text. Tests: 11 new adversarial validator and audit tests; existing 1,639 alignment/accent cases remain green. Docs: Audit convention and remaining-route ledger follow in the stacked curation change. ADR: LITURGY-001
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
anantham
left a comment
anantham
left a comment
There was a problem hiding this comment.
Gemini/Google adversarial review
REVIEWED_HEAD: 964c17acfee31f4983a33e26621712aa29b259d7
Provider run: 3f84d990-7920-45e4-af5c-0e5352275ce2 (agy, Gemini 3.1 Pro High, successful)
Finding 1: stale field and validation failure in community resolution
Severity: P1
File: data/liturgy/resolve.ts:44-47
stripAlignment removes alignTo and morphemeAlignTo but silently retains the new tokenAlignTo field through rest destructuring. resolveAll can therefore pool a foreign witness with tokenAlignTo present and alignTo absent. The new validator rejects that state with fine_alignment_without_word_alignment.
User impact: Once a community authors tokenAlignTo on a shared chant, another community's resolved document can fail validation or expose dangling fine-grained targets.
Falsifiable reproduction: Resolve a community chant whose foreign witness has valid alignTo plus tokenAlignTo. Confirm the pooled witness loses alignTo but retains tokenAlignTo, then pass the document to validateLiturgyDoc and observe fine_alignment_without_word_alignment.
Finding 2: silent audit omission and misleading metrics
Severity: P2
File: services/liturgy/alignmentAudit.ts:103-109
The audit traverses resolved LITURGY_DOCS_BY_SANGHA documents and increments englishTokens for every pooled witness before skipping witnesses without alignTo. Foreign pooled witnesses therefore inflate the denominator while contributing nothing to alignedEnglishTokens.
User impact: Curators can receive a falsely low apparent alignment-coverage ratio for shared chants, obscuring the actual review state.
Falsifiable reproduction: Audit one shared chant resolved for two sanghas with one fully aligned ten-token witness. The summary can report twenty English tokens but only ten aligned tokens, suggesting 50% rather than 100% authored-witness coverage.
Evidence and uncertainty
The self-contained successful run inspected the exact base-to-head diff plus data/liturgy/resolve.ts, services/liturgy/alignmentTargets.ts, types/liturgy.ts, components/liturgy/shapes/TripleScriptWitness.tsx, and data/liturgy/index.ts. Philological correctness and source authority remain a separate human/domain review gate.
VERDICT: REVISE
# Conflicts: # services/liturgy/validation.ts # tests/services/liturgy/semanticAlignmentValidation.test.ts
Context: route-visible, authored-witness, and source-linked English token counts were collapsed under ambiguous audit labels. Changes: expose all three populations, label pooled/unmapped inclusion in the CLI, and prove route duplication plus -1 handling in a synthetic test. Impact: curators can interpret coverage without mistaking route-visible inventory for source alignment coverage; issue detection is unchanged. Tests: 1,273 passed and 327 skipped across focused inherited/audit suites; TypeScript, focused ESLint, corpus CLI, and diff check pass. Docs: implement the audit counting contract and append LITURGY-001 implementation notes. Fixes: PR #162 adversarial-review denominator finding ADR: LITURGY-001
# Conflicts: # docs/WORKLOG.md
anantham
left a comment
anantham
left a comment
There was a problem hiding this comment.
Independent Gemini adversarial review
Provider: Google via agy
Model: gemini-3.7-flash-high
Run: 92bd65f7-a6c7-4ee3-9f7e-e66ec4364c63
Exact reviewed head: 9d73900750f0e8573044d26a9f6a513519a538df
Gemini reviewed PR #162’s isolated stacked delta against the approved PR #161 head. It verified the three explicit denominators, all 23 registered route contexts, direct reuse of renderer target resolution, review-required issue classification, JSON and human-readable CLI contracts, unaligned/supplied-token edge cases, and focused test/type/lint evidence.
Actionable findings: None.
VERDICT: APPROVE
1 participant
Summary
Add fail-loud validation and an exhaustive route audit for semantic alignment precision. The audit exposes dangerous unreviewed many-to-one groups without automatically curating sacred text.
Changes
Testing
Review Checklist
5019420029, run92bd65f7-a6c7-4ee3-9f7e-e66ec4364c63, head9d73900750f0e8573044d26a9f6a513519a538df.Stack
PR 2 of 3. Base is PR 1's exact approved branch head and this PR should merge after PR 1.