Skip to content

Preserve material distinctions in rendered stories - #118

Draft
R055LE wants to merge 3 commits into
issue-110-model-evaluationfrom
issue-115-rendered-completeness
Draft

R055LE wants to merge 3 commits into
issue-110-model-evaluationfrom
issue-115-rendered-completeness

Conversation

@R055LE

@R055LE R055LE commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

Summary

Closes #115. Stacked on #113 (issue-110-model-evaluation), which depends on #109.

  • Review the actual public render for material omissions with bounded, source-grounded diagnostics.
  • Judge implied relationships across the narrative, including distinct experimental settings.
  • Keep each item's exact citation support independent from broader completeness evidence.
  • Reject unchanged failed items and hidden-only repairs; preserve one revision and four calls maximum.
  • Share review payload/parsing between production and evaluation. Explicitly preserve required citations through revision.

Validation

  • Full suite: 634 passed, 87.38% coverage; affected suite 112 passed.
  • Final compact revision wording passed focused context/revision tests, Ruff and diff checks.
  • Independent code review: no blockers.
  • Final-head CI passed on Python 3.12 and 3.14: https://github.com/R055LE/roger/actions/runs/38474683818
  • Live reviewer calibration: 6/6 exact authored-gate matches. All six complete request records remain byte-identical at this final source.

Integrated acceptance remains incomplete

The final held-out render preserves the material experimental distinctions and caveats. Both reviews pass the completeness gate. However, the complete workflow is rejected: revision replaces the accepted stage citation with one describing only construction of the testbed while retaining stage: demonstrated. Final support review and independent assessment reject that item. The remaining grounded revision/stage compliance belongs to #107.

Earlier false acceptance, the402 interruption, and the empty-citation revision remain preserved. No accepted workflow or human editorial acceptance is claimed. The optional marketing workflow repeat was not run; its material-omission controls passed calibration.

#115 used 20 attempts/$0.5201765 known plus $0.74 uncertain reserve. Full study: 196/196 attempts, $3.06141310 known plus $1.04 retained uncertainty. Further paid testing needs a new call allowance. Provider-credit restoration succeeded; funding is no longer the blocker.

Limits

Worst-case rejected revision is 17,994/18,000 characters. An all-eight maximum-length-source combine can still exceed the unchanged review ceiling and fail closed without truncation. Repository regressions are synthetic reproductions; captured originals remain immutable.

Draft pending integrated acceptance and dependent PRs. Human editorial and Scout acquisition acceptance remain separate. No production activation or publication.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant