Skip to content

test(e2e): verify explicit previews and retain failure traces - #176

Merged
anantham merged 8 commits into
mainfrom
test/codex-production-qa
Sep 6, 2026
Merged

test(e2e): verify explicit previews and retain failure traces#176
anantham merged 8 commits into
mainfrom
test/codex-production-qa

Conversation

@anantham

@anantham anantham commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Summary

The end-to-end runner could silently reuse a stale server, while a failed test with zero retries produced no trace. Reject occupied managed-server ports, allow an explicit LF_E2E_BASE_URL for a reviewed production preview, and retain traces on failure.

Changes

Update the existing Playwright configuration and QA guide. Document pinned setup, strict preview ports, isolated fresh/warm browser fixtures and trace inspection; remove stale schema and installation instructions. No additional orchestration layer or dependencies.

Testing

Node 24.19.0: three existing route tests pass against an explicit production preview; TypeScript and focused lint pass. An occupied-port probe proves the managed runner refuses a stale listener. A disposable intentional failure with zero retries produces trace.zip; the intentionally failing test is not part of the committed suite.

Review and integration

Retargeted to main after #174, #173 and #175 merged. Final head fb26903297134539dea774f54445eb7a755bd1bc preserves the independently reviewed source; its last main merge changes ancestry only. All five jobs pass on this head (run 34036090307).

The final combined source passes 175 focused Node 24.19.0 tests, TypeScript, production build/privacy scan, three production routes and six synthetic reader contexts. Anthropic Claude Sonnet 5 independently approved the complete 12-file source packet (run gen-1788700103-OHAQEWMtZBjOLSYxnYwz); prior Codex review covers the unchanged QA config. The reviewer did not run tests or exercise a real backend.

Issues, debt inbox and WORKLOG preserve exact review/test/parent-merge evidence and actionable consolidation blockers. QA-03 remains partial: a representative complete novel and fresh/warm acceptance still need execution. Synthetic routes and reader fixtures do not prove full-book, Safari or physical-device behavior. No private runtime deployment was performed.

Use the existing Playwright config for local production previews, refuse accidental dev-server reuse and retain traces without requiring a retry. Document locked setup and fresh/warm fixture boundaries.

Verification: three production route tests pass; occupied-port refusal and first-failure trace retention verified with temporary diagnostics. No dependencies or runtime behavior changed.
@vercel

vercel Bot commented Sep 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
lexicon-forge Ready Ready Preview Sep 6, 2026 1:26pm UTC

@anantham

anantham commented Sep 5, 2026

Copy link
Copy Markdown
Owner Author

@codex review Please review the current head and focused diff against #173. Existing production route checks pass through the explicit preview URL; a disposable probe verifies occupied-port rejection, and an intentional zero-retry failure produces a trace. No dependencies or runtime services were changed.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8e30c96e8f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Issues.md Outdated
@anantham

anantham commented Sep 5, 2026

Copy link
Copy Markdown
Owner Author

Addressed the current-head review: QA-03 is explicitly partial and remains open for a scrubbed representative novel, fresh/warm session-harness checks, and a fixture/revision receipt. #176 only claims the tested runner and setup improvements. @codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e437faa735

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/infrastructure/E2E-TESTING.md Outdated
@anantham

anantham commented Sep 5, 2026

Copy link
Copy Markdown
Owner Author

The preview commands now use each worktree own ignored dist directory and require separate strict ports for concurrent previews. The guide also prohibits rebuilding that worktree while its tests run. No shared temporary output directory remains. QA-03 remains partial for representative novel/fresh-warm fixture acceptance. @codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: 0e695007d2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Preserve both worklog receipts and the partial QA acceptance status. QA implementation is unchanged; eight combined app-screen/dialog tests and TypeScript pass. Fresh CI and exact-head review follow.
Context: Consolidate the approved latency and QA stack without rewriting history.\nChanges: Merge #175 and preserve both task receipts in WORKLOG.\nImpact: The QA branch exercises the combined startup and reader source.\nTests: Node 24.19 focused 175 tests, types, build, privacy scan, three production routes and six reader contexts pass.\nDocs: WORKLOG records validation and pending independent review.
@anantham
anantham changed the base branch from perf/codex-startup-latency to perf/codex-reader-latency September 6, 2026 13:01
@anantham
anantham changed the base branch from perf/codex-reader-latency to main September 6, 2026 13:22
Context: Keep remaining consolidation work actionable while the latency parents merge.
Changes: Record exact source review, focused and browser checks, merged parent commits and four reproduced debt receipts; correct completed issue statuses.
Impact: Future agents can distinguish merged code from deferred policy and live acceptance.
Tests: Documentation only; reviewed source hashes unchanged, diff check clean.
Docs: Issues.md, WORKLOG and TECH-DEBT-INBOX.
@anantham

anantham commented Sep 6, 2026

Copy link
Copy Markdown
Owner Author

Final integration review for fb26903297134539dea774f54445eb7a755bd1bc: all 12 source/test hashes still match the combined tested packet; the final main-parent merge has an identical before/after Git tree. Only public-safe handoff/debt/merge records were added after source review.

Independent reviewer: Anthropic Claude Sonnet 5; run gen-1788700103-OHAQEWMtZBjOLSYxnYwz; APPROVE, complete 12-file inventory and end marker confirmed. Packet SHA-256 5fcb72ccaed1e98168ac61156003996aa7ec45cd731b4c6c15c984133aa742c2. No tools or private data supplied. Existing Codex review of the unchanged QA config remains applicable.

All five final-head CI jobs pass in run 34036090307. Combined local evidence: 175 focused Node 24.19.0 tests, types/build/privacy scan, three production routes and six synthetic reader runs pass. Keep QA-03 partial pending representative-novel/fresh-warm acceptance; no real scan or physical-device proof is claimed.

@anantham
anantham merged commit b1af513 into main Sep 6, 2026
8 checks passed
@anantham
anantham deleted the test/codex-production-qa branch September 6, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant