Skip to content

test(web-search): drive the connect deadline on a virtual clock - #4867

Merged
lidge-jun merged 2 commits into
devfrom
codex/2580-websearch-virtual-clock
Sep 17, 2026
Merged

lidge-jun merged 2 commits into
devfrom
codex/2580-websearch-virtual-clock

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Sep 17, 2026

Copy link
Copy Markdown
Owner

Summary

BUG-R86 routed web-search timeout semantics > fast headers plus raw byte progress can outlive connectTimeoutMs timed out at 1002.37ms against its own 1000ms ceiling on macos 2/2 of run 35186268515.

Its siblings in the same describe run in 2.58, 3.48, 2.57, 3.06 and 14.65ms. This case was three orders of magnitude slower than everything around it and failing by 2.37ms — the signature of a case racing the scheduler rather than testing anything about the scheduler.

It was spending that second on five real 12ms waits used to walk a real deadline forward. Real sleeps under a real ceiling race the same scheduler that runner load perturbs, so a case about timeout semantics had quietly become a case about how busy the machine was.

It now drives the existing clearableDeadline seam with a virtual elapsed counter. The first byte lands at a virtual 26ms against a 25ms deadline, which is exactly the condition the case exists to describe: headers that arrive fast must clear the deadline before the body is consumed, and continuing byte progress must not re-arm it. Body, cancellation, status and completion assertions are unchanged.

The 1000ms ceiling is unchanged. Nothing was widened — the wall-clock wait was removed instead.

Verification

No local suite, focused test, typecheck, build, or install was run; this lane is hosted-CI-only by task contract. Verification is static plus exact-head hosted CI.

Ablation, which is what makes this a real test rather than one that merely stopped failing: leave the deadline armed, or move its clear to first-byte progress, and the abort lands before "a" is enqueued, so the body and completion assertions fail.

The seam used already exists in this subsystem; no new pattern was introduced and no production code changed.

git diff --check clean.

Checklist

  • Wall-clock wait removed rather than the budget raised
  • No timeout widened, no retry added, no test skipped
  • Every property the case asserted is preserved
  • No production behaviour changed
  • Targets dev

Summary by CodeRabbit

  • Tests
    • Improved web search timeout coverage using deterministic virtual timing instead of wall-clock delays.
    • Added validation that response progress does not incorrectly trigger connection timeouts or cancel the response body.
    • Strengthened checks for timeout setup, cleanup, and successful request completion.
    • Added safeguards to ensure test instrumentation is restored after each test.

`fast headers plus raw byte progress can outlive connectTimeoutMs` timed out at
1002.37ms against its own 1000ms ceiling on macOS 2/2 of run 35186268515. Its
siblings in the same describe run in 2.58, 3.48, 2.57, 3.06 and 14.65ms, so this
case was three orders of magnitude slower than everything around it and sitting
2.37ms outside a bound it could not reliably clear.

It was spending that second on five real 12ms waits used to walk a real deadline
forward. Real sleeps and a real ceiling race the same scheduler, so runner load
decides the outcome - which is how a case about timeout SEMANTICS became a case
about how busy the machine was.

It now drives the existing clearableDeadline seam with a virtual elapsed counter.
The first byte lands at a virtual 26ms against a 25ms deadline, which is the exact
condition the case exists to describe: headers that arrive fast must clear the
deadline before the body is consumed, and continuing byte progress must not re-arm
it. Body, cancellation, status and completion assertions are unchanged.

The 1000ms ceiling stays exactly where it was. Nothing was widened; the wall-clock
wait was removed instead.

Ablation: leave the deadline armed, or move its clear to first-byte progress, and
the abort lands before "a" is enqueued, so the body and completion assertions fail.

No local suite, focused test, typecheck, build, or install was run.
@lidge-jun
lidge-jun requested a review from Ingwannu as a code owner September 17, 2026 06:08
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-17T06:11:10.301049Z ffddc89 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the chore Maintenance, CI, tests, refactors, or build changes (not a user-facing bug or feature). label Sep 17, 2026
@coderabbitai

coderabbitai Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 38a6f8b5-f34f-4401-ae01-c234eac2ed85

📥 Commits

Reviewing files that changed from the base of the PR and between ffddc89 and db5173c.

📒 Files selected for processing (1)
  • tests/web-search/web-search.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review.


📝 Walkthrough

Walkthrough

The web-search timeout test now uses virtual time and a mocked deadline. It validates deadline creation and cleanup, prevents synthetic timeout abortion during body progress, confirms the body is not cancelled, and restores the spy.

Changes

Web-search timeout test

Layer / File(s) Summary
Virtual timeout validation
tests/web-search/web-search.test.ts:1-2, tests/web-search/web-search.test.ts:964-1013
The test spies on clearableDeadline, drives a pull-based stream with virtual time, verifies deadline cleanup and response completion, checks that the body is not cancelled, and restores the spy in a finally block.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~12 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to db517

The timeout test is stabilized while retaining coverage that response headers clear the connect deadline before body consumption. No actionable merge risk remains.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change in tests/web-search/web-search.test.ts: replacing real-time waiting with a virtual clock for the connect deadline.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/2580-websearch-virtual-clock

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lidge-jun

Copy link
Copy Markdown
Owner Author

리뷰 · 우선순위 77 / 80

이 PR은 tests/web-search/web-search.test.ts 한 파일만 고칩니다. 대상은 BUG-R86 routed web-search timeout semantics 안의 fast headers plus raw byte progress can outlive connectTimeoutMs 테스트입니다. 프로덕션 코드는 손대지 않습니다.

예전에 이 테스트는 본문 청크마다 진짜 setTimeout(12ms)를 다섯 번 걸어서, 헤더는 빨리 오고 바이트는 connectTimeoutMs(25ms)보다 느리게 흘러도 스트림이 살아 있는지 봤습니다. 의도는 맞습니다. 그런데 진짜 잠과 진짜 데드라인이 같은 스케줄러를 쓰니까, 러너가 바쁘면 1초 천장에 걸립니다. macOS 2/2 (Actions run 35186268515)에서 1002.37ms로 자기 천장(1000ms)을 2.37ms 넘겼습니다. 같은 describe 형제들은 2~15ms인데 이 케이스만 세 자릿수 ms로 느렸습니다. 타임아웃 의미를 보는 테스트가, 머신 바쁨을 보는 테스트로 바뀐 상태였습니다.

지금 dev의 실제 계약은 src/web-search/loop.ts에 이미 있습니다. 448줄 근처에서 clearableDeadline(connectTimeoutMs, signal)로 헤더 데드라인을 걸고, 최종 헤더가 오면 573줄에서 headerDeadline.clear()로 타이머만 끕니다. 그 다음에야 본문을 읽습니다. 609줄 finally에서도 한 번 더 clear합니다. 바이트가 계속 와도 데드라인을 다시 걸지 않습니다. 이 PR은 그 기존 src/lib/abort.tsclearableDeadline 심을 spyOn으로 가로채서, 가상 경과 시간으로 같은 조건을 만듭니다. 첫 바이트가 가상 26ms에 오고 데드라인은 25ms입니다. 헤더 clear가 먼저 일어나야만 본문이 살아남습니다.

바꾼 점은 네 가지입니다. (1) 진짜 sleep 제거. (2) ReadableStreampull + highWaterMark: 0으로 바꿔 청크마다 가상 시계를 전진. (3) performance.now() >= 50 같은 벽시계 하한을 virtualElapsedMs > connectTimeoutMs, deadlineCreations === 1, deadlineClears > 0, abort 안 됨으로 교체. (4) finally에서 spy 복구. 천장 1000ms는 그대로 두고, 타임아웃 늘리기·재시도·스킵은 없습니다. #4846 행동 오라클, #4860 스트림 완료 오라클, #4833 spill 공유 시계와 같은 줄입니다.

현재 dev HEAD는 e18ca2463(#4862 config show가 connect 그래프를 안 끌어옴)이고 패키지는 2.58.0입니다. 이 PR은 그 tip 위에 얹히는 테스트 전용 수정이라 types.ts/config.ts 분할과도 무관하고 충돌 위험이 거의 없습니다. 호스티드 CI 전용 계약이라 로컬 스위트는 안 돌렸고, 이 시점 hygiene/enforce-target/label 등은 통과, Linux/macOS 테스트·키링·도커 등은 진행 중입니다. 플레익이 macOS에서 났으니 macOS exact-head가 초록인지가 핵심입니다.

라인 961-1020 - spyOn(abortModule, "clearableDeadline")timeoutMs === connectTimeoutMs일 때만 mock하고 나머지는 원본으로 넘긴다. 이 케이스가 보는 심과 맞고, 다른 타임아웃을 우연히 가로채지 않는다.
라인 pull/virtualElapsedMs - 가상 시계가 connectTimeoutMs를 넘었는데 deadlineCleared가 false면 abort한다. 헤더 clear를 빼거나 first-byte로 옮기면 "a" enqueue 전에 죽어 body/completed 단언이 실패한다(본문 ablation과 일치).
라인 expect(deadlineClears).toBeGreaterThan(0) - 프로덕션은 성공 경로(573)와 finally(609)에서 두 번 clear할 수 있어 > 0이 맞다. === 1로 조이면 깨질 수 있다.
경로 tests/web-search/web-search.test.ts / src/web-search/loop.ts - 프로덕션 변경 없음. 새 패턴 도입 없이 기존 clearableDeadline 심만 테스트에서 운전한다.
경로 performance.now 하한 제거 - 예전 >= 50은 "충분히 오래 걸렸다"만 보고, 헤더 clear 의미는 못 봤다. 가상 시계 + clear 횟수가 계약을 더 직접 본다.

메인테이너의 판단이 필요한 지점

  • macOS 2/2 exact-head가 초록이 된 뒤에만 머지할지(플레익 재현 레그), Linux 통과만으로 충분한지
  • 같은 describe에 남은 벽시계 sleep이 더 있는지 한 번 훑을지, 이번 한 케이스만으로 끝낼지
  • mock이 타이머를 안 돌리고 flag만 쓰는데, 이 추상화가 프로덕션 clear 시점과 계속 맞는지(지금 loop.ts 573과 맞음)

너의 추천

이 댓글은 grok-bot이 작성했습니다

The rewrite took this file to 2857 lines against a 2823 baseline, so the
repository file-size ratchet failed and took test 1/4 and macos 2/2 with it.

Compacted to exactly 2823. Every assertion, the virtual deadline semantics, the
ablation and the 1000ms ceiling are unchanged; only the expression is tighter.

The baseline itself is untouched. Editing the cap to fit a change is the same move
as widening a timeout to fit a slow test.

No local suite, focused test, typecheck, build, or install was run.
@lidge-jun
lidge-jun merged commit 7868f5d into dev Sep 17, 2026
29 checks passed
@lidge-jun
lidge-jun deleted the codex/2580-websearch-virtual-clock branch September 17, 2026 06:49
agentHits pushed a commit to agentHits/opencodex that referenced this pull request Sep 17, 2026
…e-jun#4867)

* test(web-search): drive the connect deadline on a virtual clock

`fast headers plus raw byte progress can outlive connectTimeoutMs` timed out at
1002.37ms against its own 1000ms ceiling on macOS 2/2 of run 35186268515. Its
siblings in the same describe run in 2.58, 3.48, 2.57, 3.06 and 14.65ms, so this
case was three orders of magnitude slower than everything around it and sitting
2.37ms outside a bound it could not reliably clear.

It was spending that second on five real 12ms waits used to walk a real deadline
forward. Real sleeps and a real ceiling race the same scheduler, so runner load
decides the outcome - which is how a case about timeout SEMANTICS became a case
about how busy the machine was.

It now drives the existing clearableDeadline seam with a virtual elapsed counter.
The first byte lands at a virtual 26ms against a 25ms deadline, which is the exact
condition the case exists to describe: headers that arrive fast must clear the
deadline before the body is consumed, and continuing byte progress must not re-arm
it. Body, cancellation, status and completion assertions are unchanged.

The 1000ms ceiling stays exactly where it was. Nothing was widened; the wall-clock
wait was removed instead.

Ablation: leave the deadline armed, or move its clear to first-byte progress, and
the abort lands before "a" is enqueued, so the body and completion assertions fail.

No local suite, focused test, typecheck, build, or install was run.

* test(web-search): fit the virtual-clock rewrite inside the ratchet cap

The rewrite took this file to 2857 lines against a 2823 baseline, so the
repository file-size ratchet failed and took test 1/4 and macos 2/2 with it.

Compacted to exactly 2823. Every assertion, the virtual deadline semantics, the
ablation and the 1000ms ceiling are unchanged; only the expression is tighter.

The baseline itself is untouched. Editing the cap to fit a change is the same move
as widening a timeout to fit a slow test.

No local suite, focused test, typecheck, build, or install was run.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Maintenance, CI, tests, refactors, or build changes (not a user-facing bug or feature).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant