Skip to content

test(run): fix the order of remote requests in unknown_after_remote_hit - #986

Merged
wan9chi merged 1 commit into
claude/e2e-proxy-early-answerfrom
claude/e2e-remote-unknown-order
Nov 9, 2026
Merged

wan9chi merged 1 commit into
claude/e2e-proxy-early-answerfrom
claude/e2e-remote-unknown-order

Conversation

@wan9chi

@wan9chi wan9chi commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Motivation

dry_run_json::unknown_after_remote_hit, added in #968, is flaky. It failed on main's macOS CI (job), on #969's, and on #984's Windows job (run). In its first step, the backend log read fetch 404, store 200, fetch 404, store 200 instead of fetch 404, fetch 404, store 200, store 200.

Changes

  • Cause: vt run after-emit runs emit first. Once emit finishes, its upload starts in the background at the same time as after-emit's fetch, and the e2e proxy logs each request when its response finishes, so either can come first. VP_RUN_INTERNAL_HIDE_PENDING_UPLOADS hides the upload's progress, but doesn't delay it.
  • Fix: each dependency now runs in its own step first: vt run emit before vt run after-emit, and vt run build before vt run after-build. Each step then makes at most one fetch followed by one upload, in a fixed order.
  • The dry runs that the case checks are unchanged: after-emit is still unknown after a remote hit that restores outputs, and after-build is still a hit.

🤖 Generated with Claude Code


Note

Low Risk
Test fixture and snapshot updates only; no production run or remote-cache logic changes.

Overview
Fixes flaky unknown_after_remote_hit dry-run e2e snapshots by splitting combined runs into separate steps: vt run emit before vt run after-emit, and vt run build before vt run after-build.

That keeps each remote-cache interaction to a deterministic fetch-then-store order (uploads from the first task no longer overlap the dependent task’s fetch when the proxy logs responses). Comments and the unknown_after_remote_hit.md snapshot were updated for the extra steps, cache-hit replay lines, and corrected expectations (e.g. after-build as full cache hit with no fetch).

snapshots.toml and the markdown snapshot are the only touched files; asserted dry-run behavior is unchanged.

Reviewed by Cursor Bugbot for commit 9226b86. Bugbot is set up for automated code reviews on this repo. Configure here.

`dry_run_json::unknown_after_remote_hit` failed on main's macOS CI and
on #984's Windows job. Its first step ran `vt run after-emit`, which
runs `emit` first. Once `emit` finishes, its upload starts in the
background at the same time as `after-emit`'s fetch, and the e2e proxy
logs each request when its response finishes. So the backend log read
`fetch, store, fetch, store` as often as the expected
`fetch, fetch, store, store`. `VP_RUN_INTERNAL_HIDE_PENDING_UPLOADS`
hides the upload's progress, but doesn't delay it.

Each dependency now runs in its own step first: `vt run emit`, then
`vt run after-emit`, and `vt run build` before `vt run after-build`.
Each step then makes at most one fetch followed by one upload. The dry
runs that the case checks are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wan9chi
wan9chi added this pull request to stack #987 November 9, 2026 12:04
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  +0.15%  [ -2.83% ..  +3.13%]  overhead   +53.90%
dynamic/access             change  +0.42%  [ -3.68% ..  +4.21%]  overhead   +22.51%
dynamic/access-relative    change  +0.30%  [ -2.60% ..  +2.76%]  overhead   +70.51%
dynamic/access-contended   change  -1.45%  [ -8.12% ..  +6.07%]  overhead   +34.60%
static/launch              change  +0.47%  [ -2.98% ..  +4.05%]  overhead  +199.48%
static/access              change  -0.23%  [ -1.81% ..  +1.29%]  overhead +1472.38%
static/access-relative     change  -0.57%  [ -2.57% ..  +1.70%]  overhead +1931.60%
static/access-contended    change  +0.09%  [ -1.90% ..  +2.72%]  overhead +2019.58%

macos

dynamic/launch             change  -0.24%  [ -2.79% ..  +2.44%]  overhead  +223.06%
dynamic/access             change  +0.26%  [ -3.64% ..  +3.34%]  overhead    +3.62%
dynamic/access-relative    change  -0.31%  [ -2.36% ..  +1.37%]  overhead  +253.60%
dynamic/access-contended   change  +1.15%  [ -2.85% ..  +3.66%]  overhead    +1.78%

windows

dynamic/launch             change  +0.12%  [ -3.04% ..  +4.50%]  overhead   +27.27%
dynamic/access             change  -0.36%  [ -1.65% ..  +1.30%]  overhead    +0.93%
dynamic/access-relative    change  +0.18%  [ -1.09% ..  +1.47%]  overhead    +1.32%
dynamic/access-contended   change  +0.00%  [ -1.48% ..  +1.59%]  overhead    +1.09%

@wan9chi
wan9chi merged commit f9e9caa into main Nov 9, 2026
23 checks passed
@wan9chi
wan9chi deleted the claude/e2e-remote-unknown-order branch November 9, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant