probes: close the gaps #40 documents — drift guard, realtime flake, mock URL - #45
Merged
Merged
Conversation
…ock URL The handoff issue (#40) records facts that cost sessions real time. Three of them were fixable in code rather than prose, so they are fixed here. Guard the duplicated prompt modules. Cloud Functions are zipped per-directory, so each function carries its own copy of the shared modules; the duplication is deliberate but the drift is not, and only crisis.js and visualization.js were ever checked. The check in crisis-probe.mjs is now a table covering all seven shared modules and all twelve copies, plus an assertion that every module in functions/shared/ appears in that table — which immediately turned up coach-personas.js and reference-photo.js as unguarded. Copies may carry the "shared/ is the source of truth" banner and nothing else. Fix the viz-realtime-probe flake. Realtime replays the WAL in order, so running that suite straight after the write-heavy ones could leave its event still in the queue when the fixed 4s wait expired — reported as four realtime failures that looked like a regression and were not. The assertions now wait for delivery, and the no-leak assertion is pinned to a positive delivery on a second subscription: a bare sleep passes just as happily against a realtime server that has stopped delivering anything at all. Accept both meanings of MOCK_OPENAI_URL. The gateway wants the base /v1 URL because the SDK appends the route; sms-image-probe.mjs calls the endpoint directly and wants /v1/chat/completions. Passing the gateway's form returned 404 and surfaced as twenty unrelated-looking assertion failures. Either form now works. Also: delete the never-deployed saveSelectedAvatar handler, which wrote a caller-supplied avatar URL onto any coachId with no token and no ownership check. Terraform never gave it an entry_point, so it was unreachable; it was only waiting for someone to wire it up and ship an IDOR. The README said "five suites" while listing seven and omitted creator-probe from the run list. Verified on a local stack: 378 checks green across all seven suites (rls 77, club 62, flow 46, viz 35, sms 55, deletion 64, creator 39), crisis-probe 143, tsc clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Harrolee
added a commit
to cekuu35/cabo
that referenced
this pull request
Aug 20, 2026
Resolves the conflict with Harrolee#45 and fixes three things in the contributed change. Rate-limit key. The limiter read `req.ip` first and fell back to `X-Forwarded-For`. Express only populates `req.ip` from that header when `trust proxy` is set, which the functions framework does not do here, so `req.ip` is Cloud Run's front end — the same value for every caller. The per-IP ceiling would have been a global one, and the pre-signup avatar step would have started 429ing everybody after a handful of builds, which is the exact funnel this endpoint exists to serve. Now keyed on the left-most X-Forwarded-For entry. saveSelectedAvatar stays deleted. Harrolee#46 restored it with resolveCaller() and isCoachOwner() in front, which does fix the IDOR, but Terraform gives this directory one entry_point and it is not that one — so it is unreachable code on a write path, one Terraform line away from being live. Both helpers remain, so bringing saving back is a small change; do it with an entry_point and a caller in the same PR. AVATAR_STYLES is required at the top rather than lazily inside the 400 branch. Adds mobile/e2e/avatar-auth-probe.mjs — 25 checks over the authorization matrix, which Harrolee#44 asked for and the PR did not have. It loads the real index.js with ./avatar-generation replaced by a recording stub, the technique crisis-probe.mjs uses on openai, so it spends no Replicate credits and can assert the part that matters: a rejected call reaches generation zero times. A 403 that still burned a credit looks identical from the outside. Supabase is deliberately not stubbed — token verification and the ownership lookup run against the real stack. Verified the probe can fail: disabling isCoachOwner() and the temp- prefix check turns 25/0 into 18/7, including an anonymous caller generating against a real coach id. Co-Authored-By: cekuu35 <cekuu35@users.noreply.github.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Picks the fixable items out of the #40 handoff issue. Three of the facts that issue records were fixable in code rather than prose.
Guard the duplicated prompt modules
Cloud Functions are zipped per-directory, so
require('../shared/...')does not survive deploy and each function carries its own copy. The duplication is deliberate; the drift is not, and it has bitten before — acoach.taglinefix once landed infunctions/shared/and never reached the deployed copy, which means a prompt change that silently does not apply in production looks exactly like a prompt change that did not work.Only
crisis.jsandvisualization.jswere ever checked. The section incrisis-probe.mjsis now a table covering all seven shared modules and all twelve copies, plus an assertion that every module infunctions/shared/appears in that table. That last check earned its keep immediately: it turned upcoach-personas.jsandreference-photo.jsas unguarded.Copies of
coach-personas.jscarry a "shared/ is the source of truth" banner and are otherwise byte-identical, so one leading block comment is stripped before comparing — the banner stays allowed, everything after it still has to match exactly.Fix the viz-realtime-probe flake
Running the full suite in sequence produced
29 passed, 4 failedinviz-realtime-probe.mjs, all four in realtime delivery. It is not a regression. Realtime replays the WAL in order, and after the four write-heavy suites it was still draining their backlog when the probe's fixed 4-second wait expired. In isolation the same code gave 33/0.The assertions now wait for delivery instead of sleeping. The no-leak assertion was the more interesting half: it asserted a negative against a bare sleep, which would pass just as happily against a realtime server that had stopped delivering anything at all. It is now pinned to a positive — the other member subscribes to their own thread, and we wait for the message to arrive there before asserting it did not arrive here. Once it has been delivered elsewhere, realtime has demonstrably processed that insert, so silence on our channel means filtered rather than in flight.
Verified by reproducing the original conditions: the four write-heavy suites back to back, then
viz-realtime-probe.mjsimmediately after. Was 29/4, now 35/0.Accept both meanings of MOCK_OPENAI_URL
harness/function-gateway.jswants the OpenAI base URL (.../v1) because the SDK appends the route itself.sms-image-probe.mjscalls the endpoint directly and wants.../v1/chat/completions. Same variable, two meanings.Passing the gateway's form to the probe returns 404 on every model call, which surfaces as ~20 unrelated-looking assertion failures — "a scene was sent", "exactly one Replicate call — 0", "the image is about drums" — rather than as a wiring mistake. I lost a cycle to exactly this. Either form now works.
Also
saveSelectedAvatar. It took acoachIdand an avatar URL from the request body and wrote them straight onto that row incoach_profiles, with no token check and no ownership check. Terraform never gave it anentry_point, so it was never deployed and never reachable — it was only waiting for someone to wire it up and ship an IDOR. A comment in its place explains what is missing if saving comes back.creator-probe.mjsfrom the run list. Documents theMOCK_OPENAI_URLambiguity and the realtime ordering constraint.The live exposure this does not fix —
coach-avatar-generatoritself is public, unauthenticated and burns Replicate credits — is #44. It needs a product decision rather than a patch, because/coach-builder/*is deliberately unauthenticated and requiring a session there would kill the top of the signup funnel.Verification
Local stack, all seven suites plus the model-free probe:
rls-probeclub-probeflow-probeviz-realtime-probesms-image-probeaccount-deletion-probecreator-probeprompt-eval/crisis-probe378 checks across the seven, and
npx tsc --noEmitclean frommobile/.sms-image-probeverified green under bothMOCK_OPENAI_URLforms.Does not close #40. That issue is a standing read-me-first for anyone picking up work here,
not a work item — closing it would hide it. Its body has been refreshed to match what this PR
changes, and every number in it re-measured on merged
main.🤖 Generated with Claude Code