docs: correct the handler-cost figures — the 68ms warning is cold-start - #57
Merged
Merged
Conversation
Re-validated the three claims behind "the per-request registration cost stays", because
one number in them was wrong.
Wrong: I cited Skybridge's boot warning ("The Skybridge handler took 68ms — it runs on
every request") as if 68ms were the recurring per-request cost in the container. It is
not. That line prints exactly once, from the cold first build inside `ready()`, so it
carries the JIT warm-up. Measured steady state over HTTP against the same image:
/status 2.1ms
initialize 29.5ms <- builds all 53 tool schemas, uses none of them
tools/list 41.3ms
So ~27ms, matching the host measurement of ~23ms rather than the 68ms the warning
suggests. The finding is unchanged in kind and about 2.5x smaller in size than that line
reads. Worth writing down because the warning's own wording invites the misreading.
The other two claims held up under a sharper check:
- zod does not memoize. One hoisted `z.object` converted 300 times: median 0.84ms every
time, and `input(opts) === input(opts)` is false — a fresh object per call. A
per-registration `z.object(shape)` costs 1.03ms. Hoisting buys 18%, where a memo would
buy ~100%. The earlier check was three single samples with visible noise; this one is
300 iterations at the SDK's actual conversion target ("draft-2020-12").
- a pre-built schema is a compile error, verified with tsc rather than by reading the
types: "Index signature for type 'string' is missing in type 'ZodObject<...>'", and
the handler's arguments degrade to `unknown`.
Also confirmed skybridge 2.0.0 is still the newest release, so none of this is already
fixed upstream.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-validation of the three claims behind "the per-request registration cost stays" (finding #1 in #56). One of the numbers was wrong.
The wrong number
I cited Skybridge's boot warning — "The Skybridge handler took 68ms — it runs on every request" — as if 68ms were the recurring per-request cost in the container. It is not. That line prints exactly once, from the cold first build inside
ready(), so it carries the JIT warm-up.grep -c "handler took"on the container log after 80 requests:1.Steady state over HTTP against the same image:
/statusinitializetools/listSo ~27 ms, matching the host measurement of ~23 ms rather than the 68 ms the warning suggests. The finding is unchanged in kind and about 2.5x smaller in size. Worth recording because the warning's own wording ("it runs on every request, so this cost is paid per request") is what invites the misreading — the sentence is true, the sample attached to it is not the steady-state one.
The container number also turned out to be ~the host number, so there is no hidden Cloud Run multiplier to worry about.
The two claims that held
zod does not memoize. The earlier check was three single samples with visible noise (9.5 / 2.6 / 5.6 ms — the third slower than the second, which should have made me look twice). Redone with 300 iterations at the SDK's actual conversion target (
draft-2020-12, read out ofJSON_SCHEMA_CONVERSION_TARGET):Hoisting a
z.objectbuys 18%, where a memo would buy ~100%.A pre-built schema does not typecheck. Verified with tsc on a two-tool probe rather than by reading the type declarations:
The second error is the important one: forcing it through would type every handler argument
unknownacross all 53 tools.Also confirmed
skybridge@2.0.0is still the newest published release, so none of this is already fixed upstream.Scope
Comments and CHANGELOG only. No code, no behaviour change. 454 tests, typecheck clean.