Repository navigation
Conversation
…ared harness and nghttp2-internal tests
…not wired in yet): the reader feeds Conn.recv, runs whole no_wait requests and writes one batch; other streams on their own task with the nghttp2 engine's request build; StreamSink on Conn's submit functions (complete, head, a streamed body in two 16 KiB blocks waiting on its stream's room, interim, end); output under a flush token with the rest to an on-demand flusher; ConnLock no lock on a pinned zio executor; Conn.raiseWindow for stream_window_reading; placement.single_executor_guarantee
…and .table), SAFETY reasons on its casts and undefined fields
…re the submit), a connection error cancels every handler and frees their room, the body wake always notifies, the writer never keeps a borrowed block, output rounds go into the buffered writer with one flush (and one timer per write buffer), the next response block taken only when more bytes come, an empty end rides on the queued DATA
…ough complete then the flusher, attach into the connection's group, abort on a reset or the teardown), parked ones on parked.zig's Notice with one parked_open count the teardown waits for, RFC 8441 tunnels (tunnel_window raised at the extended CONNECT, appendParts/kickOut copied into a chain of 16 KiB blocks cut at each kick and let go once written, a send from another executor hopping to the connection's, RST_STREAM(NO_ERROR) once a finished tunnel's END_STREAM is framed), test_sink_bodies for gRPC; output from outside the connection's tasks only goes to the flusher
…gine (h2/test_server.zig); WebSocket over HTTP/2 echo and a cross-executor send, and gRPC unary (no streaming body writer) and streams, on it
…p to 16 KiB, doubling) and all let go once written, a queued cut grown by the next sends (Conn.growData: one DATA run per block, no cut limit), the send cap in bytes, a block's mark cleared at each new cut; a parked release kicks the flusher before its count drops; INTENTIONAL.md: the hop per foreign send and the detached abort after a clean EOF
…r wake, release): idle connections park with their read buffer, output blocks, stream slab, table and spare streams given back (D16, #97 for idle connections; a partial frame stays on its task, nginx's rule); keep-alive and timeouts.write ("no progress", per write buffer's worth) on the run's registered Timer, kept to the shelf's sweep, the flusher with a Timer of its own, no timer per flush; finished responses the peer gives no credit for reset at their send deadline by the flusher (a futex wait, so std.Io.Threaded too); the two-phase GOAWAY drain with its busy hold (#135), reaching readers waiting for input through a per-shelf drain bell (park.Shelf.drainWake, Lazy.wake_on_drain) or under the lock where the connection has a mutex, and the close once the final GOAWAY is out and nothing is open; a keep-alive close sends GOAWAY(NO_ERROR); Conn.atFrameBoundary, releaseIdle and initAt (no 7 KB copy through the task's stack); a test-only engine choice per server (test_h2_engine, h2_conns_anole) for the switch to flip; h2_park_test.zig on Server.run ReleaseFast frames (sub rsp + pushes): start 632 (14,456 before initAt), adopterServe 128, readerLoop 1,536, serveStream 2,176, writeOut 1,392, flusherRun 304, serveAdopted 336, handleConnectionWrapper 656 (#223's), handleRequests 4,672; nghttp2 serve 1,120. ALPN h2 to a handler: 656+64+336+128+1,536+2,176 = 4,896 B, under h1's keep-alive path.
…ide (close_notify, then shutdown(.send)) and wakes its reader for the lingering close, never our receive side (a reset for the peer's late frames); no flusher joins the group once it is joined for the last time (spawned under the lock, the teardown leaves the registry and closes the group before its cancel, closeRun closes it and joins again); the registry lock an IoLock (the drain's walk waits for each connection's mutex, an L2 panic before); the keep-alive from when the connection went idle, not from each empty wake; no park-delay wakes while streams are open, the last stream's end wakes the reader instead; no send-deadline waiter for responses whose END_STREAM is out
…zig): server.zig imports it as `h2` and its Registry is `h2_conns`; the nghttp2 engine stays for tests only until M6 (`test_h2_engine = .nghttp2`, its Registry `h2_conns_nghttp2`, `dusty.h2_nghttp2_for_tests`), and `h2_engine_for_tests` is the new engine; the preface sniff moves to the new engine. Its tests pinned to nghttp2 are those of that engine's hooks and pools (INTENTIONAL.md, the server engine 6 and 7); the wake test, the task-cap test and the half-close test run on both. Engine fixes the suite found: a handler's task is spawned after `recv`, not inside it, and a spawn under the lock that is no lock lets it go (zio's spawn back-pressure yields on a pinned executor with a long run queue: another task of the connection ran in the middle of `recv`; the lock is counted with zio's held_locks in Debug); after a clean EOF with handlers running, a PING now and every 500 ms tells a half-close from a peer that is gone, whose handlers are canceled (the nghttp2 engine's probes, which the C ABI's reset tests rely on); the group is closed before its one join, so no spawn from outside it races the join's end (std.Io.Threaded's Group.await assert); a GOAWAY's unknown code is logged as a number (the fuzz seeds panicked); a file body gets the sink's first block before sendFileAll, which asserts a buffer (respond_fd over HTTP/2). New tests: the default engine, a drain reaching both engines, a file body, a burst of 200 tasked requests in one read on zio, the half-close probe. h2spec strict 441/441 (h2c shared, h2c-only, TLS) with known-failures.txt empty
…r engine: h2/root.zig, src/h2/root.zig, inmem.zig, parked.zig, h3/server.zig, h2/test_server.zig, websocket/h2_test.zig, known-failures.txt's header, and -Dhttp2's description (the server engine in Zig, the client via nghttp2)
…ps no buffer (an extended CONNECT's or parked stream's chain) goes through a 16 KiB buffer of the request's, where rebase refuses (it asserted, or spun in ReleaseFast); a connection's run holds its flusher's task slot (as the nghttp2 engine's writer did), so max_tasks bounds the tasks again, and a send-deadline waiter takes its own; the half-close probe sends its next PING only after a whole interval without a handler's end, as the nghttp2 engine; closeRun's comment no longer claims a detached attach can't race the join under a thread pool
…pc-go, Chromium, Firefox and Envoy complete, PRIORITY_UPDATE unseen from the browsers, every budget keeps 2.5x
…ot an error (grpc-go's client closes every connection with GOAWAY(NO_ERROR) naming its own highest stream), INTENTIONAL.md session 6
…codes and the server's SETTINGS in the report, a /done result channel for the browser pages, a Go net/http client, direct-TLS and raw PRIORITY groups, PASS/FAIL lines and a headroom table in summarize.py
… checks in the Node, curl and browser scenarios, client GOAWAY codes in summarize.py, README claims reworded
…'s additions ported onto it (the new engine as the harness's first engine, the nghttp2 one as its second)
…k sends them grpc's tools/http2_interop TestSoonUnknownFrameType failed on h2c and TLS (PR #226 CI). It sends every frame type from 11 to 255, empty, back to back after the preface, then a PING. Two things ended the connection: - Type 0x10 is RFC 9218's PRIORITY_UPDATE, which the engine validates: empty, it is a FRAME_SIZE_ERROR (RFC 9113 4.2), as in Go's x/net (parsePriorityUpdateFrame) and nghttp2 with the extension on. The tool predates RFC 9218; run-http2.sh now has its loop skip 0x10 as it skips ALTSVC, and fails if the loop changes. - The other 244 ran past the empty-frame budget (D15 point 2 counts unknown frames there), whose burst was its rate, 100. Limits gets empty_frame_burst, 1000 (nghttp2 takes these in a glitch burst of 10000); the rate stays 100/s. Test: h2 limits: every unknown frame type at once after the preface is ignored, and a PING answered.
The new engine only logged a peer's GOAWAY and went on: the connection stayed until its keep-alive or the park (about 60 s in h2/client_test.zig), and then closed without the lingering close. nghttp2's want_read is 0 once a GOAWAY came and no stream is active, and the nghttp2 engine then shut its side (FIN) and lingered. PR #226's macOS job failed 'a peer's GOAWAY ends the connection with a FIN, and what it sends after draws no reset' with ECONNRESET: by the late FIN the server had closed, so the test's PING drew a reset. On Linux the same test passed only because a read after the FIN keeps returning EOF even once a reset arrives (tcp_recvmsg checks SOCK_DONE before sk_err); it took 60 s there too. drained() now holds for the peer's GOAWAY as for our final one: the writer shuts the send side once nothing is open (shutDrained) and the reader ends with the lingering close. Test: h2 engine: after the peer's GOAWAY the open stream is answered, then the connection ends with a FIN (failed first: timed out waiting for the shut).
PR #226's ubuntu core-test: 'a connection error mid-stream lets the streaming handler go and the connection close' timed out waiting for the handler's failed write. Between the read that ended in the connection error and closeRun's cancelStreams, the reader flushes the GOAWAY. A write completing meanwhile (the flusher's wakeRooms) woke the streaming handler; roomFree is true once the connection is dead, and nextBlock checks only reset/send_shut, so the handler was handed a block that out still borrowed (its bytes could be overwritten while queued for the write), went on submitting DATA that would never be framed, and ended its 1 MB response without an error. On the box the reader always got to closeRun first. The reader now calls cancelStreams(true) under the lock in which recv failed, so every send path sees its stream reset from that moment. Test: h2 engine: a stream woken between a connection error and the close fails its write, never reusing a block still borrowed (a hook, test_on_conn_error, does that write's wake-up; failed first: the handler returned without failing).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is M3 of the HTTP/2 takeover (research/h2h3-takeover/PLAN.md). The server now serves HTTP/2 on Anole's own engine,
src/http/core/h2/server.zig, built on M1's codecs and M2's sans-IO protocol core. nghttp2 stays in test builds only, for the differential and nghttp2-specific tests, until M6. There is no engine option (decision Q3), so merging this switches HTTP/2 to the new engine.What's in it
Conn.recv. Whole noWait requests run inline and the rest run on their own task. Output goes under a flush token, with an on-demand flusher. Each stream has its own room for back-pressure, so there's no connection-wide broadcast. On pinned zio there's no connection lock.Verification
test,core-test,capi-test,test-flags,test -Dhttp2=false,lint, ReleaseSafetestandtest-stress.M4 gate
The gate is "measurably better" (PLAN §8.3): at least 3% fewer instructions:u per request on h2c and unary gRPC, and an A/B no worse on req/s, p99 and RSS. Measured against v0.5.4:
In the h2c and gRPC cells the load generator is the bottleneck, so req/s and p99 there only show that nothing got worse. CPU per request is the real signal.
TLS needs about 10× the per-record work because each frame piece is its own TLS record: the flusher's writeVecAll goes through dusty_tls's drain one slice at a time. M4's record-aligned TLS framing item fixes that.
Known follow-ups (M4 and issues)
hpack.Encoder.putStringis the top function the engine adds.Closes #97 for HTTP/2 connections.