Skip to content

A full per-connection receive queue ends the connection instead of pausing the recv #3

Description

@MDA2AV

What happens

A connection whose handler is parked on something other than a read - a send, ioxd_delay, a socket client call - while its peer keeps sending has its multishot recv filling the per-connection receive queue (struct spsc, RX_QUEUE = 64 delivered buffers). When the queue is full, ioxd__conn_on_recv returns the buffer, ends the input with -ENOBUFS and cancels the recv (lib/io/conn.c, the ioxd__spsc_full branch). The handler's next read fails and the connection is closed: the peer sees a reset, or a reply cut short.

Where it was seen

The HttpArena entry's streamed echo (POST /echo: read 8 KB into the reply slab, send it, read the next 8 KB) on the arena's CI runner. A 100 KB body over TLS arrived there as small completions - well under 2 KB each - so more than 64 buffers queued while the handler waited on its first send, and the client got 0 bytes back (HttpArena PR #1473, run 34418961856). On this machine the same body arrives in 2 KB completions, 50 buffers, and never reaches the mark, so make check and the arena validator both pass.

Reproduced locally with the tiny build's queue (-DRX_QUEUE=4, as make check-tiny builds it): a 100 KB echo over TLS or plain dies after 16 KB, curl exit 18/56.

What it means

The queue is meant to bound how much one connection can hold of the shared buffer group (ioxd__conn_on_recv notes: "rather than let one peer hoard the buffer group, end its input"), but a slow consumer with a fast producer is an ordinary situation - any handler that sends while a large body is still arriving - and it ends in a dropped connection rather than in flow control.

One possible shape

Pause the recv at the mark the way a prologue pauses it (pausing, landing in RECV_PAUSED), let the socket's own window hold the peer, and re-arm from the reader as the queue drains; keep whatever the kernel had already posted before the cancel landed (the ring would have to grow for that burst, since the kernel can post the rest of a CQ batch after the decision). Commit 85b61bd was an implementation along those lines; it was reverted because it was not asked for, and is only a reference.

Also observed, not explained

On the same CI runner, with that implementation pinned, the Content-Length 100 KB echo passed and the chunked 100 KB echo still came back with 0 bytes (run 34420184914), in under a second, so not a timeout. Locally it passes with every chunk size and split pattern tried. Possibly a different problem in the chunked body path, possibly not; it has not been diagnosed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions