Skip to content

Sockets: bound the sys-botbase receive buffer; add non-blocking ClientSocket::abort() - #1853

Merged
Mysticial merged 5 commits into
mainfrom
sbb3-unbounded-buffer
Nov 7, 2026
Merged

Mysticial merged 5 commits into
mainfrom
sbb3-unbounded-buffer

Conversation

@pa-devbot

Copy link
Copy Markdown
Collaborator

Fixes unbounded sys-botbase message buffering (security audit, High).

The bug

TcpSysbotBase_Connection::on_receive_data() appended every non-CR/non-LF byte to m_receive_buffer (std::deque<char>) with no line-length limit, total cap, or discard state. A configured server (or an active attacker on the unauthenticated plaintext TCP connection) could stream bytes without \n forever, growing process memory until allocation failure. The surrounding catch (...) swallowed std::bad_alloc without clearing the buffer or disconnecting, so the peer kept applying pressure. No version exchange needed.

Fix

  • Bound the line at SBB_MAX_RECEIVE_LINE_BYTES (64 KiB — generous for the protocol). The accumulation is extracted into a testable static helper, TcpSysbotBase_Connection::accumulate_receive_line(), which stops consuming, clears the partial line, and flags overflow once a line would exceed the cap.
  • Abort on overflow: on_receive_data() logs a protocol error, updates the status line, clears the buffer, and aborts the connection.

The hard part: aborting from a receive callback

ClientSocket::close() blocks until the receive thread exits and is documented as unsafe from a listener callback — but on_receive_data() runs on that thread, so calling close() there would deadlock. Added a non-blocking ClientSocket::abort() to every backend:

  • Marks the socket DESTRUCTING and releases the underlying socket, then returns without waiting.
  • The receive thread observes DESTRUCTING on its next iteration and exits on its own; a later close() completes the teardown to NOT_RUNNING. No on_disconnect fires, matching close().
  • POSIX/WinSocket: set DESTRUCTING + close_socket_locked() under the lock, notify. Qt: set DESTRUCTING, cancel in-flight sends, and dispatch the blocking socket removal to a worker thread (it then waits for the event thread, not itself).

The socket is deliberately left in DESTRUCTING (not NOT_RUNNING) so a concurrent connect() can't race the still-exiting receive thread; the SysbotBase connection object is recreated on reconnect.

Tests

  • 8 ReceiveLine cases pin the bound: normal, multi-line, partial, exactly-max, max+1, overflow with/without a trailing newline, and CR handling.
  • abort() from the receive callback (new socket test, runs for Qt/POSIX/WinSocket): a client whose on_receive_data() calls abort(); verifies it returns without deadlocking and the socket is no longer CONNECTED.

Full suite: 20,365 passed / 0 failed / 3 skipped (Qt + POSIX here; WinSocket on Windows CI).

…tSocket::abort()

A configured sys-botbase server (or an active attacker on the
unauthenticated plaintext connection) could stream bytes without a '\n'
forever. on_receive_data() appended every byte to m_receive_buffer, a
std::deque<char>, with no line-length limit, so process memory grew until
allocation failure. The surrounding catch swallowed the std::bad_alloc
without clearing the buffer or disconnecting, so the peer kept applying
pressure.

Fix:
- Cap a single line at SBB_MAX_RECEIVE_LINE_BYTES (64 KiB, generous for the
  protocol). The accumulation is extracted into the testable static
  TcpSysbotBase_Connection::accumulate_receive_line(), which stops
  consuming, clears the partial line, and flags overflow once a line would
  exceed the cap.
- On overflow, on_receive_data() logs a protocol error, updates the status
  line, clears the buffer, and aborts the connection.

Aborting from a receive callback is the hard part: ClientSocket::close()
blocks until the receive thread exits and is documented as unsafe from a
listener callback (which runs on that thread - calling it would deadlock).
Add a non-blocking ClientSocket::abort() to every backend: it marks the
socket DESTRUCTING and releases the underlying socket, then returns without
waiting. The receive thread observes this and exits on its own; a later
close() completes the teardown to NOT_RUNNING. No on_disconnect is fired,
matching close().

Tests:
- 8 ReceiveLine cases pin the bound (normal, multi-line, partial, exactly
  max, max+1, overflow with/without a trailing newline, CR handling).
- A new "abort() from the receive callback" socket test connects a client
  whose on_receive_data() calls abort(), and verifies it returns without
  deadlocking and the socket is no longer CONNECTED. Runs for Qt, POSIX,
  and WinSocket.

Full suite: 20,365 passed / 0 failed / 3 skipped.
accumulate_receive_line() only needs to be visible to on_receive_data() and its unit tests (which live in the same translation unit), so it is no longer part of TcpSysbotBase_Connection's public API. It moves to an anonymous-namespace free function in SysbotBase_Connection.cpp; the now-unneeded <vector> include and the public declaration leave the header.
The first implementation deferred pool removal of the socket to a worker thread, which could crash the Qt event loop under load (an event handler throwing std::system_error). Instead mark the socket DESTRUCTING, cancel in-flight sends, and call the socket's own non-blocking abort(); the resulting disconnect/error signals see DESTRUCTING and are no-ops, and a later close() removes the socket and returns it to NOT_RUNNING.
…se()

Per review, the two client-socket teardown methods are one primitive (stop, optionally wait for the receive thread), so the naming now matches the call sites:

- close() is the non-blocking stop (the former abort()): it marks the socket DESTRUCTING and releases the underlying socket; safe to call from a listener callback. It no longer joins the thread or returns NOT_RUNNING.
- close_and_wait() is the blocking teardown (the former close()): joins the receive thread and returns NOT_RUNNING. It is now private, driven by the destructor and by connect().

connect() now calls close_and_wait() up front: since close() leaves the socket DESTRUCTING, reconnecting on the same socket joins the old thread there (connect is always user-initiated, never a callback, so blocking is safe). This keeps same-object reconnect working.

Qt: close() posts QTcpSocket::abort() to the socket's event thread (QMetaObject::invokeMethod, queued) instead of calling it cross-thread, which tripped Qt's QSocketNotifier thread check and crashed the event loop when the tests call close() off the event thread.

Tests: the "close() from the receive callback" test (formerly "abort() ...") drives the public close(); the send-timeout test asserts DESTRUCTING after the (now non-blocking) close() instead of NOT_RUNNING.
The Mac CI crash on the rename commit (std::system_error "mutex lock
failed: Invalid argument" thrown from a socket listener callback, on
both Mac architectures) was a use-after-free of the test listeners:
with close() now non-blocking, the socket keeps delivering callbacks
until the socket's destructor runs, but every socket test declared its
listeners after the sockets, so at scope exit each listener was
destroyed first - with the socket's thread still able to call back
into it (pthread_mutex_lock on the freed Waiter's mutex returns EINVAL
on macOS, which Qt turns into std::terminate).

Declare every listener before the socket that adds it in all 15 socket
tests, so listeners are destroyed after their sockets, and document
the lifetime requirement on the Listener structs.
@Mysticial
Mysticial merged commit e17992b into main Nov 7, 2026
8 checks passed
@Mysticial
Mysticial deleted the sbb3-unbounded-buffer branch November 7, 2026 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants