Conversation
a bounded queue of transferable values shared between threads by identity: a process-wide registry maps ids to channels, every vm that holds one binds its own wrapper, and a wrapper crosses to a worker as its id through native __serialize/__unserialize. send and recv block with optional timeouts, trySend refuses instead of waiting, close drains then ends every foreach, and channels ride inside task arguments, results, and other channels. everything that crosses a thread is now a payload: the bytes plus a retained reference to each channel inside, so a channel created in a worker survives the worker's teardown until the caller binds it. a result that fails the transfer check settles the task with the exception. cross thread allocations go through libc malloc in release: smp_allocator keeps a freelist per thread and a producer/consumer pair grew rss without bound. an exception thrown from __unserialize or __wakeup now propagates instead of turning into the parse warning, and unserialize reuses the class table's key for the object's class name instead of copying it into the request arena per object.
futureAwait can take delivery between the task settling and the worker's complete(), which then appended the task and wrote a wake byte nobody read. linux charges a one-byte unix socket send hundreds of bytes of buffer, so a few hundred such races filled it and every worker blocked in send forever; the memory soak hung on ci while passing on macos. complete now skips a task that is already delivered, and both ends of the wake are non-blocking so a backlog nobody collects cannot stall a worker either. the retention step gets a five minute limit
on the four-cpu ci runner the soak grew 17 to 93 MB on some runs and stayed flat on others with no cross-thread frees involved: smp_allocator starts every thread on slot zero and moves it only on contention, so two busy worker vms scattered their freelists across slots and kept mapping slabs. the whole pool, worker vm heaps included, now uses the transfer allocator; the scaling benchmark is unchanged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 2 of #68: channels.
Zphp\Channelis a bounded queue of transferable values shared between threads by identity. A process-wide registry maps ids to channels, every VM that holds one binds its own wrapper, and a wrapper crosses to a worker as its id through native__serialize/__unserialize.send()andrecv()block with optional timeouts,trySend()refuses instead of waiting,close()drains and then ends everyforeach, and channels ride inside task arguments, results, and other channels.Everything that crosses a thread is now a payload: the bytes plus a retained reference to each channel inside, so a channel created in a worker survives the worker's teardown until the caller binds it. A result that fails the transfer check settles the task with the exception. Cross-thread allocations go through libc malloc in release builds, because
smp_allocatorkeeps a freelist per thread and a producer/consumer pair grew RSS without bound.Also in this change: an exception thrown from
__unserializeor__wakeuppropagates instead of becoming the parse warning (tests/unserialize_magic_throws.php), andunserialize()reuses the class table's key for an object's class name instead of copying it into the request arena per object.Tests:
tests/workers/channel.phpinmake workers(single-thread semantics, fan-out, backpressure, pipelines, timeouts, close, worker-created channels, dropped wrappers, a non-transferable result);tests/workers/memory.phpgains channel traffic for the memory-soak job.