feat(daemon): add the daemon IPC surface (poll now + status) - #36
Merged
Conversation
The daemon's forthcoming IPC surface (PROJECT.md §4.1) needs a well-known local socket path. Add a pure socket_file() under the state dir, sibling to pid_file() (state_dir()/"daemon.socket"): the Unix-domain-socket node on POSIX, a loopback-port sidecar file on Windows. Like the other path helpers it only computes the path and never touches the filesystem.
The daemon's forthcoming IPC surface (PROJECT.md §4.1) streams a poll's progress to the TUI/CLI as it runs, so each poll needs to report per-source / per-pair progress without knowing anything about transports. Add atlas.daemon.progress — a small ProgressUpdate model, a ProgressCallback alias, and an emit_progress() helper that reports best-effort (a callback that raises is logged and swallowed; a UI sink dying is not a poll failure). Both poll modules import from here rather than from each other, so there is no edge between the scoring and discovery poll modules. Thread an optional on_progress param (default None, so every existing caller and test is unaffected) through run_discovery_poll, run_aggregator_poll, and run_scoring_poll. Each emits a start bracket (with the source count where known; None for scoring's nested profile x posting loops), an item per unit processed (a failed or inactive source still emits, so the stream reflects every unit touched), and a done bracket.
The first half of the daemon's local IPC surface (PROJECT.md §4.1): the
transport-free heart, split out so it is fully covered without a real socket.
Add atlas.daemon.ipc with a newline-delimited-JSON protocol — an IpcRequest
("status" | "poll") and a discriminated IpcEvent union (StatusEvent,
ProgressEvent, ResultEvent, ErrorEvent) — plus a pure codec (encode/decode
request+event) that rejects malformed/unknown bytes with an IpcProtocolError so
garbled input never crashes the accept loop.
handle_request(request, *, engine, config, store, provider, owner, pid_path,
emit, fetcher) dispatches one request, pushing events through an injected emit
callback: "status" replies with a StatusEvent (reusing daemon_status); "poll"
runs the discovery + aggregator + scoring pass (each in its own session,
reusing the daemon's claim owner so it never double-scores against the
scheduled tick), streaming per-phase ProgressEvents wired from the polls'
on_progress seam and a terminal ResultEvent. Any poll failure becomes a
secret-free ErrorEvent rather than propagating into the transport. The poll
functions are lazy-imported to avoid a daemon<->discovery import cycle.
Add IpcError / IpcProtocolError / IpcUnavailableError to daemon/errors.py.
The second half of the daemon's IPC surface (PROJECT.md §4.1): the socket layer, split so the framing is pure and fully tested and only the real bind/accept/ connect I/O carries a justified # pragma: no cover (AGENTS.md §6.2). A Connection is a binary newline-framed duplex stream (a socket's makefile in production, a paired-BytesIO fake in tests). handle_connection (server side) reads one request, decodes it — a malformed line is answered with an ErrorEvent, never a crash — and runs the injected Dispatch with an emit that frames each event back. stream_events / ipc_request (client side) send the request and stream decoded events to a callback; a refused/absent socket becomes an IpcUnavailableError, and the connection is always closed. The IpcServer Protocol is the injectable listener seam; the real _DefaultIpcServer accepts on a background daemon thread and is built by a pragma'd default_ipc_server() factory (mirroring default_scheduler). The transport dispatches on sys.platform: AF_UNIX on POSIX, loopback TCP with a port sidecar on Windows (stdlib-only, no pywin32) — the AF_UNIX reference guarded for the win32 mypy run like os.startfile in platform/opener.py. Adds FakeConnection + FakeIpcServer test doubles with Protocol-conformance checks; the client/server framing loops are exercised against the BytesIO fake with no real socket.
Wire the IPC surface into the daemon lifecycle and give the CLI a client. start_daemon gains optional ipc_server / dispatch / socket_path params: when all three are supplied it removes a stale socket, serves the IPC surface before the blocking scheduler.start(), and stops it in a finally when the scheduler returns. The plain lifecycle tests pass none and skip serving, so the existing path is unchanged. daemon_start builds a dispatch closure over the daemon's own engine / config / store / provider / owner (reusing the claim owner so an on-demand poll never double-scores against the scheduled tick) and passes default_ipc_server(). Add 'atlas daemon poll [--json]': a thin client that connects over the socket, triggers one poll, streams each ProgressEvent to stderr (so --json on stdout stays pipe-safe), and prints the terminal ResultEvent — exiting 1 on an ErrorEvent or when the daemon is not running (IpcUnavailableError). Pure render_poll_progress / render_poll_result helpers live in cli/daemon.py.
Close Journey B's loop from inside the TUI: the Discover screen can now ask the
running daemon to poll on demand, rather than waiting for the next scheduled tick.
AtlasApp gains an injected socket_path (the atlas tui launcher passes
socket_file()) and a run_poll_now(on_event) that acts as the IPC client;
exposed via a socket_path property so the screen can guard on it. The Discover
screen gets a `g` ("poll now") binding: when the daemon isn't reachable
(socket_path is None) it's a safe warning no-op, otherwise it dispatches a
thread worker that streams per-phase progress toasts (via call_from_thread) and
returns the events. on_worker_state_changed now branches on the worker group —
the poll worker summarizes the terminal ResultEvent and refreshes the queue (so
newly-scored postings appear), surfacing an ErrorEvent or a WorkerState.ERROR
(e.g. IpcUnavailableError) as an error toast without leaving the screen.
Tick the "Daemon + scheduler + IPC" roadmap item (PROJECT.md §15) and move the Phase-2 "Next up" pointer to desktop notifications — the last remaining item. Add a "What has landed" block to STATUS.md and update the phase-progress table. Record the loopback-TCP-on-Windows transport decision in PROJECT.md §18.1, and add the Unreleased CHANGELOG entries (IPC protocol + transport, the progress seam, socket_file(), atlas daemon poll, and the TUI "poll now" binding).
Windows CI was killed mid-run by an async KeyboardInterrupt: two new tests drove the IPC "status" path against the *real* default_process_control, whose is_running(pid) calls os.kill(pid, 0). On POSIX that's a harmless liveness probe, but on Windows signal.CTRL_C_EVENT == 0, so it delivers a console Ctrl-C to the process group — surfacing as a KeyboardInterrupt in whatever test ran next (hence the random, per-job failure points). Inject a FakeProcessControl in test_handle_status_running (IPC) and in the daemon-start dispatch test (CLI), matching the pattern the existing daemon lifecycle tests already use, so liveness never touches a real OS process.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Adds the daemon IPC surface — the last unchecked "Daemon + scheduler + IPC" sub-item of Phase 2 (PROJECT.md §4.1). Until now the daemon ran a
BlockingSchedulerwith no way for a running TUI/CLI to talk to it; work could only happen on the fixed schedule. This adds a small local socket so the TUI/CLI can trigger an on-demand poll and stream its progress, closing Journey B's loop from inside the app.Scope of this PR (a deliberately focused slice): the
status+pollactions, done fully — real progress streaming and a TUI hook. Dispatching AI actions (tailor/cover/score) over IPC is a documented follow-up; desktop notifications are the next Phase-2 item.What changed (per commit)
feat(config)—socket_file()path helper (state_dir()/daemon.socket).feat(daemon)— a progress-callback seam (atlas.daemon.progress) threaded through all three polls via an optionalon_progress(defaultNone, existing callers unaffected).feat(daemon)— the IPC wire protocol (newline-delimited JSON:IpcRequest+ a discriminatedIpcEventunion), a pure codec, and the transport-freehandle_requestcore (status/poll, reusing the daemon's claim owner so an on-demand poll never double-scores against the tick). NewIpcError/IpcProtocolError/IpcUnavailableError.feat(daemon)— the cross-platform transport seam: pure framing (handle_connection,stream_events,ipc_request) tested against an in-memory duplex fake; only the real socket bind/accept/connect pragma'd.IpcServerProtocol + adefault_ipc_server()factory;sys.platformdispatch (AF_UNIX on POSIX, loopback TCP on Windows — stdlib-only).feat(daemon)— serve IPC fromstart_daemon(before the blockingscheduler.start(), stop infinally, unlink a stale socket first) and addatlas daemon poll [--json].feat(tui)— ag"poll now" binding on the Discover screen: a thread worker that streams per-phase progress toasts and refreshes the queue; a safe no-op when no daemon socket is configured.docs— STATUS.md / PROJECT.md / CHANGELOG.Design notes
platform/opener.pyseam pattern STATUS.md prescribed: a Protocol +sys.platform-dispatched, pragma'd transport, with a purehandle_requestas the tested core (STATUS.md line 48).pywin32); theAF_UNIXreference is guarded for themypy --platform win32run. Recorded as a resolved decision in PROJECT.md §18.1.emit), not a generator — it threads cleanly through the existing poll loops and is trivially testable with anemitthat appends to a list.Testing
socket/threading; Pydantic already present).ruff format --check,ruff check,mypy --strictandmypy --strict --platform win32, andpytestat 100% line+branch (1140 tests).handle_requestevent sequences (status / poll / error), client-server framing against aBytesIOduplex fake,start_daemonIPC ordering with aFakeIpcServer,atlas daemon pollviaCliRunner, and the TUIgbinding viaPilot. Only the real socket I/O carries a justified# pragma: no cover._DefaultIpcServer+default_connectover an actual Unix domain socket — apollrequest streamedProgressEvent→ResultEventand the socket file was removed onstop(); an absent socket raisedIpcUnavailableError.Refs: PROJECT.md §4.1, §15