diff --git a/CLAUDE.md b/CLAUDE.md index 2d65b702c..a60cfdc3e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -10,6 +10,7 @@ Responsive scrolling and interaction in the live Toad application: about 16 ms o - Read the source before editing: trace each fact through its declaration, storage, lifecycle and every consumer. - When behavior is hard to explain, simplify. Remove states, layers and distinctions; do not add new ones to cover the old ones. - Prefer deleting code to adding it. If the current design is the problem, say so and propose the replacement. +- WireHistory and TranscriptHistory keep only irreducible page-source and source-identity differences. Shared behavior belongs in LineHistory; behavior shared with other parents belongs in a capability those parents compose. Compare the complete sibling family, not only the reported symptom. - Work moved to a thread is still waiting: this CPython build has a GIL, so a CPU-bound thread takes turns with the UI. - The UI is a view of canonical history. Never keep a second copy of queue, turn, goal or message state in a widget. - Report plainly: what changed, what evidence supports it, what still fails, the next step. No status ceremony; no hashes unless asked. @@ -20,6 +21,7 @@ Responsive scrolling and interaction in the live Toad application: about 16 ms o - Verify in the real application on Tristan's saved sessions with the configured provider. This is standing authorization. Fork sessions for new input; never replay an uncertain input. - Performance evidence is one table: `docs/performance/scroll-series.md`, one line per change, from `tools/performance/capture_live.py` on the fixed scenario (a fork of the saved session; held PageUp from the input box; wheel bursts with reversal; an active response; End; tab return). Record total frame time and input-to-paint latency, median and p95. No per-change receipts. - Known environment: unset `NO_COLOR` and `TOAD_VALIDATION`; run real `st` on an isolated display. +- Privileged tooling is limited to read-only `sudo -n py-spy dump/record`. Never use sudo for Python or process-code injection, including an owned frontend. Load the existing live meter at normal-user frontend startup when attribution needs instrumentation. ## Workflow diff --git a/docs/performance/scroll-series.md b/docs/performance/scroll-series.md index 3185d93b6..7ed0be7ac 100644 --- a/docs/performance/scroll-series.md +++ b/docs/performance/scroll-series.md @@ -130,3 +130,24 @@ Normal launcher with a source overlay, plain `st` on owned :197; installed depen - Confirmed physical native → already-open #comms → native clicks at194c61acc retained the same window, history and page range; source identity and width87 were unchanged, and **all15/15 common prepared line arrays survived**. This closes the earlier unchanged-source warm-array gap for this bounded journey. Evidence: `wire-acceptance/warm-result.json` and receipts/screenshots. No autonomous throughput claim. AST source corpus:283 modules, zero parse omissions; original before/after owner and consumer sites under `.artifacts/held-pageup/resume10604/`, including the corrected Wire override family. Dynamic binding remains qualified, not full NRA proof. Current PR ratchet has LineHistory GodClassExcess+4 against e8716b963 (earlier+6), plus WireHistory ForeignAbsenceProbe+1: the existing empty-wire-interval check now reads membership from PreparedTranscriptPage rather than a separate parameter. No zero-debt claim. No automated Toad tests or synthetic app. All owned frontends/helpers closed; own Xvfb preserved. #675 remains draft, not merged/installed, and full throughput acceptance is blocked on the observed-read delays. Original sessions, BLOCKED goal and UNKNOWN attempts remain untouched. + +## 2026-10-16: installed sidebar-resize baseline (13050), no repair yet + +Normal-user startup of the exact installed T160107Z CLI (Toad d51f2b08a/Core8099d1f6e), plain `st` on owned :210; real growing #comms, no source overlay or provider input. The existing live frame meter was loaded at frontend startup, not injected. One 20-column physical edge drag took1.019s and produced12 changed percentages; only two affected-width frames painted. Input is exclusively through the reviewed `XvfbDisplay.input` owner. This is a sidebar-only diagnostic, **not the fixed scrolling scenario**. + +| Change | Affected-width frame median/p95 ms | Input-to-paint median/p95 ms | Result | +|---|---:|---:|---| +| Instrumented installed baseline, no production repair | 525.8 / 825.9 (2 frames) | 280.3 / 784.3 (24 mouse inputs) | Per12 actual width changes: event-carrier queue wait mean1.94ms (p955.85, zero intervening frames); layout_sidebars5.12ms (18 calls); active native arrangement32.76ms excluding fragment measurement; fragment measure0.79ms (plain wrapping0.27ms subset); visible damage rendering plus composition19.42ms.98 plain fragment rewraps across7 measured widths:29 last-painted visible /69 offscreen, mean2.42/5.75 per changed-width step. No Stylesheet.apply/update_nodes in the drag window despite567 inline rule-update calls. Native arrangement/render work dominates, not plain wrapping. The full12s window's9.0/46.4ms frame figures include idle paints and must not stand in for the drag. | +| Frame-sampled width e6bcf4aa, source candidate; no acceptance | 229.6 / 898.0 (3 frames) | 310.0 / 798.3 (23/24 inputs paired) | Same gesture1.027s/12 changes, same dependencies/meter; growing #comms, not controlled A/B. Core layout event declaration/subscription/producers deleted; no Core queue hop. Native frame start consumes width, sidebar Resize feedback deleted.7 layout starts produced3 paints: one-layout-per-paint target still fails. Sidebar layout32.84ms total versus61.50 baseline; native arrangement414.64ms, fragment measure12.75ms, paint+composition224.75ms.85 plain rewraps:21 visible/64 offscreen. No responsiveness improvement demonstrated. Existing #673 rejected-publication fix is integrated only after this run; not included in these numbers. | +| Frame-sampled width plus existing #673 retention-result fix e757526a; source candidate, still fails | 1157.0 / 1157.0 (1 frame) | 626.0 / 1115.1 (24/24 inputs paired) | Same gesture/meter; added native child-placement counter for subsequent traversal comparisons.10 layout starts/1 paint;1402 child placements/194 unique widgets (75 ThreadRows,5 TranscriptPageViews).105 page plain rewraps:21 visible/84 offscreen. Rejected publications no longer open reader changes, but accepted batches still interrupt unfinished native layouts. Not an improvement. | +| Render-result publication waits for existing native Screen refresh; source candidate, acceptance unfinished | 47.9 / 84.3 (21 frames) | 33.8 / 77.1 (25/25 inputs paired) | Same gesture/meter/counter as preceding row; worst frame120.5ms.12 width changes,23 layout starts,21 paints through actual final50-column bar geometry.2392 child placements/193 unique widgets;168 page plain rewraps48 visible/120 offscreen, plus48 wraps outside page-measure scope (visibility unknown, not relabelled). The frame now finishes before a worker batch can keep the reader and replace its geometry. #673 acceptance remains after that wait. No new timer/queue/cache; existing screen callback owner. Growing histories and host pressure prevent controlled causal/A-B or uninstrumented acceptance claims. Requested p95<50ms/worst<33ms and offscreen-zero still fail. | + +This is an attribution run with call/frame instrumentation, not uninstrumented latency acceptance. Phase spans run from first accepted width to first final-width paint; queue waits are not CPU work. Frame work includes all UI callbacks; worker Markdown parsing is not the main-thread plain-wrap timing. The py-spy100Hz nonblocking profile supports source attribution at `Compositor.reflow_subtrees_steps:1093`, `arrange_widget:1436`, `add_widget:1520/1553`, `SubtreeGeometry.capture:274`, and `Screen._render_damage_steps:1737`;146 samples and129 read errors preclude quantitative sampling percentages. The initial pausing1000Hz profile fell4.4s behind and is retained as an instrumentation failure, not application-latency evidence. Raw captures, source counts and scene identity: `~/.cache/agent-scratch/toad-sidebar-frame-owner/`; source repair awaits Claude's plan review. No automated tests, fake responses or apps. + +## 2026-10-16: P37 shared-candidate diagnostic, not accepted + +Private installed Toad f45a06650/Core3adf, including P37/P57/P60; no overlay. Plain `st` on isolated :218, existing saved active `core-retry`, seven-second held PageUp, wheel reversal, End and tab return through `XvfbDisplay.input`. The startup-loaded common frame meter covered every gesture. This is **not** the fixed fork/active-response scenario; no provider input was submitted. The recorder was not started, so there is no continuous video and no scrolling acceptance. Original data: `~/.cache/agent-scratch/toad-outgoing-live/delivery-labels-p57-p60-before-20261016-1640/`. + +| Change | Total frame median/p95 ms | Input-to-paint median/p95 ms | Result | +|---|---:|---:|---| +| Viewport-owned source requests, before removing the remaining fragment-count clip | 12.2 / 71.6 | 22.1 / 148.6 | 484 frames,277 inputs during the gesture window. Two distinct bottom source page starts before scrolling,35 decoded page responses including repeated live refreshes (P60 owns that repetition). Held PageUp alone:299 frames,155 inputs, frame12.6/70.9ms, input17.7/124.8ms. Fourteen blocked PageUp inputs grouped into five lazy-edge waits:144.1,223.5,32.6,68.9,59.7ms. Screenshots show older history after PageUp and latest body after End/tab return; they cannot prove continuous motion. `rts-results/p37.json` says not_accepted and recording:null. No before/after improvement claim. | diff --git a/src/toad/acp/agent_process.py b/src/toad/acp/agent_process.py index cfedbb1d3..24106a6d5 100644 --- a/src/toad/acp/agent_process.py +++ b/src/toad/acp/agent_process.py @@ -55,7 +55,7 @@ async def start(self, agent, processes=()): selection = RouteSelection.for_child(env, cwd) with ExitStack() as acquisition: try: - acquisition.enter_context(selection.route.admit_client()) + acquisition.enter_context(selection.route.client_scope()) await AcpIngress(selection, Path(cwd), self.attached_root(agent)).preflight() except (OSError, ValueError, RuntimeError) as error: agent.session.record_failure(LogAgentFail( diff --git a/src/toad/acp/agent_session.py b/src/toad/acp/agent_session.py index 7128cf6af..bc24420d6 100644 --- a/src/toad/acp/agent_session.py +++ b/src/toad/acp/agent_session.py @@ -1,10 +1,14 @@ -"""ACP session handshake, durable metadata and reconnection custody.""" +"""ACP session handshake, durable metadata and reconnection.""" from __future__ import annotations from agent_comms.declared_family import DeclaredFamily from agent_comms.child_process import join_retirement import asyncio import os +from pathlib import Path from dataclasses import dataclass, replace +from agent_comms.acp_ingress import AcpIngress +from agent_comms.errors import RelationViolationError +from agent_comms.route_selection import RouteChanged, RouteSelection import toad from toad import constants, jsonrpc from toad.core import events as core_events @@ -15,7 +19,7 @@ from toad.db import DB, SessionMeta from agent_comms.acp_failure import ACPFailure from agent_comms.input_attempt import NotSentInput -from agent_comms.session_load import EnsuringSessionLoadAdmission, ExistingSessionLoadAdmission, FailedSessionLoadAdmission, SessionLoadAdmission +from agent_comms.session_load import EnsuringSessionLoadPlan, ExistingSessionLoadPlan, FailedSessionLoadPlan, SessionLoadPlan PROTOCOL_VERSION = 1 @@ -72,6 +76,9 @@ class RefusedSessionDisposition(FailedSessionDisposition): class AgentSession: """Own readiness and durable session effects; binding identity stays on the controller.""" + + reconnect_timeout_seconds = 30 + def __init__(self, agent, pk): self.agent = agent self.task = None @@ -82,7 +89,7 @@ def __init__(self, agent, pk): self.chosen_name = None self.connected = False self.reconnecting = False - self.load_admission: SessionLoadAdmission = EnsuringSessionLoadAdmission() + self.load_plan: SessionLoadPlan = EnsuringSessionLoadPlan() self.settled = asyncio.Event() @property @@ -118,11 +125,15 @@ def start(self): self.task = asyncio.create_task(self.run()) self.task.add_done_callback(self.session_finished) - async def retire(self): + def begin_retirement(self): self.close() if self.retirement is None: self.retirement = asyncio.create_task(self._retire()) - await join_retirement(self.retirement) + self.retirement.add_done_callback(self.retirement_finished) + return self.retirement + + async def retire(self): + await join_retirement(self.begin_retirement()) async def _retire(self): pending = tuple(task for task in (self.task, *self.responses) @@ -141,8 +152,6 @@ def session_finished(self, task): error = task.exception() if error is not None and self.accepts_updates: self.startup_failed(f"{type(error).__name__}: {error}") - self.retirement = asyncio.create_task(self._retire()) - self.retirement.add_done_callback(self.retirement_finished) def retirement_finished(self, task): if task.cancelled(): @@ -161,6 +170,7 @@ def record_failure( self.failed() self.disposition = state(event) self.agent.events.publish(event) + self.begin_retirement() def session_failed(self, failure): self.record_failure( @@ -189,7 +199,7 @@ def supports_load(self): return self.capabilities.load_session def starting(self): - """Start admission clears readiness before any asynchronous work.""" + """Starting clears readiness before any asynchronous work.""" self.connected = False self.settled.clear() self.agent.controller.reset_configuration() @@ -269,20 +279,14 @@ async def observe_owner(self, presentation) -> None: if self.agent.coordination is None: return binding = presentation.binding - if not self.load_admission.superseded_by(binding): + if not self.load_plan.superseded_by(binding): return - await self.reconnect(ExistingSessionLoadAdmission(binding)) + await self.reconnect(ExistingSessionLoadPlan(binding)) - async def reconnect(self, admission: SessionLoadAdmission | None = None) -> None: + async def reconnect(self, plan: SessionLoadPlan | None = None) -> None: """Reattach the existing view after login or an explicit owner start.""" if self.reconnecting: return - if self.agent.controller.session.bound and not self.supports_load: - raise ValueError("This agent cannot resume its session.") - from pathlib import Path - from agent_comms.acp_ingress import AcpIngress - from agent_comms.route_selection import RouteSelection - requested_cwd = str(self.agent.project_root_path.resolve()) self.reconnecting = True try: @@ -291,17 +295,23 @@ async def reconnect(self, admission: SessionLoadAdmission | None = None) -> None RouteSelection.for_child, os.environ.copy(), requested_cwd) await AcpIngress(selection, Path(requested_cwd), self.agent.process.attached_root(self.agent)).preflight() - except Exception as error: - raise ValueError( - f"Reconnect not attempted: maintenance admission denied: {error}" - ) from error + except (RouteChanged, RelationViolationError) as error: + self.startup_failed(f"Reconnect blocked before restart: {error}") + return await self.agent.stop() - self.load_admission = admission or EnsuringSessionLoadAdmission() + self.load_plan = plan or EnsuringSessionLoadPlan() self.settled.clear() await self.agent.start((self.agent.process,)) - await asyncio.wait_for(self.settled.wait(), timeout=30) - if not self.connected: - raise ValueError("The agent could not reconnect.") + try: + await asyncio.wait_for( + self.settled.wait(), timeout=self.reconnect_timeout_seconds) + except TimeoutError: + self.startup_failed( + f"Reconnect timed out after {self.reconnect_timeout_seconds} seconds " + "waiting for the session handshake to settle." + ) + except (OSError, jsonrpc.APIError) as error: + self.startup_failed(f"Reconnect failed: {type(error).__name__}: {error}") finally: self.reconnecting = False @@ -413,17 +423,17 @@ async def load(self) -> None: if agent_data := session.meta_json.agent_data: self.agent.definition = agent_data - admission, self.load_admission = self.load_admission, EnsuringSessionLoadAdmission() + plan, self.load_plan = self.load_plan, EnsuringSessionLoadPlan() with self.agent.request(): session_load_response = api.session_load(cwd, [], request_session_id, - admission.metadata()) + plan.metadata()) try: response = await session_load_response.wait() except jsonrpc.APIError as error: authority.require() if self.agent.queue_attachment.is_current_request(queue_token): - if failed_command := FailedSessionLoadAdmission.from_failure(error.data): - self.load_admission = failed_command + if failed_command := FailedSessionLoadPlan.from_failure(error.data): + self.load_plan = failed_command raise authority.require() if ( @@ -496,7 +506,7 @@ async def set_name(self, name: str) -> None: def publish_configuration(self, response): - self.load_admission = SessionLoadAdmission.at_response(response.field_meta) + self.load_plan = SessionLoadPlan.at_response(response.field_meta) if (modes := response.modes) is not None: self.agent.controller.publish_modes(modes) self.agent.configuration.receive(response.config_options) diff --git a/src/toad/acp/tool_calls.py b/src/toad/acp/tool_calls.py index 893d9b0a8..8dffc39d6 100644 --- a/src/toad/acp/tool_calls.py +++ b/src/toad/acp/tool_calls.py @@ -2,7 +2,7 @@ from dataclasses import dataclass, replace from acp.schema import ToolCall, ToolCallStart, ToolCallUpdate -from agent_comms.transcript_receipts import AssignedSourceIdentity +from agent_comms.transcript_sources import AssignedSourceIdentity from toad.core import events from toad.acp.client_session import ClientSessionRequest from toad.acp.status import ToolCallStatus diff --git a/src/toad/app.py b/src/toad/app.py index b384b98df..5f357c085 100644 --- a/src/toad/app.py +++ b/src/toad/app.py @@ -4,6 +4,7 @@ from inspect import isabstract from agent_comms.route_selection import RouteSelection from toad.comms_root import CoordinationAccess +from toad.widgets.message_notifications import VisibleReadMarking from toad.thread_actions import ThreadActions from toad.widgets.comms_transfer import Transfers @@ -56,6 +57,7 @@ from toad.application_actions import ApplicationAction, ApplicationCommands from toad.widget_actions import DeclaredWidgetActions from toad.application_lifetime import ApplicationLifetime +from toad.resource_release import AsyncResourceRelease if TYPE_CHECKING: from toad.db import DB @@ -247,7 +249,7 @@ class ApplicationNotRunningError(RuntimeError): """The application has no event loop to accept cross-thread work.""" -class ToadApp(CoreEventReceiver, DeclaredWidgetActions, WorkspaceSessionShutdown, ThemedApp, App, inherit_bindings=False): +class ToadApp(VisibleReadMarking, CoreEventReceiver, DeclaredWidgetActions, WorkspaceSessionShutdown, ThemedApp, App, inherit_bindings=False): """The top level app.""" CSS_PATH = ["toad.tcss", "screens/comms.tcss"] @@ -319,19 +321,18 @@ def __init__( async def _close_all(self) -> None: - await self.thread_navigation.close() - await self.thread_actions.close() - await self.coordination_access.stop_observing() - await super()._close_all() - # Native clients and the preparation owner have joined actual I/O. - # A failed retirement keeps its route resource instead of certifying exit. - await self.coordination_access.close() - - async def on_unmount(self) -> None: - self.colours.stop() - self.terminal_attention.close() - await self.navigation_reader.aclose() - await self.render_processes.aclose() + # View release can fail. The application still owns its observation + # pipe and process pools; an Unmount event may never follow that error. + async with AsyncResourceRelease() as release: + release.push_async_callback(self.render_processes.aclose) + release.push_async_callback(self.navigation_reader.aclose) + release.callback(self.terminal_attention.close) + release.callback(self.colours.stop) + release.push_async_callback(self.coordination_access.close) + release.push_async_callback(super()._close_all) + release.push_async_callback(self.coordination_access.stop_observing) + release.push_async_callback(self.thread_actions.close) + release.push_async_callback(self.thread_navigation.close) if self._background_render_tasks: await asyncio.gather(*tuple(self._background_render_tasks), return_exceptions=True) @@ -529,8 +530,8 @@ async def registry_names(self) -> list[str]: local_threads = self.local_coordination_threads() return [name for name in registered if name not in local_threads] - async def mark_visible_thread_read(self) -> None: - """Report the painted native cursor, never an executor's inbox cursor.""" + def read_visible(self): + """Capture a painted native/CLI cursor before scheduling its guarded write.""" from toad.screens.main import MainScreen from toad.widgets.conversation import Conversation, Window @@ -575,11 +576,14 @@ def mark() -> None: access.write(selected, mark) - try: - await asyncio.to_thread(acknowledge) - except (OSError, ValueError): - # Source replacement or deletion is resolved by the next snapshot. - return + async def mark(): + try: + await asyncio.to_thread(acknowledge) + except (OSError, ValueError): + # Source replacement or deletion is resolved by the next snapshot. + return + + return mark() def schedule_from_thread(self, callback: Callable[[], object]) -> None: diff --git a/src/toad/comms_root.py b/src/toad/comms_root.py index 1eafb4baa..347c03506 100644 --- a/src/toad/comms_root.py +++ b/src/toad/comms_root.py @@ -116,7 +116,6 @@ def __init__(self, app: ToadApp) -> None: self.opening = threading.Lock() self.preparation = app.preparation self.retiring: set[asyncio.Task[None]] = set() - self.marking: asyncio.Task[None] | None = None # One status row per open thread, whichever service it was read from. self.thread_status = OpenThreadStatus(app.call_next) # Views showing the sidebar model now, and consumers waiting for current rows. @@ -167,7 +166,7 @@ def require(self, selected: RouteSelection) -> Comms: if observed is not None and observed.selection == selected: return observed.service with ExitStack() as acquisition: - acquisition.enter_context(selected.route.admit_client()) + acquisition.enter_context(selected.route.client_scope()) service = wire(selected.route) if service.root.resolve() != selected.root or RouteSelection.capture() != selected: raise RouteChanged("Comms route changed while opening the service") @@ -240,10 +239,9 @@ def revision_observed(self, result: RevisionObserved, observed: ObservedCommsSer self.events.publish(CoordinationObserved(result.revision)) if previous is None or result.revision.external_transcripts_changed_since(previous): self.events.publish(ExternalTranscriptChanged()) - if self.sidebar_views and (self.marking is None or self.marking.done()): - # A shown sidebar acknowledges the painted native cursor, as each - # sidebar read did before; the next observation includes it. - self.marking = asyncio.ensure_future(self.app.mark_visible_thread_read()) + if self.sidebar_views: + # The painted view owns read work, not this source observer. + self.app.mark_visible() @handles(SidebarObserved) def sidebar_observed(self, result: SidebarObserved, observed: ObservedCommsService) -> None: diff --git a/src/toad/conversation_kind.py b/src/toad/conversation_kind.py index 4a93c7cc3..8f92783c0 100644 --- a/src/toad/conversation_kind.py +++ b/src/toad/conversation_kind.py @@ -5,6 +5,8 @@ import asyncio from dataclasses import replace from abc import abstractmethod +from typing import ClassVar +from agent_comms.historical_views import ChannelDisplayHistory, ConversationHistoryView, DMDisplayHistory from agent_comms.declared_family import DeclaredFamily from agent_comms.comms import Comms from agent_comms.message_page import MessagePage @@ -14,6 +16,8 @@ class ConversationKind(DeclaredFamily, affix="Conversation"): + history_view: ClassVar[type[ConversationHistoryView]] + @classmethod def threads_needed(cls, snapshot, me, target): return () @@ -34,22 +38,6 @@ def page(cls, comms: Comms, target: str, **bounds) -> MessagePage: ... @abstractmethod def resolve(cls, comms: Comms, me: str, target: str) -> tuple[str, str]: ... - @classmethod - def display_identity(cls, page: MessagePage) -> tuple | None: - if page.historical_display is not None: - displayed = page.historical_display.displayed - return ( - "historical", - displayed.viewer, - displayed.viewer_created_at, - page.history_revision, - ) - return cls.current_identity(page) - - @classmethod - @abstractmethod - def current_identity(cls, page: MessagePage) -> tuple | None: ... - @classmethod def label(cls, target: str) -> str: return target @@ -118,6 +106,8 @@ async def mark_read(cls, comms, target, project, page): ... class ChannelConversation(ConversationKind): + history_view = ChannelDisplayHistory + @classmethod def channels_needed(cls, root, target): return ((root, target),) @@ -137,23 +127,6 @@ def resolve(cls, comms: Comms, me: str, target: str) -> tuple[str, str]: me = comms.registry.require(me).name return me, comms.channels.catalog.read().resolve(target).name - @classmethod - def current_identity(cls, page: MessagePage) -> tuple | None: - scope = page.display_scope - if scope is None or scope.displayed is None: - return None - displayed = scope.displayed - return ( - displayed.viewer, - displayed.viewer_created_at, - displayed.bus_identity, - scope.channel, - scope.targets, - scope.any_mode, - scope.participant_names if scope.any_mode else frozenset(), - page.history_revision, - ) - @classmethod def compose_activity(cls, view) -> ComposeResult: return () @@ -224,6 +197,8 @@ async def mark_read(cls, comms, target, project, page): class DmConversation(ConversationKind): + history_view = DMDisplayHistory + @classmethod def historical_thread(cls, target: str) -> str: return target @@ -249,25 +224,6 @@ def page(cls, comms: Comms, target: str, **bounds) -> MessagePage: def resolve(cls, comms: Comms, me: str, target: str) -> tuple[str, str]: return comms.registry.require(me).name, comms.registry.require(target).name - @classmethod - def current_identity(cls, page: MessagePage) -> tuple | None: - basis = page.display_basis - if basis is None: - return None - return ( - basis.root_identity, - basis.bus_identity, - basis.worktree, - basis.requested_peer, - basis.viewer, - basis.viewer_created_at, - basis.viewer_names, - basis.peer, - basis.peer_created_at, - basis.peer_names, - page.history_revision, - ) - @classmethod def label(cls, target: str) -> str: return f"@{target}" diff --git a/src/toad/core/events.py b/src/toad/core/events.py index f601a594b..a21b27a7a 100644 --- a/src/toad/core/events.py +++ b/src/toad/core/events.py @@ -12,7 +12,7 @@ from agent_comms.declared_family import DeclaredFamily from agent_comms.field_codec import PathText from agent_comms.presentation import WireRevision -from agent_comms.transcript_receipts import AssignedSourceIdentity +from agent_comms.transcript_sources import AssignedSourceIdentity from toad.plan import PlanItem from toad.acp.status import ToolCallStatus from toad.acp.sdk_boundary import ToolCallWire @@ -114,11 +114,6 @@ class ThreadActionsChanged(CoreEvent): """The original application changed an admitted thread action.""" -@dataclass(frozen=True) -class SidebarLayoutChanged(CoreEvent): - """The original sidebar layout changed; native panes derive their paint.""" - - @dataclass(frozen=True) class CoordinationObserved(CoreEvent): """Core's stores changed; the revision Core observed, None without a service. diff --git a/src/toad/navigation_preparation.py b/src/toad/navigation_preparation.py index d8a0ca5d0..948a398b2 100644 --- a/src/toad/navigation_preparation.py +++ b/src/toad/navigation_preparation.py @@ -14,7 +14,8 @@ from agent_comms.registry_document import RegistrySnapshot from agent_comms.declared_family import DeclaredFamily from agent_comms.thread_execution import ConversationPreparation -from agent_comms.session_load import ExistingSessionLoadAdmission +from agent_comms.session_load import ExistingSessionLoadPlan +from toad.owner_preparation import ExistingOwnerPlan from toad.session_tracker import CommsViewKey from toad.conversation_kind import ConversationKind @@ -99,10 +100,10 @@ def bind(self, screen) -> None: @dataclass(frozen=True) class NativeThreadNavigation(ThreadNavigation): - admission: ExistingSessionLoadAdmission + plan: ExistingSessionLoadPlan def bind(self, screen) -> None: - screen.presentation.sources.load_admission = self.admission + screen.presentation.sources.load_plan = self.plan @dataclass(frozen=True) @@ -157,13 +158,11 @@ def external(self) -> ThreadNavigation: return ExternalThreadNavigation(self.root, self.thread, self.project) def native(self) -> ThreadNavigation: - from toad.owner_preparation import ExistingOwnerAdmission - binding = self.thread.execution.owner_binding(self.snapshot, self.thread) - admission = ExistingOwnerAdmission.from_binding(binding) - if admission is None: + plan = ExistingOwnerPlan.from_binding(binding) + if plan is None: return ThreadNavigation(self.root, self.thread, self.project) - return NativeThreadNavigation(self.root, self.thread, self.project, admission) + return NativeThreadNavigation(self.root, self.thread, self.project, plan) class ThreadOpening: diff --git a/src/toad/owner_preparation.py b/src/toad/owner_preparation.py index 7136f055a..12b44a68f 100644 --- a/src/toad/owner_preparation.py +++ b/src/toad/owner_preparation.py @@ -4,13 +4,13 @@ from agent_comms.comms import Comms from agent_comms.thread_presentation import ThreadPresentation, ThreadOwnerBinding, LiveThreadOwnerBinding -from agent_comms.session_load import ExistingSessionLoadAdmission +from agent_comms.session_load import ExistingSessionLoadPlan from agent_comms.mro_dispatch import handles from toad.core.projection import MroProjection -class ExistingOwnerAdmission(MroProjection): - """Project Core's binding into read-only attachment admission, without starting an owner.""" +class ExistingOwnerPlan(MroProjection): + """Project Core's binding into a read-only attachment plan without starting an owner.""" @handles(ThreadOwnerBinding) def unavailable(self, binding): @@ -18,10 +18,10 @@ def unavailable(self, binding): @handles(LiveThreadOwnerBinding) def live(self, binding): - return ExistingSessionLoadAdmission(binding) + return ExistingSessionLoadPlan(binding) @classmethod - def from_binding(cls, binding: ThreadOwnerBinding) -> ExistingSessionLoadAdmission | None: + def from_binding(cls, binding: ThreadOwnerBinding) -> ExistingSessionLoadPlan | None: return cls().dispatch_sync(binding) diff --git a/src/toad/screens/workspace.py b/src/toad/screens/workspace.py index 8a728cd7d..1286dbe18 100644 --- a/src/toad/screens/workspace.py +++ b/src/toad/screens/workspace.py @@ -194,6 +194,13 @@ def release_frame_callback(self, owner, callback) -> bool: def _use_viewport_layout(self) -> bool: return self.is_current + def _start_stepped_layout(self, size=None, scroll=False, step_ns=None): + # Pointer events change only the placement owner and request a frame. + # Acquire its latest target before native layout takes the frame's + # invalidations, never from a Resize callback between layout steps. + self.app.workspace_chrome.layout_sidebars(self) + return super()._start_stepped_layout(size, scroll, step_ns) + def _layout_steps(self, size: Size | None = None, scroll: bool = False, deadline=None): changes = tuple((change, change.layout_requests) for window in self.history_windows if (change := window.history_change) is not None) diff --git a/src/toad/session_entry.py b/src/toad/session_entry.py index 6dbc7a351..6ccc63a61 100644 --- a/src/toad/session_entry.py +++ b/src/toad/session_entry.py @@ -13,6 +13,8 @@ from agent_comms.declared_family import DeclaredFamily from agent_comms.comms import wire +from agent_comms.route_selection import RouteSelection +from toad.owner_preparation import ExistingOwnerPlan from agent_comms.thread_identity import ThreadIncarnation from agent_comms.ui_model.unread import UnobservedUnread, UnreadPresentation @@ -109,8 +111,6 @@ def source(self, sessions: SessionNavigation) -> MainScreen | None: return sessions.app.workspace_sessions.views.get(self.mode) async def reconnect(self, sessions, selected, snapshot, targets): - from agent_comms.route_selection import RouteSelection - if sessions.get(self.mode) is not self or RouteSelection.capture() != selected: return source = self.source(sessions) @@ -123,16 +123,14 @@ async def reconnect(self, sessions, selected, snapshot, targets): if original is None: return thread = snapshot.require(snapshot.canonical_name(original.name)) - from toad.owner_preparation import ExistingOwnerAdmission - binding = thread.execution.owner_binding(snapshot, thread) - admission = ExistingOwnerAdmission.from_binding(binding) - if admission is None: + plan = ExistingOwnerPlan.from_binding(binding) + if plan is None: return if agent := source.presentation.operational_agent: - await agent.session.reconnect(admission) + await agent.session.reconnect(plan) else: - source.presentation.sources.attach_started(source, admission) + source.presentation.sources.attach_started(source, plan) def accepts_launch(self, sessions: SessionNavigation, agent_identity: str, session_id: str) -> bool: source = self.source(sessions) diff --git a/src/toad/session_presentation.py b/src/toad/session_presentation.py index 0e0e2cabb..8372faf22 100644 --- a/src/toad/session_presentation.py +++ b/src/toad/session_presentation.py @@ -1,4 +1,4 @@ -"""Session-owned mounted presentation, with one bounded workspace admission owner.""" +"""Session-owned mounted presentation, with one bounded workspace owner.""" import asyncio from abc import ABC, abstractmethod @@ -13,7 +13,8 @@ from textual.widgets.text_area import TextAreaState from agent_comms.acp_extension import CoordinationChangedUpdate -from toad.owner_preparation import ExistingOwnerAdmission +from agent_comms.session_load import EnsuringSessionLoadPlan +from toad.owner_preparation import ExistingOwnerPlan from toad.input_history import InputHistories from toad.agent_surface import AttachedSurfaceBinding @@ -101,16 +102,17 @@ class OperationalSessionSources: """Own existing operational resources while the view is detached.""" def __init__(self) -> None: - from agent_comms.session_load import EnsuringSessionLoadAdmission - self.agent = None self.shell = None - self.load_admission = EnsuringSessionLoadAdmission() + self.load_plan = EnsuringSessionLoadPlan() def status_row(self, conversation: Conversation): return self.activity_row(conversation).with_delivery(conversation.input_delivery) def activity_row(self, conversation: Conversation): + # A Core history source owns status even when its ACP load fails. + if conversation.transcript.readable: + return conversation.thread_activity.row return conversation.turns.binding.status_row(conversation.thread_activity.row) def agent_info(self, conversation: Conversation) -> str: @@ -146,13 +148,13 @@ def input_ready(self, conversation: Conversation) -> bool: if conversation.transcript.readable: presentation = conversation.thread_presentation return (presentation is not None - and ExistingOwnerAdmission.from_binding(presentation.binding) is not None) + and ExistingOwnerPlan.from_binding(presentation.binding) is not None) return True def unavailable_reason(self, conversation: Conversation) -> str: presentation = conversation.thread_presentation if (conversation.transcript.readable and presentation is not None - and ExistingOwnerAdmission.from_binding(presentation.binding) is None): + and ExistingOwnerPlan.from_binding(presentation.binding) is None): return "Thread is stopped. Choose Start thread from its menu; your draft is kept." agent = conversation.agent if agent is not None and agent.configuration.model.requires_selection: @@ -160,7 +162,7 @@ def unavailable_reason(self, conversation: Conversation) -> str: return "Agent is not ready. Please wait while the agent connects…" def start_agent(self, conversation: Conversation) -> None: - """The operational source owns connection creation and load admission.""" + """The operational source owns connection creation and the load plan.""" from toad.acp.agent import Agent from toad.core import events as core_events @@ -169,16 +171,16 @@ def start_agent(self, conversation: Conversation) -> None: assert conversation._agent_data is not None agent = Agent(conversation.project_path, conversation._agent_data, conversation._agent_session_id, conversation._session_pk) - agent.session.load_admission = self.load_admission + agent.session.load_plan = self.load_plan conversation.agent = agent conversation._native_agent_started_here = True conversation.bind_agent(agent) agent.controller.start_operation(agent.start(conversation.app.session_navigation.processes)) conversation.publish_core(core_events.SessionSubtitleChanged(conversation.agent_title)) - def attach_started(self, screen: "MainScreen", admission) -> None: + def attach_started(self, screen: "MainScreen", plan) -> None: """An explicit Start action may attach the same retained view to its new owner.""" - self.load_admission = admission + self.load_plan = plan screen._agent = screen._agent or screen.app.agent_data conversation = screen.presentation.widget if conversation is not None: diff --git a/src/toad/tool_output.py b/src/toad/tool_output.py index 1fdbe4c30..ca9aad9c3 100644 --- a/src/toad/tool_output.py +++ b/src/toad/tool_output.py @@ -19,7 +19,7 @@ from agent_comms.transcript_events import SentTranscript, ToolEndTranscript from agent_comms.tool_results import ToolDiff, ToolText from agent_comms.sent_message import PublishedSentMessage -from agent_comms.transcript_receipts import AssignedTranscriptSource +from agent_comms.transcript_sources import AssignedTranscriptSource from textual.await_complete import AwaitComplete from rich.text import Text diff --git a/src/toad/transcript_filter.py b/src/toad/transcript_filter.py index 09ab5f3e8..e24250ecf 100644 --- a/src/toad/transcript_filter.py +++ b/src/toad/transcript_filter.py @@ -111,7 +111,7 @@ def scan_available(self, owner): def scan_needed(self, owner): # Once mounted, the projection's ordinary pager owns edge admission. - return owner.window.scroll_y <= owner.prefetch_distance + return owner.window.scroll_y <= owner.window.lookahead.ahead_rows(owner.window.outer_size.height) async def advance(self, filtering, snapshot): from toad.widgets.transcript_history import ProjectedTranscriptHistory, _DrawingStopped @@ -143,8 +143,7 @@ async def advance(self, filtering, snapshot): acquisition.pop_all() drew = bool(projection.fragments_in_range) owner._update_edges() - if not drew: - projection.request_older() + projection.request_preparation() return drew except _DrawingStopped: return False @@ -229,7 +228,7 @@ def resume(self, filtering: TranscriptFilter, snapshot: HistorySourceSnapshot, d owner = filtering.owner if snapshot.current(owner) and drew and filtering.has_older: # Committed painted height decides whether this batch fills the view. - owner.call_after_refresh(owner._check_edges) + owner.call_after_refresh(owner.request_preparation) else: self.request_next(filtering) @@ -240,7 +239,7 @@ def request_next(self, filtering: TranscriptFilter) -> None: ... class AutomaticScanDemand(ScanDemand): def request_next(self, filtering): if filtering.scan_needed(): - filtering.owner.call_later(filtering.owner._check_edges) + filtering.owner.call_later(filtering.owner.request_preparation) class RequestedScanDemand(ScanDemand): @@ -314,6 +313,7 @@ def changed(self) -> None: if self.scanning: self.worker.cancel() owner._update_edges() + owner.window.prepare_history(owner) owner.request_preparation() def scan_needed(self) -> bool: diff --git a/src/toad/transcript_preparation.py b/src/toad/transcript_preparation.py index 9680da7f8..3fe6ab5ba 100644 --- a/src/toad/transcript_preparation.py +++ b/src/toad/transcript_preparation.py @@ -9,7 +9,8 @@ from typing import ClassVar, TYPE_CHECKING from weakref import ReferenceType, ref -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor +from agent_comms.transcripts import TranscriptPage from agent_comms.transcript_events import TranscriptEvent from agent_comms.sent_message import OutgoingMessageReference @@ -82,6 +83,19 @@ def initial_slice(fragments, batch_size: int, newest: bool) -> slice: start = max(0, len(fragments) - batch_size) if newest else 0 return slice(start, min(len(fragments), start + batch_size)) + def advance_empty(self, prepared: PreparedTranscriptPage, older: bool) -> None: + """Keep rendered members while a nonmatching source interval advances.""" + page = prepared.page + self.page = replace(self.page, + before=page.before if older else self.page.before, + after=self.page.after if older else page.after, + has_older=page.has_older if older else self.page.has_older, + has_newer=self.page.has_newer if older else page.has_newer) + + def can_extend(self, older: bool) -> bool: + """Whether this already-decoded page has an unadmitted local range.""" + return self.start > 0 if older else self.stop < len(self.fragments) + def configure(self, *, batch_size, newest=True): if self.fragment_range is None: self.set_fragment_range(self.initial_slice(self.fragments, batch_size, newest)) @@ -209,6 +223,10 @@ class PageRequest: before: TranscriptCursor | None = None after: TranscriptCursor | None = None + @classmethod + def adjacent(cls, page: TranscriptPage, older: bool) -> PageRequest: + return cls(before=page.before) if older else cls(after=page.after) + @dataclass(frozen=True) class FilteredTranscriptBatch: @@ -298,10 +316,22 @@ def release_held(self) -> None: def closed(self) -> bool: return self.scope.closed + def accepts(self, loader, through) -> bool: + """Whether this reader still belongs to the same immutable source cut.""" + return self.loader is loader and self.through == through + @abstractmethod async def get(self, request: PageRequest) -> PreparedTranscriptPage: pass + @abstractmethod + async def prepare(self, request: PageRequest) -> None: + """Prepare the original runtime resource without delivering another copy.""" + + async def prepare_next(self, page: TranscriptPage, older: bool) -> None: + if not self.closed and (page.has_older if older else page.has_newer): + await self.prepare(PageRequest.adjacent(page, older)) + def close(self) -> None: self.runtime.discard_scope(self.scope) if self.projection_source is not None: @@ -416,6 +446,10 @@ async def get(self, request: PageRequest) -> PreparedTranscriptPage: result = await self.runtime.submit(self.work(self.scope, self.loader, self.through, request)) return result.for_view() + async def prepare(self, request: PageRequest) -> None: + if self.loader is not None: + await self.runtime.prepare(self.work(self.scope, self.loader, self.through, request)) + class ProjectedTranscriptSource(PreparedPageSource): """A projected prefix with a fixed boundary and ordinary bidirectional reads. @@ -470,35 +504,28 @@ async def get(self, request: PageRequest) -> PreparedTranscriptPage: reader = (self._upstream if older and self._upstream is not None and not self._upstream.closed else self._raw) initial = await reader.get(request) - prepared = initial - while True: - prepared = await self._project(prepared) - if prepared.fragments: - break - more = prepared.page.has_older if older else prepared.page.has_newer - if not more: - if not older: - boundary = await self.boundary() - return replace(boundary, page=replace( - boundary.page, before=initial.page.before, - has_older=initial.page.has_older, - )) - break - prepared = await reader.get(PageRequest( - before=prepared.page.before if older else None, - after=prepared.page.after if not older else None, + prepared = await self._project(initial) + if not older and not prepared.fragments and not prepared.page.has_newer: + boundary = await self.boundary() + return replace(boundary, page=replace( + boundary.page, before=initial.page.before, + has_older=initial.page.has_older, )) - # Unmatched intervals advance source cursors, not empty widget pages. - # Retain the whole scanned span so reversing direction cannot skip a - # matching record or repeatedly rediscover an empty interval. - page = replace( - prepared.page, - before=prepared.page.before if older else initial.page.before, - after=initial.page.after if older else prepared.page.after, - has_older=prepared.page.has_older if older else initial.page.has_older, - has_newer=True, - ) - return replace(prepared, page=page) + # One requested source page, including an empty matching interval. + # Only the viewport or an explicit filter scan may request another; + # a projection cannot decode arbitrary older pages behind that request. + return replace(prepared, page=replace(prepared.page, has_newer=True)) + + async def prepare(self, request: PageRequest) -> None: + if self.closed: + return + if (request.before == self.through + or request.after == self._boundary.page.before or self._raw is None): + await self.boundary() + return + reader = (self._upstream if request.before is not None + and self._upstream is not None and not self._upstream.closed else self._raw) + await reader.prepare(request) def close(self) -> None: super().close() diff --git a/src/toad/transcript_publication.py b/src/toad/transcript_publication.py index 1209f111b..ba6bb3869 100644 --- a/src/toad/transcript_publication.py +++ b/src/toad/transcript_publication.py @@ -9,7 +9,8 @@ from functools import partial from typing import TYPE_CHECKING -from agent_comms.transcripts import TranscriptCursor, TranscriptPage, TranscriptReadIdentity +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage, TranscriptReadIdentity from agent_comms.acp_extension import TranscriptDraftUpdate, TranscriptSnapshotUpdate from agent_comms.mro_dispatch import MroDispatch, handles from textual.worker import Worker, WorkerCancelled @@ -41,7 +42,7 @@ def native_current(self) -> bool: from toad.widgets.committed_presentation import CommitParticipant return self.current() and all( - widget.commit_claim.admits_native(widget, self.captured) + widget.commit_claim.native_current(widget, self.captured) for widget in self.contents.children if isinstance(widget, CommitParticipant) ) @@ -57,11 +58,11 @@ def source_bound(self, cursor: TranscriptCursor) -> TranscriptCursor: prefixes = tuple(bound for history in self.owner.histories if history in self.captured and history.state.reports_coverage for bound in (history.committed_cursor,) - if bound.receipts_within(cursor)) + if bound.assigned_within(cursor)) prefix = max((bound for bound in prefixes if bound.session_file == cursor.session_file), key=lambda bound: bound.offset, - default=next(iter(prefixes), TranscriptCursor(cursor.session_file, 0))) - return TranscriptCursor(prefix.session_file, prefix.offset, cursor.receipts) + default=next(iter(prefixes), NativeCursor(cursor.session_file, 0))) + return cursor.at_native_prefix(prefix) def source_current(self, cursor: TranscriptCursor) -> bool: return self.current() and self.source_bound(cursor) == cursor diff --git a/src/toad/transcript_source_preparation.py b/src/toad/transcript_source_preparation.py index 7fae86bb9..3ea3113ab 100644 --- a/src/toad/transcript_source_preparation.py +++ b/src/toad/transcript_source_preparation.py @@ -112,11 +112,7 @@ def prepare_viewport(self) -> None: if self.can_draw_on_screen: self.state.check_edges(self) - def _check_edges(self) -> None: - """The source declaration admits pages from its committed native box.""" - raise NotImplementedError - - def _request_page(self, older: bool) -> None: + def request_page(self, older: bool) -> None: self.schedule_source_work(partial(self._prepare_edge, older)) async def _prepare_edge(self, older: bool) -> None: @@ -249,8 +245,3 @@ def drawn_extent(self) -> tuple[int, int]: def source_resumed(self) -> None: """A parked source publishes again: report what its drawn pages mean to their owner.""" raise NotImplementedError - - @property - def prefetch_distance(self) -> int: - """Start background reads before the earlier edge enters the viewport.""" - return self.window.lookahead.ahead_rows(self.window.outer_size.height) diff --git a/src/toad/transcript_state.py b/src/toad/transcript_state.py index cbfde0882..f6c60533b 100644 --- a/src/toad/transcript_state.py +++ b/src/toad/transcript_state.py @@ -108,7 +108,7 @@ def request_latest(self, owner, request: "LatestViewportRequest") -> None: owner.schedule_source_work(partial(owner._jump_latest, request)) def check_edges(self, owner) -> None: - owner._check_edges() + owner.window.prepare_source(owner) @classmethod def successors(cls): diff --git a/src/toad/widgets/committed_presentation.py b/src/toad/widgets/committed_presentation.py index 97b9955e8..b41d4a40b 100644 --- a/src/toad/widgets/committed_presentation.py +++ b/src/toad/widgets/committed_presentation.py @@ -3,14 +3,17 @@ from __future__ import annotations from abc import ABC, abstractmethod -from collections.abc import Callable, Iterable +from collections.abc import Callable, Iterable, Iterator +from functools import partial from dataclasses import dataclass from contextlib import asynccontextmanager from typing import TYPE_CHECKING -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor +from agent_comms.transcripts import TranscriptPage from textual.widget import Widget -from agent_comms.transcript_events import UserTranscript +from agent_comms.transcript_events import TranscriptEvent, UserTranscript +from toad.transcript_state import LatestViewportRequest from agent_comms.sent_message import PublishedSentMessage, OutgoingMessageReference from toad.core.source_events import TranscriptCoverage from agent_comms.acp_extension import InputStartedUpdate @@ -22,7 +25,7 @@ from toad.widgets.conversation import Conversation from toad.widgets.history_anchor import HistoryWindow from toad.transcript_preparation import PreparedTranscriptPage - + from toad.widgets.transcript_fragments import TranscriptFragment @dataclass(frozen=True) class CommitEvidence: @@ -32,12 +35,11 @@ class CommitEvidence: native_inputs: frozenset[str] = frozenset() outgoing: frozenset[OutgoingMessageReference] = frozenset() - class CommitClaim(ABC): def capture(self, settled: bool) -> bool: return True - def admits_native(self, widget: Widget, captured: frozenset[Widget]) -> bool: + def native_current(self, widget: Widget, captured: frozenset[Widget]) -> bool: return True @property @@ -51,20 +53,18 @@ def represents_input(self, input_id: str) -> bool: def covered(self, widget: Widget, evidence: CommitEvidence) -> bool: pass - class CapturedClaim(CommitClaim): """A settled source-owned block captured before the snapshot read.""" def capture(self, settled: bool) -> bool: return settled - def admits_native(self, widget: Widget, captured: frozenset[Widget]) -> bool: + def native_current(self, widget: Widget, captured: frozenset[Widget]) -> bool: return widget in captured def covered(self, widget: Widget, evidence: CommitEvidence) -> bool: return widget in evidence.captured and widget is not evidence.retained_history - @dataclass(frozen=True) class ToolOutputClaim(CapturedClaim): """A compact tool cannot retire before its separate bodies are supplied.""" @@ -74,17 +74,15 @@ class ToolOutputClaim(CapturedClaim): def covered(self, widget: Widget, evidence: CommitEvidence) -> bool: return super().covered(widget, evidence) and self.outgoing <= evidence.outgoing - class RetainedSourceClaim(CapturedClaim): """An original saved source is transferable independently of live output.""" def capture(self, settled: bool) -> bool: return True - def admits_native(self, widget: Widget, captured: frozenset[Widget]) -> bool: + def native_current(self, widget: Widget, captured: frozenset[Widget]) -> bool: return True - @dataclass(frozen=True) class SequenceClaim(CommitClaim): """A wire notice needs its exact persisted counterpart, not a save signal.""" @@ -98,7 +96,6 @@ def required_sequences(self) -> frozenset[int]: def covered(self, widget: Widget, evidence: CommitEvidence) -> bool: return self.sequence is not None and self.sequence in evidence.sequences - @dataclass(frozen=True) class OutgoingMessageClaim(CommitClaim): message: PublishedSentMessage @@ -106,7 +103,6 @@ class OutgoingMessageClaim(CommitClaim): def covered(self, widget: Widget, evidence: CommitEvidence) -> bool: return self.message.reference in evidence.outgoing - class NativeInputClaim(CommitClaim): """An original started input retires only against its saved native identity.""" @@ -117,7 +113,6 @@ def native_id(self) -> str | None: ... def covered(self, widget: Widget, evidence: CommitEvidence) -> bool: return self.native_id in evidence.native_inputs - @dataclass(frozen=True) class StartedInputClaim(NativeInputClaim): """The original native start receipt owns request and journal identity.""" @@ -131,7 +126,6 @@ def native_id(self): def represents_input(self, input_id: str) -> bool: return self.source.input_id == input_id - @dataclass(frozen=True) class TranscriptInputClaim(NativeInputClaim): source: UserTranscript @@ -140,7 +134,6 @@ class TranscriptInputClaim(NativeInputClaim): def native_id(self): return self.source.native_id - class CommitParticipant: """Nominal widget contract; unrelated/local widgets are retained by default.""" @@ -148,21 +141,17 @@ class CommitParticipant: def commit_claim(self) -> CommitClaim: raise NotImplementedError - class SnapshotPresentation(CommitParticipant): @property def commit_claim(self) -> CommitClaim: return CAPTURED_CLAIM - class CheckpointBarrier: """A local interactive session whose ordering cannot be replaced by a snapshot.""" - CAPTURED_CLAIM = CapturedClaim() RETAINED_SOURCE_CLAIM = RetainedSourceClaim() - class CommittedHistory(ConversationBlock, SnapshotPresentation): """A saved source block that precedes the conversation's live tail.""" @@ -172,59 +161,112 @@ class CommittedHistory(ConversationBlock, SnapshotPresentation): def commit_claim(self) -> CommitClaim: return RETAINED_SOURCE_CLAIM - @property - def committed_cursor(self) -> TranscriptCursor: - raise NotImplementedError - - @property - def checkpoint_available(self) -> bool: - raise NotImplementedError - def accepts_commit(self, cursor: TranscriptCursor) -> bool: - previous = self.committed_cursor - return cursor.contains(previous) + return cursor.contains(self.committed_cursor) - async def advance_committed(self, through: TranscriptCursor, is_current: Callable[[], bool]) -> bool: - raise NotImplementedError + def covered_sequences(self, sequences: frozenset[int]) -> frozenset[int]: + return frozenset(sequence for sequence in sequences if self.covers_incoming(sequence)) - def retain_committed(self, through: TranscriptCursor) -> None: - raise NotImplementedError + def covers_outgoing(self, message: PublishedSentMessage) -> bool: + return message.reference in self.outgoing_coverage - def covers_incoming(self, sequence: int) -> bool: - raise NotImplementedError + def source_resumed(self) -> None: + if self._source_state.reports_coverage: + self.publish_core(TranscriptCoverage(tuple(self.coverage_events))) + + async def _page_added(self, page: TranscriptPage, fragments: tuple[TranscriptFragment, ...]) -> None: + if self._source_state.reports_coverage: + from toad.widgets.conversation import Conversation + # The live blocks this page covers are retired as part of adding + # the page, without holding the conversation's message pump. A + # saved viewer outside a conversation has no live blocks. + for ancestor in self.ancestors: + if isinstance(ancestor, Conversation): + ancestor.transcript.covered(TranscriptCoverage(page.events), self).call_when_ready(ancestor) + break + + def publish_committed(self) -> None: + """Acquire source ownership now; report coverage after its first painted frame.""" + self._source_state = self._source_state.publish() + self.call_after_refresh(self.publish_coverage) + self.request_preparation() + + def publish_coverage(self) -> None: + if self._source_state.reports_coverage: + self.publish_core(TranscriptCoverage(tuple(self.coverage_events))) - def covered_sequences(self, sequences: frozenset[int]) -> frozenset[int]: - return frozenset(sequence for sequence in sequences if self.covers_incoming(sequence)) + @property + def coverage_events(self) -> Iterator[TranscriptEvent]: + return (event for page in self.pages for event in page.page.events) @property - def outgoing_coverage(self) -> frozenset[OutgoingMessageReference]: - raise NotImplementedError + def outgoing_coverage(self): + if self._source_state.reports_coverage: + references = frozenset(reference for page in self.pages + for reference in page.prepared.outgoing_coverage) + projection = self.filter.state.overlay + return references | (projection.outgoing_coverage if projection is not None else frozenset()) + return frozenset() - def covers_outgoing(self, message: PublishedSentMessage) -> bool: - return message.reference in self.outgoing_coverage + def covers_incoming(self, sequence: int) -> bool: + if not self._source_state.reports_coverage: + return False + if self.committed_cursor.covers_incoming(sequence): + return True + if sequence in TranscriptCoverage(tuple(self.coverage_events)).sequences: + return True + return self.filter.covers_incoming(sequence) def covers_input(self, native_id: str | None) -> bool: - raise NotImplementedError + """Whether the loaded saved pages hold the input Core started under this identity.""" + return (self._source_state.reports_coverage + and native_id in TranscriptCoverage(tuple(self.coverage_events)).native_inputs) + @property + def committed_cursor(self) -> TranscriptCursor: + return self.through + def retain_committed(self, through: TranscriptCursor) -> None: + """Extend access to saved source without moving the displayed page window.""" + if not self.accepts_commit(through): + raise ValueError("Commit replaces or rewinds the retained source") + self.extend_source(through) + + async def advance_committed( + self, through: TranscriptCursor, is_current: Callable[[], bool], + ) -> bool: + """Show the new tail as one prepared destination, not a walk through the gap.""" + async with self.window.history_lock: + if not self.checkpoint_available or not is_current(): + return False + # Reject scans started with the old bound without discarding their + # already accepted filtered overlay or its backward cursor. + self.retain_committed(through) + operation = self.reserve_source_work() + return await operation.execute(self, partial(self._advance_committed, is_current)) + + async def _advance_committed(self, is_current: Callable[[], bool]) -> bool: + if not is_current() or not self.state.can_draw: + return False + from toad.widgets.history_anchor import LatestPosition + + request = LatestViewportRequest(LatestPosition.hold(self.window)) + return await self._jump_latest(request, is_current) def retirement_candidates(widgets: Iterable[Widget], evidence: CommitEvidence) -> list[Widget]: return [widget for widget in widgets if isinstance(widget, CommitParticipant) and widget.commit_claim.covered(widget, evidence)] - def required_sequences(widgets: Iterable[Widget]) -> frozenset[int]: return frozenset(sequence for widget in widgets if isinstance(widget, CommitParticipant) for sequence in widget.commit_claim.required_sequences) - def protected_blocks(view: Conversation, candidates: Iterable[Widget]) -> set[Widget]: endpoints = set(view.screen._interaction_widgets()) if view.cursor_block is not None: endpoints.add(view.cursor_block) return protected_presentations(candidates, endpoints) - @dataclass(frozen=True) class PreparedCommit: history: CommittedHistory | None @@ -237,7 +279,6 @@ def outgoing(self) -> frozenset[OutgoingMessageReference]: return self.page.outgoing_coverage return self.history.outgoing_coverage - class CheckpointPlan(ABC): """A viewport-owned policy; the coordinator does not branch on block types.""" @@ -268,7 +309,6 @@ async def publication(self, view: Conversation, prepared: PreparedCommit): def finish(self, view: Conversation, cursor: TranscriptCursor) -> None: """A retained viewport does not claim that newer records were painted.""" - class FollowTailCheckpoint(CheckpointPlan): def __init__(self, window: HistoryWindow) -> None: self.revision = window.scroll_revision @@ -306,7 +346,6 @@ def restore_reader(self, window): if self.current(window): window.anchor() - class RetainViewportCheckpoint(CheckpointPlan): def current(self, window: HistoryWindow) -> bool: # Extending the source leaves the loaded rows and reader position intact. @@ -346,6 +385,5 @@ def commit(self, prepared, cursor): assert prepared.history is not None prepared.history.retain_committed(cursor) - def checkpoint_plan(window: HistoryWindow) -> CheckpointPlan: return FollowTailCheckpoint(window) if window.follows_tail else RetainViewportCheckpoint() diff --git a/src/toad/widgets/comms_chat.py b/src/toad/widgets/comms_chat.py index 942073851..764d17ce8 100644 --- a/src/toad/widgets/comms_chat.py +++ b/src/toad/widgets/comms_chat.py @@ -114,7 +114,7 @@ def __init__( self._refreshed_from = None self._bound_root = Path(wire_root).resolve() if wire_root is not None else None self.input_histories.bind_scope(f"comms:{FieldCodec.encode(kind)}:{target}") - self.message_history = WireHistory(self) + self.message_history = WireHistory(view=self) def compose(self) -> ComposeResult: with Window(): @@ -200,7 +200,7 @@ async def on_transcript_source_work_finished(self, event) -> None: # reads the observer's snapshot, which only now shows it. await self._refresh() return - if history.revision == access.revision: + if history.revision.stores == access.revision: return snapshot = history.source_snapshot() if not snapshot.current(history): @@ -217,7 +217,7 @@ async def on_transcript_source_work_finished(self, event) -> None: return if (source is history.source and snapshot.current(history) and source.comms is access.observed_service - and history.revision != revision): + and history.revision.stores != revision): await self._refresh() def prepare_prompt(self) -> None: diff --git a/src/toad/widgets/conversation.py b/src/toad/widgets/conversation.py index c6acfae9f..233709061 100644 --- a/src/toad/widgets/conversation.py +++ b/src/toad/widgets/conversation.py @@ -1210,6 +1210,8 @@ async def action_provider_login(self) -> None: await self.prune_window(0, 0) self.output.boundary() await agent.session.reconnect() + if not agent.session.ready: + return else: await agent.session.authenticate(method_id) self.flash( diff --git a/src/toad/widgets/history_anchor.py b/src/toad/widgets/history_anchor.py index 49dd0ecab..94154ac5f 100644 --- a/src/toad/widgets/history_anchor.py +++ b/src/toad/widgets/history_anchor.py @@ -531,10 +531,42 @@ def prepare_history(self, history: "TranscriptSourcePreparation") -> None: """A history about to be mounted loads the page ranges the kept position needs.""" if self.reader_position is not None: self.reader_position.prepare_history(history) + if self.follows_tail: + self.lookahead.destination(self.outer_size.height, pages_read=len(history.pages)) def latest_limits(self, budget: PresentationBudget) -> tuple[int, int]: """One native destination's row capacity and bounded fragment budget.""" - return max(1, self.outer_size.height), self.lookahead.destination_batch(budget, self.outer_size.height) + return (self.presentation_budget.runway_rows(self.outer_size.height), + self.lookahead.destination_batch(budget, self.outer_size.height)) + + def runway(self, history) -> tuple[int, int] | None: + """Published source rows above and below this viewport; no offscreen layout.""" + visible = self.screen._compositor.visible_widgets + geometry, window = visible.get(history), visible.get(self) + if geometry is None or window is None or not geometry[0].overlaps(window[0]): + return None + region, viewport = geometry[0], window[0] + return viewport.y - region.y, region.bottom - viewport.bottom + + def prepare_source(self, history) -> None: + """The viewport admits source work from the scene it actually displays.""" + if (not history.checkpoint_available or not self.screen.is_active + or not history.selected_categories or self.runway(history) is None): + return + if history.follows_source_tail and history.has_newer: + self.jump_to_latest() + elif history.filter.active: + # Creating a projection uses the original decoded boundary; its + # ordinary pager still asks this viewport for each source page. + history.filter.check_edges() + else: + self.lookahead.prepare_source(history) + + def request_older(self) -> None: + """An explicit earlier-history control is reader intent, even without travel.""" + self.lookahead.requested(-max(1, self.scrollable_content_region.height)) + self._schedule_settle() + self.request_preparation() def history_geometry_targets(self) -> tuple[Widget, ...]: """The widgets the kept position is measured from: layout places them even off screen.""" diff --git a/src/toad/widgets/message_notifications.py b/src/toad/widgets/message_notifications.py index 177b62524..f5c6b16cb 100644 --- a/src/toad/widgets/message_notifications.py +++ b/src/toad/widgets/message_notifications.py @@ -78,6 +78,28 @@ def show_error(self, error: Exception) -> None: self.show(UnavailableNotifications(str(error))) +class VisibleReadMarking: + """One painted-read operation at a time, for pagers and native/CLI sessions.""" + + def __init__(self, *args, **kwargs): + self._read_worker = None + super().__init__(*args, **kwargs) + + def mark_visible(self) -> None: + if self._read_worker is not None and not self._read_worker.is_finished: + return + if (work := self.read_visible()) is not None: + self._read_worker = self.run_worker(work, group="painted-read") + + def prepare_viewport(self) -> None: + super().prepare_viewport() + if self.can_draw_on_screen: + self.mark_visible() + + def source_resumed(self) -> None: + self.mark_visible() + + class HistoryNotifications: """Visible history rows subscribe to Core; paging never decides when outcomes changed.""" diff --git a/src/toad/widgets/outgoing_message.py b/src/toad/widgets/outgoing_message.py index 04102d5a1..b2e95ad57 100644 --- a/src/toad/widgets/outgoing_message.py +++ b/src/toad/widgets/outgoing_message.py @@ -10,7 +10,7 @@ from toad.message_rows import MessageRow from toad.response_delivery import ResponseDelivery from toad.widgets.agent_response import AgentResponse -from toad.widgets.message_divider import MessageClock +from toad.widgets.message_divider import LiveMessageClock from toad.widgets.message_filter import CategorizedBlock, OutboundCategory from toad.widgets.message_notifications import MessageNotifications from toad.widgets.transcript_lines import MessageRowBlock @@ -49,6 +49,6 @@ def compose(self) -> ComposeResult: yield AgentResponse(message.text, delivery=ResponseDelivery.from_route(message.require_reply_route()), show_divider=self.show_header, - clock=MessageClock.recorded(message.timestamp)) + clock=LiveMessageClock()) if self.handling_references: yield MessageNotifications() diff --git a/src/toad/widgets/presentation_window.py b/src/toad/widgets/presentation_window.py index 7c1675654..c232598bb 100644 --- a/src/toad/widgets/presentation_window.py +++ b/src/toad/widgets/presentation_window.py @@ -1,10 +1,12 @@ """Shared budgets for a source-backed window of native widget presentation.""" -from dataclasses import dataclass +from dataclasses import dataclass, field, fields from abc import ABC, abstractmethod from collections.abc import Iterable from itertools import zip_longest +from functools import partial from math import ceil +from operator import gt, ge from time import monotonic, get_clock_info from typing import TYPE_CHECKING @@ -37,26 +39,25 @@ class PresentationBudget: satisfy a widget budget. """ - max_items: int = 24 - batch_items: int = 4 - reserve_batches: int = 4 - minimum_widgets: int = 300 - widgets_per_row: int = 10 - buffer_viewports: int = 3 - lookahead_seconds: float = 1.0 - scroll_idle_seconds: float = 0.2 + max_items: int = field(default=24, metadata={"zero_bound": gt}) + batch_items: int = field(default=4, metadata={"zero_bound": gt}) + reserve_batches: int = field(default=4, metadata={"zero_bound": ge}) + minimum_widgets: int = field(default=300, metadata={"zero_bound": gt}) + widgets_per_row: int = field(default=10, metadata={"zero_bound": ge}) + buffer_viewports: int = field(default=3, metadata={"zero_bound": gt}) + destination_pages: int = field(default=2, metadata={"zero_bound": gt}) + lookahead_seconds: float = field(default=1.0, metadata={"zero_bound": gt}) + delivery_safety_factor: float = field(default=2.0, metadata={"zero_bound": gt}) + scroll_idle_seconds: float = field(default=0.2, metadata={"zero_bound": gt}) def __post_init__(self) -> None: - for name in ("max_items", "batch_items", "minimum_widgets", "buffer_viewports"): - value = getattr(self, name) - if type(value) is not int or value < 1: - raise ValueError(f"{name} must be a positive integer") - for name in ("reserve_batches", "widgets_per_row"): - value = getattr(self, name) - if type(value) is not int or value < 0: - raise ValueError(f"{name} must be a non-negative integer") - if self.lookahead_seconds <= 0 or self.scroll_idle_seconds <= 0: - raise ValueError("Preparation timing must be positive") + for declaration in fields(self): + value = getattr(self, declaration.name) + bound = declaration.metadata["zero_bound"] + # Python permits an integer wherever a float is declared, but a + # bool is not a numeric budget. The declaration supplies the bound. + if type(value) not in (declaration.type, int) or not bound(value, 0): + raise ValueError(f"{declaration.name} violates its declared {bound.__name__} zero bound") def item_limit(self, protected_items: int) -> int: return max(self.max_items, protected_items + self.reserve_batches * self.batch_items) @@ -64,6 +65,10 @@ def item_limit(self, protected_items: int) -> int: def widget_limit(self, viewport_rows: int) -> int: return max(self.minimum_widgets, viewport_rows * self.widgets_per_row) + def can_read_destination(self, pages: int) -> bool: + """A bottom destination has bounded source reads, even for sparse rows.""" + return pages < self.destination_pages + def runway_rows(self, viewport_rows: int) -> int: return max(1, viewport_rows) * self.buffer_viewports @@ -107,10 +112,20 @@ def rows(self, horizon: float) -> float: ... def moved(self, travel: float, elapsed: float) -> "PreparationDemand": return MovingPreparation(travel / elapsed) - def permits(self, older: bool) -> bool: - """Whether this edge still serves the reader's current direction.""" + def accepts_result(self, older: bool) -> bool: + """An admitted read may finish after input settles; it admits no next read.""" return True + def read_rows(self, preparation, history, older: bool, runway: tuple[int, int]) -> int: + """Stationary input never requests another source page.""" + return 0 + + def source_read(self) -> None: + """Only a bounded destination counts the pages admitted by its request.""" + + def prepare_ahead(self, preparation, history, page, older: bool) -> None: + """Only measured movement speculates beyond its current destination.""" + def destination_painted(self) -> "PreparationDemand": return self @@ -135,25 +150,52 @@ def moved(self, travel: float, elapsed: float) -> PreparationDemand: def rows(self, horizon: float) -> float: return self.velocity * horizon - def permits(self, older: bool) -> bool: + def accepts_result(self, older: bool) -> bool: return older == (self.velocity < 0) + def read_rows(self, preparation, history, older: bool, runway: tuple[int, int]) -> int: + if not self.accepts_result(older): + return 0 + ahead = preparation.ahead_rows(preparation.window.outer_size.height) + return max(0, ceil(ahead) - runway[0 if older else 1]) + + def prepare_ahead(self, preparation, history, page, older: bool) -> None: + if self.accepts_result(older): + # The existing runtime owns the single decoded/prepared result. + # This native worker only waits for that original bounded work; + # foreground paging shares it, without a page store in the widget. + history.run_worker(partial(history._reader().prepare_next, page, older), + group="transcript-read-ahead", exclusive=True, + exit_on_error=False) -@dataclass(frozen=True) + +@dataclass class DestinationPreparation(PreparationDemand): - viewport_rows: int + target_rows: int + pages_read: int = 0 def rows(self, horizon: float) -> float: - return self.viewport_rows + return self.target_rows def moved(self, travel: float, elapsed: float) -> PreparationDemand: # End's own destination layout is not an intermediate scroll demand. # An explicit reversal immediately replaces the burst. return super().moved(travel, elapsed) if travel < 0 else self - def permits(self, older: bool) -> bool: - # The existing latest-page operation owns a jump, never edge traversal. - return False + def accepts_result(self, older: bool) -> bool: + # A newer destination supersedes older work through its source snapshot. + return older + + def read_rows(self, preparation, history, older: bool, runway: tuple[int, int]) -> int: + if not older: + return 0 + edge = history.pages[0].prepared + if not edge.can_extend(older) and not preparation.budget.can_read_destination(self.pages_read): + return 0 + return max(0, self.target_rows - history.drawn_extent[0]) + + def source_read(self) -> None: + self.pages_read += 1 def destination_painted(self) -> PreparationDemand: return StationaryPreparation() @@ -232,8 +274,8 @@ def settle(self) -> None: self.demand = StationaryPreparation() self.delivery_seconds = 0.0 - def destination(self, rows: int) -> None: - self.demand = DestinationPreparation(rows) + def destination(self, rows: int, *, pages_read: int = 0) -> None: + self.demand = DestinationPreparation(self.budget.runway_rows(rows), pages_read) self.sampled_at = monotonic() def destination_complete(self) -> None: @@ -253,7 +295,8 @@ def travel_rows(self) -> float: elapsed = monotonic() - self.sampled_at if elapsed >= self.idle_seconds: return 0 - return self.demand.rows(max(self.budget.lookahead_seconds, self.delivery_seconds)) + return self.demand.rows(max(self.budget.lookahead_seconds, + self.delivery_seconds * self.budget.delivery_safety_factor)) def ahead_rows(self, viewport_rows: int) -> int: # Travel is a demand, not a native retention limit. The existing worker @@ -262,16 +305,22 @@ def ahead_rows(self, viewport_rows: int) -> int: destination = (window.scroll_target_y - window.scroll_y if window.app.animator.is_being_animated(window, "scroll_y") else 0) baseline = self.budget.runway_rows(viewport_rows) - requested = max(baseline, ceil(abs(self.travel_rows)), ceil(abs(destination))) - # The shared preparation runtime is the admission bound. Express its - # fragment capacity in measured rows so source paging cannot outrun it. - capacity = ceil(self.window.app.preparation.max_entries * self.window.rows_per_fragment) - return min(requested, max(baseline, capacity)) + return max(baseline, ceil(abs(self.travel_rows)), ceil(abs(destination))) def destination_batch(self, budget: PresentationBudget, viewport_rows: int) -> int: # A bottom viewport must be complete before its one tree publication. # One selected fragment supplies at least one drawn row; visible # destination items may exceed the nominal working-set item budget. - return min(self.window.app.preparation.max_entries, max( - budget.item_limit(0), viewport_rows, - )) + return max(budget.item_limit(0), viewport_rows) + + def read_rows(self, history, older: bool) -> int: + if not (history.has_older if older else history.has_newer): + return 0 + runway = self.window.runway(history) + return 0 if runway is None else self.demand.read_rows(self, history, older, runway) + + def prepare_source(self, history) -> None: + for older in (True, False): + if self.read_rows(history, older): + history.request_page(older) + return diff --git a/src/toad/widgets/side_bar.py b/src/toad/widgets/side_bar.py index 1f67e26f2..12975ab93 100644 --- a/src/toad/widgets/side_bar.py +++ b/src/toad/widgets/side_bar.py @@ -518,7 +518,7 @@ def _resize(self, screen_x: int) -> None: width = self._start_width + direction * (screen_x - self._start_x) percent = round(100 * width / max(1, bar.screen.size.width)) if app.sidebar_layout.width(bar.id, percent): - app.events.publish(core_events.SidebarLayoutChanged()) + bar.screen.refresh(layout=True) def action_resize(self, direction: int) -> None: bar = self.query_ancestor(SideBar) @@ -815,17 +815,13 @@ def _apply_layout(self, resolved) -> bool: ) return True - def on_resize(self) -> None: - if self.is_mounted and self.screen.is_current: - cast("ToadApp", self.app).workspace_chrome.layout_sidebars(self.screen) - @on(SidebarSlider.Changed) def on_sidebar_slider(self, event: SidebarSlider.Changed) -> None: event.stop() if event.slider.kind == "width" and self.id is not None: app = cast("ToadApp", self.app) if app.sidebar_layout.width(self.id, event.value): - app.events.publish(core_events.SidebarLayoutChanged()) + self.screen.refresh(layout=True) @on(SidebarAction.Pressed) def on_sidebar_action(self, event: SidebarAction.Pressed) -> None: @@ -834,7 +830,7 @@ def on_sidebar_action(self, event: SidebarAction.Pressed) -> None: return app = cast("ToadApp", self.app) event.action.apply(app.sidebar_layout, self.id) - app.events.publish(core_events.SidebarLayoutChanged()) + self.screen.refresh(layout=True) @on(widgets.Collapsible.Collapsed) @on(widgets.Collapsible.Expanded) @@ -890,8 +886,7 @@ def watch_collapsed(self, collapsed: bool) -> None: panel.widget.sidebar_visibility_changed() self._presented_layout = None if self.is_mounted and self.id in cast("ToadApp", self.app).sidebar_layout.placements: - cast("ToadApp", self.app).workspace_chrome.layout_sidebars(self.screen) - cast("ToadApp", self.app).events.publish(core_events.SidebarLayoutChanged()) + self.screen.refresh(layout=True) def render(self) -> str: return ("<" if self.right else ">") if self.collapsed else "" diff --git a/src/toad/widgets/transcript_fragments.py b/src/toad/widgets/transcript_fragments.py index 779ca568a..746946043 100644 --- a/src/toad/widgets/transcript_fragments.py +++ b/src/toad/widgets/transcript_fragments.py @@ -242,6 +242,10 @@ def outgoing_sources(self): def for_body_sources(self, supplied): return self + def text_rows(self, lines, theme) -> range: + """The fragment's content occupies its drawn rows unless it owns a header.""" + return range(lines.height) + def blocks(self, shown: Shown) -> tuple[LineBlock, ...]: """What this fragment draws in committed history when shown so, one block per visual part. @@ -305,10 +309,12 @@ def lines_for(self, width: int, theme): def invalidate_lines(self): self.__dict__.pop("_lines", None) - def hold_lines(self, request, lines) -> None: - """Retain the owning request, unless the fragment changed its source blocks.""" + def hold_lines(self, request, lines) -> bool: + """Report whether these rows were retained for the fragment's source.""" if request.blocks is self.line_blocks: self._lines = request, lines + return True + return False def plain_lines(self, width: int, theme): """Plain wrapped lines drawn until styled lines arrive; one per width.""" diff --git a/src/toad/widgets/transcript_history.py b/src/toad/widgets/transcript_history.py index 937612650..4cf221582 100644 --- a/src/toad/widgets/transcript_history.py +++ b/src/toad/widgets/transcript_history.py @@ -15,7 +15,8 @@ from typing import TYPE_CHECKING from weakref import ref -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor +from agent_comms.transcripts import TranscriptPage from agent_comms.ui_model.observation import ReadNotifications from agent_comms.mro_dispatch import MroDispatch, handles from agent_comms.transcript_events import ( @@ -59,7 +60,7 @@ selection_painted, ) from toad.widgets.committed_presentation import CommittedHistory, TranscriptInputClaim -from toad.core.source_events import TranscriptCoverage +from toad.widgets.history_anchor import HeldReaderDestination, LatestPosition from toad.widgets.message_filter import ( all_categories, CategorizedBlock, MessageCategory, apply_block_filter, ) @@ -390,10 +391,17 @@ async def _render_pending(self) -> None: async def _publish(self, ready: list[tuple[TranscriptFragment, TranscriptLinesRenderTask, PreparedLines]]) -> None: from toad.widgets.history_anchor import HistoryWindow + # Native frame completion owns when new row geometry may replace the + # measured scene. A worker finishing between layout steps must not keep + # a reader and cancel that frame before it has ever been displayed. + await self.screen.wait_for_refresh() + # Holding rows does not change the page's measured layout. Retain every + # result before opening a history change: keeping the reader can + # interrupt an unfinished layout, even without accepted rows. + if not any([task.hold(self, fragment, lines) for fragment, task, lines in ready]): + return window = self.query_ancestor(HistoryWindow) async with window.preserve_history(root=self): - for fragment, task, lines in ready: - task.hold(self, fragment, lines) self.range_changed() @classmethod @@ -437,7 +445,6 @@ async def select_range( async with window.preserve_history(root=self): return await self.prepared.replace_range(self, selected, current) - class LineHistory(HistoryNotifications, OpensMessageNames, TranscriptSourcePreparation, ConversationBlock, CategorizedBlock, VerticalGroup): """History drawn as lines, a bounded range of pages at a time. @@ -449,6 +456,7 @@ class LineHistory(HistoryNotifications, OpensMessageNames, TranscriptSourcePrepa CACHE_SUBTREE_GEOMETRY = True MAX_FRAGMENTS = 240 + page_buffer = TranscriptPageBuffer @property def message_category(self) -> None: @@ -477,6 +485,46 @@ def __init__(self, prepared: PreparedTranscriptPage, loader: Callable[..., Await # Bound when mounted; a history closed before its mount has no window to leave. self.window: HistoryWindow | None = None + async def park_document(self) -> PreparedPageSource: + """Stop reading and hand the drawn pages to their reader, to be rebuilt when the tab returns.""" + await self.retire_source(parked=True) + source = self._reader() + projection = self.filter.state.overlay + projected = await projection.park_document() if projection is not None else None + pages = tuple(page.prepared for page in self.pages) + position = self.window.reader_position + await source.runtime.run_thread(source.measure_held, pages, projected, + () if position is None else position.ranges) + source.park(pages, projected) + return source + + @classmethod + def from_source(cls, source: PreparedPageSource, **source_fields) -> LineHistory: + """Rebuild a parked history from its pages; it covers no live blocks until it resumes.""" + pages = source.held_pages + if not pages or source.closed: + raise ValueError("No retained transcript admission") + history = cls(pages[0], source.loader, **source_fields) + history.through = source.through + history._page_buffer = source + history._source_state = ParkedSourceTranscript(LiveTranscript()) + history.pages.extend(TranscriptPageView( + page, batch_size=history.budget.batch_items, + ) for page in pages[1:]) + projected = source.projection_source + if projected is not None: + from toad.transcript_filter import Filtered + projection = ProjectedTranscriptHistory.from_source(projected, owner=history) + history.filter.state = Filtered(projection) + return history + + def acquire_document(self) -> None: + """The rebuilt pages are mounted: the reader no longer keeps them for a later rebuild.""" + if self._page_buffer is not None: + self._page_buffer.release_held() + if (projection := self.filter.state.overlay) is not None: + projection.acquire_document() + def resume_source(self) -> None: super().resume_source() if (projection := self.filter.state.overlay) is not None: @@ -537,10 +585,22 @@ async def prepare_rows(self, current: Callable[[], bool], *, window: HistoryWind current, window=window, target=target, rows_needed=max(0, rows_needed), ) + @property + def fragments(self) -> tuple[TranscriptFragment, ...]: + return tuple(fragment for page in self.pages for fragment in page.fragments) + @property def fragments_in_range(self) -> tuple[TranscriptFragment, ...]: return tuple(fragment for page in self.pages for fragment in page.fragments_in_range) + def read_fragments(self) -> tuple[TranscriptFragment, ...]: + """Fragments whose own text has a row in the viewport.""" + if not self.is_attached or not self.can_draw_on_screen: + return () + return tuple( + fragment for page in self.pages for fragment, rows, lines in page.visible_rows(self.window) + if (text := fragment.text_rows(lines, self.app.colours.line_theme)) and rows.start < text.stop and text.start < rows.stop) + def visible_fragments(self) -> tuple[TranscriptFragment, ...]: return tuple(fragment for page in self.pages for fragment in page.visible_fragments(self.window)) @@ -670,7 +730,8 @@ def paging_window(self): # loaded ranges and the filter. A check that finds a read running is # skipped, and the read asks again only when one of these changed # under it, so a filter chosen during a read must count. - return self.through, self.filter.state, tuple(page.capture_range() for page in self.pages) + return self.through, self.filter.state, tuple( + (page.page.before, page.page.after, page.capture_range()) for page in self.pages) def capture_reader_ranges(self) -> tuple[PageFragmentRange, ...]: """Retain the original source ranges that a returning reader needs.""" @@ -733,18 +794,18 @@ def projected_source(self, selected) -> ProjectedTranscriptSource: ) @property - def _follow_source_tail(self) -> bool: + def follows_source_tail(self) -> bool: return self.window.follows_tail @property def has_older(self) -> bool: page = self.pages[0] - return page.start > 0 or page.page.has_older + return page.prepared.can_extend(True) or page.page.has_older @property def has_newer(self) -> bool: page = self.pages[-1] - return page.stop < len(page.fragments) or page.page.has_newer + return page.prepared.can_extend(False) or page.page.has_newer @property def needs_latest_publication(self) -> bool: @@ -822,12 +883,13 @@ def source_checkpoint_available(self) -> bool: def _reader(self) -> PreparedPageSource: reader = self._page_buffer - if reader is None or reader.loader is not self.loader or reader.through != self.through: + if reader is None or not reader.accepts(self.loader, self.through): if reader is not None: reader.close() - self._page_buffer = reader = TranscriptPageBuffer( + self._page_buffer = reader = self.page_buffer( self.loader, self.through, self.app.preparation, ) + reader.through = self.through return reader @property @@ -835,23 +897,6 @@ def drawn_extent(self) -> tuple[int, int]: pages = [page for page in self.pages if page.line_count] return sum(page.line_count for page in pages), sum(page.stop - page.start for page in pages) - @property - def destination_needs_older(self) -> bool: - """Whether one more complete older fragment fits above the destination.""" - if not self.has_older or not self._follow_source_tail: - return False - missing = self.window.scrollable_content_region.height - self.drawn_extent[0] - if missing <= 0: - return False - edge = self.pages[0] - if edge.start == 0: - return edge.page.has_older - fragment = edge.fragments[edge.start - 1] - if not fragment.shown_in(self.selected_categories): - return True - lines = fragment.lines_for(self.line_width, self.app.colours.line_theme) - return lines is None or lines.height <= missing - def _size_updated(self, size, virtual_size, container_size, layout=True) -> bool: # Edges are read from this history's laid-out box. A check made # before layout gave it one (a filter just chosen, a first page) finds @@ -861,47 +906,6 @@ def _size_updated(self, size, virtual_size, container_size, layout=True) -> bool self.request_preparation() return changed - def _check_edges(self) -> None: - if (not self.checkpoint_available - or not self.screen.is_active or not self.selected_categories): - return - runway = self._runway() - if runway is None: - return - above, below = runway - # Original published geometry admits source work, even while a body is - # preparing. Its worker owns read/prepare/native mutation; the existing - # viewport frame owner alone decides when those bodies may be painted. - if self._follow_source_tail and self.has_newer: - self._request_page(False) - return - if self.filter.active: - self.filter.check_edges() - return - if self.destination_needs_older: - # A destination is a bottom viewport, not one transport page. Fill - # its visible rows before upward input has scroll travel to spend. - self._request_page(True) - return - demand = self.window.lookahead.demand - for older, available, runway in ((True, self.has_older, above), - (False, self.has_newer, below)): - if available and runway <= self.prefetch_distance and demand.permits(older): - self._request_page(older) - return - - def _runway(self) -> tuple[int, int] | None: - """Drawn rows above and below the window's viewport; None while this history is off screen.""" - # Off-screen pagers must not ask for their region: after a scroll that - # falls back to rebuilding geometry for the *whole* mounted transcript. - # The layout signal also calls us once this pager enters the viewport. - visible = self.screen._compositor.visible_widgets - geometry, window = visible.get(self), visible.get(self.window) - if geometry is None or window is None or not geometry[0].overlaps(window[0]): - return None - region, viewport = geometry[0], window[0] - return viewport.y - region.y, region.bottom - viewport.bottom - @on(JumpToLatest.Requested) def on_jump(self, event: JumpToLatest.Requested) -> None: event.stop() @@ -910,10 +914,9 @@ def on_jump(self, event: JumpToLatest.Requested) -> None: @on(HistoryEdge.Requested) def on_earlier_history(self, event: HistoryEdge.Requested) -> None: event.stop() + self.window.request_older() if self.filter.active: self.filter.request_older() - elif self.has_older and self.state.accepts_source_work: - self._request_page(True) async def _publish_latest(self, current: Callable[[], bool]) -> bool: """Prepare and publish the complete bottom destination as one tree change.""" @@ -923,7 +926,9 @@ async def _publish_latest(self, current: Callable[[], bool]) -> bool: prepared = edge.prepared.for_view() else: prepared = (await self._reader().get(PageRequest(before=self.through))).for_view() + latest = prepared rows, limit = window.latest_limits(self.budget) + pages_read = 1 destination: list[PreparedTranscriptPage] = [] while rows > 0 and limit > 0: rows -= await prepared.prepare_rows( @@ -939,18 +944,23 @@ async def _publish_latest(self, current: Callable[[], bool]) -> bool: limit -= take if rows <= 0 or self.loader is None: break - if not prepared.page.has_older or limit <= 0: + if (not prepared.page.has_older or limit <= 0 + or not window.presentation_budget.can_read_destination(pages_read)): break - prepared = (await self._reader().get(PageRequest(before=prepared.page.before))).for_view() - if not destination: - return False - + prepared = (await self._reader().get(PageRequest.adjacent(prepared.page, True))).for_view() + pages_read += 1 async with window.history_lock: if not current() or self.pages[-1] is not edge: return False async with window.preserve_history(root=self): previous = tuple(self.pages) views = deque() + if not destination: + # An empty source is still an answered source. Reuse the + # existing widget; no empty page or perpetual placeholder. + edge.prepared = latest + edge.range_changed() + views.append(edge) try: for prepared in destination: async with TranscriptPageView.acquire( @@ -960,46 +970,131 @@ async def _publish_latest(self, current: Callable[[], bool]) -> bool: except _DrawingStopped: return False self.pages = views - await self.remove_children(previous) + await self.remove_children(page for page in previous if page not in views) self._update_edges() return True - async def _load_page(self, older: bool, prepared: PreparedTranscriptPage | None = None, - *, fresh: bool = False) -> bool: + async def show_newest(self, through, initial: PreparedTranscriptPage) -> None: + """Replace what is drawn with the newest page of the source as read now.""" + await self._clear(initial) + self.extend_source(through) + await self._jump_latest(LatestViewportRequest(LatestPosition.hold(self.window))) + + async def _clear(self, initial: PreparedTranscriptPage) -> None: + """Forget every drawn page and everything prepared: they belong to another reading.""" + async with self.window.history_lock: + async with self.window.preserve_history(root=self): + self._generation += 1 + if self._page_buffer is not None: + self._page_buffer.close() + self._page_buffer = None + retired = tuple(self.pages) + view = TranscriptPageView(initial, + batch_size=self.budget.batch_items) + view.visible_categories = self.selected_categories + await self.mount(view, before=self.newer) + self.pages = deque([view]) + self.through = view.page.after + self.remove_children(retired) + self._update_edges() + + async def extend(self, through) -> bool: + """The bus grew to `through`: draw this conversation's new messages for a tail reader. + + A reader elsewhere keeps their place and is offered the jump to them. + False when the drawn pages changed during the read; the next refresh + reads again. + """ + self.through = through + edge = self.pages[-1] + if (edge.page.after != self.through + and (destination := HeldReaderDestination.held(self.window)) is not None + and await destination.source_arrived(self)): + return True + if edge.page.after == self.through: + return True + if self.has_newer: + # The drawn pages stop short of the end: paging from their edge + # reaches the new messages. + if not edge.page.has_newer: + edge.page = replace(edge.page, has_newer=True) + return True + snapshot = self.source_snapshot() + prepared = await self._reader().get(PageRequest(after=edge.page.after)) + # A page read from a bus the route no longer selects is never drawn. + if (not await self.source_is_current(snapshot) + or self.pages[-1] is not edge or self.has_newer): + return False + if not await self.accept_extension(edge, prepared): + return False + if not prepared.fragments: + edge.prepared.advance_empty(prepared, False) + elif not self.window.follows_tail: + edge.page = replace(edge.page, has_newer=True) + self._update_edges() + else: + await self._load_page(False, prepared) + if not snapshot.current(self): + return False + tail = self.pages[-1] + if tail.page.after != self.through: + # Reader movement can reject preparation. The original source + # cut still exposes that gap to paging and End. + tail.page = replace(tail.page, has_newer=True) + self._update_edges() + if self.pages[-1].page.has_newer and self.window.follows_tail: + # More arrived than one page holds: go to the newest instead + # of drawing every page between. + await self._jump_latest(LatestViewportRequest(LatestPosition.hold(self.window))) + return True + + async def source_is_current(self, snapshot) -> bool: + return snapshot.current(self) + + async def accept_extension(self, edge, prepared) -> bool: + """The immutable transcript reader already validates its source cut.""" + return True + + async def _load_page(self, older: bool, prepared: PreparedTranscriptPage | None = None) -> bool: """Publish one readable range; report whether its edge still needs rows.""" snapshot = self.source_snapshot() window, loader = snapshot.window, self.loader + # Only an append beyond the fully drawn, followed end is an arrival. + # Local range growth, older paging and destination restoration are not. + fresh = not older and not self.has_newer and self.follows_source_tail def current() -> bool: if not snapshot.current(self): return False - return fresh or window.follows_tail or window.lookahead.demand.permits(older) + return (window.follows_tail if fresh + else window.lookahead.demand.accepts_result(older)) try: if not current(): return False edge = self.pages[0] if older else self.pages[-1] page_range = edge.capture_range() - local = edge.start > 0 if older else edge.stop < len(edge.fragments) + edge_cursor = edge.page.before, edge.page.after + demand = window.lookahead.demand + missing = (window.lookahead.ahead_rows(window.outer_size.height) if fresh + else window.lookahead.read_rows(self, older)) + local = edge.prepared.can_extend(older) staged = edge.prepared.for_view() if local else prepared if not local: if loader is None: return False if staged is None: - staged = await self._reader().get(PageRequest( - before=edge.page.before if older else None, - after=edge.page.after if not older else None, - )) - # The viewport owns the bounded runway demand. Publish that range - # together instead of forcing layout/writer joins per viewport. - runway = self._runway() - missing = self.prefetch_distance - (0 if runway is None else runway[0 if older else 1]) + staged = await self._reader().get(PageRequest.adjacent(edge.page, older)) + demand.source_read() + # Speculation warms the same scoped runtime entry as the next + # foreground request; it does not copy decoded pages into a widget. + window.lookahead.demand.prepare_ahead(window.lookahead, self, staged.page, older) boundary = (edge.start if older else edge.stop) if local else (len(staged.fragments) if older else 0) await staged.prepare_rows( self, current, target=self, selected=self.selected_categories, boundary=boundary, older=older, rows_needed=max(1, missing), - limit=self.app.preparation.max_entries, + limit=self.fragment_limit, ) if not current(): return False @@ -1017,16 +1112,12 @@ def current() -> bool: self._require_can_draw() self.state.delivered(self) advanced = self.pages[0] if older else self.pages[-1] - if advanced is edge and edge.capture_range() == page_range: + if (advanced is edge and edge.capture_range() == page_range + and (edge.page.before, edge.page.after) == edge_cursor): return False if fresh or not current(): return False - if not (self.has_older if older else self.has_newer): - return False - if older and self.destination_needs_older: - return True - runway = self._runway() - return runway is not None and runway[0 if older else 1] < self.prefetch_distance + return bool(window.lookahead.read_rows(self, older)) except _DrawingStopped: return False finally: @@ -1047,7 +1138,12 @@ async def _extend_and_trim( previous_start = self.pages[0], self.pages[0].start overlay_visible = self.filter.projection_visible() protected_pages: set[TranscriptPageView] = set() - if local: + if not prepared.fragments: + # A nonmatching interval advances the original source span, + # not the native tree. No empty page widget or decoded cache. + edge.prepared.advance_empty(prepared, older) + await self._page_added(prepared.page, prepared.fragments) + elif local: previous = {id(fragment) for fragment in edge.fragments_in_range} if not await edge.prepared.extend(edge, prepared, current): raise _DrawingStopped @@ -1071,7 +1167,7 @@ async def _extend_and_trim( self.budget.item_limit(len(protected)), self._visible_fragment_budget(), ) excess = self.fragment_count - limit - trim_older = self._follow_source_tail or not older + trim_older = self.follows_source_tail or not older while (excess > 0 or len(self.pages) > limit) and (self.fragment_count > 1 or len(self.pages) > 1): selected = None for side in (trim_older, not trim_older): @@ -1110,8 +1206,7 @@ async def _extend_and_trim( self.filter.canonical_moved(previous_start, overlay_visible) self._update_edges() - -class TranscriptHistory(LineHistory, CommittedHistory): +class TranscriptHistory(CommittedHistory, LineHistory): """The line history of a saved transcript. Its pages stand in for the conversation's live blocks: it reports which @@ -1132,129 +1227,6 @@ def __init__(self, prepared: PreparedTranscriptPage, loader: Callable[..., Await super().__init__(prepared, loader, **kwargs, source_state=LiveTranscript() if committed else ProvisionalTranscript()) - async def park_document(self) -> PreparedPageSource: - """Stop reading and hand the drawn pages to their reader, to be rebuilt when the tab returns.""" - await self.retire_source(parked=True) - source = self._reader() - projection = self.filter.state.overlay - projected = await projection.park_document() if projection is not None else None - pages = tuple(page.prepared for page in self.pages) - position = self.window.reader_position - await source.runtime.run_thread(source.measure_held, pages, projected, - () if position is None else position.ranges) - source.park(pages, projected) - return source - - @classmethod - def from_source(cls, source: PreparedPageSource) -> TranscriptHistory: - """Rebuild a parked history from its pages; it covers no live blocks until it resumes.""" - pages = source.held_pages - if not pages or source.closed: - raise ValueError("No retained transcript admission") - history = cls(pages[0], source.loader) - history.through = source.through - history._page_buffer = source - history._source_state = ParkedSourceTranscript(LiveTranscript()) - history.pages.extend(TranscriptPageView( - page, batch_size=history.budget.batch_items, - ) for page in pages[1:]) - projected = source.projection_source - if projected is not None: - from toad.transcript_filter import Filtered - projection = ProjectedTranscriptHistory.from_source(projected, owner=history) - history.filter.state = Filtered(projection) - return history - - def acquire_document(self) -> None: - """The rebuilt pages are mounted: the reader no longer keeps them for a later rebuild.""" - if self._page_buffer is not None: - self._page_buffer.release_held() - if (projection := self.filter.state.overlay) is not None: - projection.acquire_document() - - def source_resumed(self) -> None: - if self._source_state.reports_coverage: - self.publish_core(TranscriptCoverage(tuple(self.coverage_events))) - - async def _page_added(self, page: TranscriptPage, fragments: tuple[TranscriptFragment, ...]) -> None: - if self._source_state.reports_coverage: - from toad.widgets.conversation import Conversation - # The live blocks this page covers are retired as part of adding - # the page, without holding the conversation's message pump. A - # saved viewer outside a conversation has no live blocks. - for ancestor in self.ancestors: - if isinstance(ancestor, Conversation): - ancestor.transcript.covered(TranscriptCoverage(page.events), self).call_when_ready(ancestor) - break - - def publish_committed(self) -> None: - """Acquire source ownership now; report coverage after its first painted frame.""" - self._source_state = self._source_state.publish() - self.call_after_refresh(self.publish_coverage) - self.request_preparation() - - def publish_coverage(self) -> None: - if self._source_state.reports_coverage: - self.publish_core(TranscriptCoverage(tuple(self.coverage_events))) - - @property - def coverage_events(self) -> Iterator[TranscriptEvent]: - return (event for page in self.pages for event in page.page.events) - - @property - def outgoing_coverage(self): - if self._source_state.reports_coverage: - references = frozenset(reference for page in self.pages - for reference in page.prepared.outgoing_coverage) - projection = self.filter.state.overlay - return references | (projection.outgoing_coverage if projection is not None else frozenset()) - return frozenset() - - def covers_incoming(self, sequence: int) -> bool: - if not self._source_state.reports_coverage: - return False - if self.committed_cursor.covers_incoming(sequence): - return True - if sequence in TranscriptCoverage(tuple(self.coverage_events)).sequences: - return True - return self.filter.covers_incoming(sequence) - - def covers_input(self, native_id: str | None) -> bool: - """Whether the loaded saved pages hold the input Core started under this identity.""" - return (self._source_state.reports_coverage - and native_id in TranscriptCoverage(tuple(self.coverage_events)).native_inputs) - - @property - def committed_cursor(self) -> TranscriptCursor: - return self.through - - def retain_committed(self, through: TranscriptCursor) -> None: - """Extend access to saved source without moving the displayed page window.""" - if not self.accepts_commit(through): - raise ValueError("Commit replaces or rewinds the retained source") - self.extend_source(through) - - async def advance_committed( - self, through: TranscriptCursor, is_current: Callable[[], bool], - ) -> bool: - """Show the new tail as one prepared destination, not a walk through the gap.""" - async with self.window.history_lock: - if not self.checkpoint_available or not is_current(): - return False - # Reject scans started with the old bound without discarding their - # already accepted filtered overlay or its backward cursor. - self.retain_committed(through) - operation = self.reserve_source_work() - return await operation.execute(self, partial(self._advance_committed, is_current)) - - async def _advance_committed(self, is_current: Callable[[], bool]) -> bool: - if not is_current() or not self.state.can_draw: - return False - from toad.widgets.history_anchor import LatestPosition - - request = LatestViewportRequest(LatestPosition.hold(self.window)) - return await self._jump_latest(request, is_current) - class ProjectedTranscriptHistory(TranscriptHistory): """A source projection with the ordinary pager's admission and eviction policy.""" @@ -1285,7 +1257,7 @@ def older_page_available(self) -> bool: def request_older(self) -> None: if self.older_page_available: - self._request_page(True) + self.request_page(True) async def load_older(self) -> None: if self.older_page_available: @@ -1301,7 +1273,7 @@ def selected_categories(self) -> frozenset[type[MessageCategory]]: return all_categories() @property - def _follow_source_tail(self) -> bool: + def follows_source_tail(self) -> bool: # This window is a historical prefix, not the live canonical tail. return False diff --git a/src/toad/widgets/transcript_lines.py b/src/toad/widgets/transcript_lines.py index f650bd61b..a2eaf8617 100644 --- a/src/toad/widgets/transcript_lines.py +++ b/src/toad/widgets/transcript_lines.py @@ -737,9 +737,8 @@ def current_for(self, view) -> bool: return (self.context.current_for(view) and self.preparation.matches(view.line_width, view.app.colours.line_theme)) - def hold(self, view, fragment, lines) -> None: - if self.current_for(view): - fragment.hold_lines(self, lines) + def hold(self, view, fragment, lines) -> bool: + return self.current_for(view) and fragment.hold_lines(self, lines) @staticmethod def show(view, fragment, shown, rendered) -> None: diff --git a/src/toad/widgets/wire_history.py b/src/toad/widgets/wire_history.py index 967c9fccf..3f5455d54 100644 --- a/src/toad/widgets/wire_history.py +++ b/src/toad/widgets/wire_history.py @@ -3,27 +3,26 @@ from __future__ import annotations import asyncio -from collections import deque from dataclasses import replace from typing import TYPE_CHECKING from agent_comms.messages import Message from toad.screens.session_view import SessionView -from toad.transcript_preparation import PageRequest, PreparedPageSource, PreparedTranscriptPage +from toad.transcript_preparation import PreparedTranscriptPage from toad.transcript_source_preparation import HistorySourceSnapshot from toad.transcript_state import LatestViewportRequest -from toad.widgets.history_anchor import HeldReaderDestination, LatestPosition +from toad.widgets.history_anchor import LatestPosition from toad.widgets.message_filter import OtherCategory, all_categories -from toad.widgets.transcript_history import LineHistory, TranscriptPageView +from toad.widgets.message_notifications import VisibleReadMarking +from toad.widgets.transcript_history import LineHistory from toad.message_rows import IrcMessageStyle, WireMessageStyle -from toad.wire_history import AfterSequence, WireFragment, WirePage, WirePageBuffer, WireSource +from toad.wire_history import PreparedWirePage, WirePage, WirePageBuffer, WireSource if TYPE_CHECKING: - from agent_comms.presentation import WireRevision + from agent_comms.historical_views import ConversationRevision - -class WireHistory(LineHistory): +class WireHistory(VisibleReadMarking, LineHistory): """The line history of a channel or direct conversation. Paging, preparation, trimming and the reader's position are the line @@ -32,16 +31,14 @@ class WireHistory(LineHistory): seen, notification states and two styles. """ + page_buffer = WirePageBuffer - def __init__(self, view) -> None: - # The view's message filter is read while the line history is built. + def __init__(self, prepared: PreparedTranscriptPage | None = None, + loader: WireSource | None = None, *, view) -> None: self.view = view - super().__init__(PreparedTranscriptPage.empty(WirePage.unread()), None) - # The store revision and the viewer's reading the drawn pages belong to. - self.revision: WireRevision | None = None - self.identity: tuple | None = None - # A read mark is being written; the next one waits for it. - self.marking = False + super().__init__(PreparedWirePage.empty(WirePage.unread()) if prepared is None else prepared, loader) + # Core's source observation is independent of paint readiness. + self.revision: ConversationRevision | None = None # The source @@ -60,7 +57,7 @@ def style(self) -> WireMessageStyle: @property def undrawn(self) -> bool: """Nothing of the bound source has been drawn yet.""" - return self.revision is None + return all(view.page.undrawn for view in self.pages) @property def source_identity(self): @@ -87,148 +84,47 @@ async def source_is_current(self, snapshot: HistorySourceSnapshot) -> bool: return (await self.bus_is_current(source.comms) and source is self.source and snapshot.current(self)) - def source_failed(self, error) -> None: - self.view.status = f"Wire error: {error}" - - def _reader(self) -> PreparedPageSource: - reader = self._page_buffer - if reader is None or reader.loader is not self.loader: - if reader is not None: - reader.close() - self._page_buffer = reader = WirePageBuffer(self.loader, self.through, self.app.preparation) - # Pages of earlier messages stay prepared while the bound moves. - reader.through = self.through - return reader - @property def selected_categories(self): # Every wire message is in one category: a filter shows all or none. return (all_categories() if OtherCategory in self.view.visible_categories else frozenset()) - def source_resumed(self) -> None: - # Returning to the tab shows its messages again: what is on screen is read. - self.mark_visible() - # Reading the source async def read_changes(self) -> bool: """Draw what changed in the source; False when the stores are as last drawn.""" source = self.source observed = await self.app.preparation.run_thread( - source.observe, self.revision, self.through, self.identity) - if observed is None or source is not self.source: - return False - if observed.replaced: - await self.show_newest(observed.through) - if self.has_newer and self.pages[-1].page.source is None: - # The newest page was not drawn (the reader moved, or the view - # was left): the same stores must be read again. - return False - elif not await self.extend(observed.through): + source.observe, self.revision) + if source is not self.source: return False - self.revision = observed.revision - return True - - async def show_newest(self, through: AfterSequence) -> None: - """Replace what is drawn with the newest page of the source as read now.""" - await self._clear() - self.extend_source(through) + if observed is not None: + # Admit Core's observation before asynchronous drawing. A pending + # viewport must not turn the same scope into another replacement. + self.revision = observed.revision + if observed.replaced: + await self.show_newest(observed.through, PreparedWirePage.empty(WirePage.unread())) + return not self.undrawn + if not self.undrawn: + return await self.extend(observed.through) + self.extend_source(observed.through) + elif not self.undrawn: + # An admitted observation may still have an unfinished page read. + return (self.pages[-1].page.after != self.through + and await self.extend(self.through)) + # The existing prepared reader owns this pending page. Retry admission, + # not clearing its source and starting the same read again. await self._jump_latest(LatestViewportRequest(LatestPosition.hold(self.window))) - source = self.pages[-1].page.source - if source is not None: - self.identity = self.source.kind.display_identity(source) - - async def _clear(self) -> None: - """Forget every drawn page and everything prepared: they belong to another reading.""" - async with self.window.history_lock: - async with self.window.preserve_history(root=self): - self._generation += 1 - if self._page_buffer is not None: - self._page_buffer.close() - self._page_buffer = None - self.revision = self.identity = None - if len(self.pages) == 1 and self.pages[0].page.source is None: - return - retired = tuple(self.pages) - view = TranscriptPageView(PreparedTranscriptPage.empty(WirePage.unread()), - batch_size=self.budget.batch_items) - view.visible_categories = self.selected_categories - await self.mount(view, before=self.newer) - self.pages = deque([view]) - self.through = view.page.after - self.remove_children(retired) - self._update_edges() - - async def extend(self, through: AfterSequence) -> bool: - """The bus grew to `through`: draw this conversation's new messages for a tail reader. - - A reader elsewhere keeps their place and is offered the jump to them. - False when the drawn pages changed during the read; the next refresh - reads again. - """ - if through.cursor > self.through.cursor: - self.through = through - edge = self.pages[-1] - if (edge.page.after != self.through - and (destination := HeldReaderDestination.held(self.window)) is not None - and await destination.source_arrived(self)): - return True - if edge.page.after == self.through: - return True - if self.has_newer: - # The drawn pages stop short of the end: paging from their edge - # reaches the new messages. - if not edge.page.has_newer: - edge.page = replace(edge.page, has_newer=True) - return True - snapshot = self.source_snapshot() - prepared = await self._reader().get(PageRequest(after=edge.page.after)) - # A page read from a bus the route no longer selects is never drawn. - if (not await self.source_is_current(snapshot) - or self.pages[-1] is not edge or self.has_newer): - return False - if (self.identity is not None - and self.source.kind.display_identity(prepared.page.source) != self.identity): - await self.show_newest(self.through) - elif not prepared.fragments: - edge.page = replace(edge.page, after=prepared.page.after) - elif not self.window.follows_tail: - edge.page = replace(edge.page, has_newer=True) - self._update_edges() - else: - await self._load_page(False, prepared, fresh=True) - if self.pages[-1].page.has_newer and self.window.follows_tail: - # More arrived than one page holds: go to the newest instead - # of drawing every page between. - await self._jump_latest(LatestViewportRequest(LatestPosition.hold(self.window))) - return True - - async def _extend_and_trim(self, edge, older, local, protected, prepared, - *, current, fresh=False): - if not local and not prepared.fragments: - # An empty wire interval advances its cut, not native membership. - if not current(): - return - page = prepared.page - edge.page = (replace(edge.page, before=page.before, has_older=page.has_older) if older - else replace(edge.page, after=page.after, has_newer=page.has_newer)) - self._update_edges() - return - await super()._extend_and_trim(edge, older, local, protected, prepared, - current=current, fresh=fresh) - - def prepare_viewport(self) -> None: - super().prepare_viewport() - if self.can_draw_on_screen: - # The shared pager already checked feedback; wire pages also mark paint. - self.mark_visible() + return not self.undrawn # What is drawn - @property - def wire_fragments(self) -> tuple[WireFragment, ...]: - return tuple(fragment for page in self.pages for fragment in page.fragments) + async def accept_extension(self, edge, prepared) -> bool: + if edge.page.accepts(prepared.page.source, self.source.kind.history_view): + return True + await self.show_newest(self.through, PreparedWirePage.empty(WirePage.unread())) + return False @property def messages(self) -> tuple[Message, ...]: @@ -239,14 +135,6 @@ def draws(self, message: Message) -> bool: """Whether the drawn rows include this message.""" return any(drawn.view_key == message.view_key for drawn in self.messages) - def read_fragments(self) -> tuple[WireFragment, ...]: - """Fragments whose own text has a row in the viewport.""" - if not self.is_attached or not self.can_draw_on_screen: - return () - return tuple( - fragment for page in self.pages for fragment, rows, lines in page.visible_rows(self.window) - if (text := fragment.text_rows(lines, self.app.colours.line_theme)) and rows.start < text.stop and text.start < rows.stop) - async def toggle_style(self) -> None: """Draw every message in the other style, here and in pages read from now on.""" if self.source is None or not self.can_draw_on_screen: @@ -254,7 +142,7 @@ async def toggle_style(self) -> None: style = self.style.next() self.open(self.source.restyled(style)) await self._show([(fragment, replace(fragment.shown, style=style)) - for fragment in self.wire_fragments]) + for fragment in self.fragments]) self.window.scroll_end(animate=False) def notification_read(self, references): @@ -264,36 +152,29 @@ def notification_read(self, references): # Read marks - def mark_visible(self) -> None: - """Mark read the messages whose text is on screen, when Core's page evidence allows.""" - if self.marking or not self.is_attached or self.source is None: - return + def read_visible(self): + if not self.is_attached or self.source is None: + return None if not any(fragment.evidence is not None for fragment in self.read_fragments()): - return - self.marking = True - self.view.run_worker(self._mark_read(self.source_snapshot()), group="comms-painted-read") + return None + return self._mark_read(self.source_snapshot()) async def _mark_read(self, snapshot: HistorySourceSnapshot) -> None: - try: - if not await self.source_is_current(snapshot): - if snapshot.current(self): - self.view.display = False - elif self.is_attached: - # The history was restyled or redrawn while this waited: - # what is on screen now is marked from its own snapshot. - self.view.call_after_refresh(self.mark_visible) - return - seen = tuple(fragment for fragment in self.read_fragments() if fragment.evidence is not None) - historical = next((fragment.evidence for fragment in seen - if fragment.evidence.historical_display is not None), None) - if historical is not None: - await self._mark_historical( - historical, tuple(fragment for fragment in seen if fragment.evidence is historical), - snapshot) - elif (selected := self.source.kind.seen_page(self, seen)) is not None: - await self._mark_page(*selected, snapshot) - finally: - self.marking = False + if not await self.source_is_current(snapshot): + if snapshot.current(self): + self.view.display = False + elif self.is_attached: + self.view.call_after_refresh(self.mark_visible) + return + seen = tuple(fragment for fragment in self.read_fragments() if fragment.evidence is not None) + historical = next((fragment.evidence for fragment in seen + if fragment.evidence.historical_display is not None), None) + if historical is not None: + await self._mark_historical( + historical, tuple(fragment for fragment in seen if fragment.evidence is historical), + snapshot) + elif (selected := self.source.kind.seen_page(self, seen)) is not None: + await self._mark_page(*selected, snapshot) async def _mark_historical(self, page, marked, snapshot: HistorySourceSnapshot) -> None: from agent_comms.route_selection import implicit_root, run_selected_write @@ -306,7 +187,7 @@ async def _mark_historical(self, page, marked, snapshot: HistorySourceSnapshot) implicit=implicit_root()) except ValueError: # The attached history changed; this page's evidence marks nothing any more. - marked = tuple(fragment for fragment in self.wire_fragments if fragment.evidence is page) + marked = tuple(fragment for fragment in self.fragments if fragment.evidence is page) if snapshot.current(self): for fragment in marked: fragment.evidence = None @@ -322,7 +203,7 @@ async def _mark_page(self, page, marked, snapshot: HistorySourceSnapshot) -> Non # The viewer, peer, channel scope or bus changed after the page # was read: draw the conversation as it is read now. if snapshot.current(self): - await self._clear() + await self._clear(PreparedWirePage.empty(WirePage.unread())) await self.view._refresh() finally: if self.is_attached: diff --git a/src/toad/wire_history.py b/src/toad/wire_history.py index 8737f0013..876039f05 100644 --- a/src/toad/wire_history.py +++ b/src/toad/wire_history.py @@ -17,7 +17,7 @@ from agent_comms.comms import Comms from agent_comms.message_page import MessagePage from agent_comms.messages import Message -from agent_comms.presentation import WireRevision +from agent_comms.historical_views import ConversationHistoryView, ConversationRevision from toad.conversation_kind import ConversationKind from toad.line_blocks import Body, Gap, LineBlock, MutedRole, Summary @@ -98,6 +98,33 @@ def unread(cls) -> WirePage: """What is drawn before the first read: nothing, up to sequence zero.""" return cls((), None, AfterSequence(0), False, False, None) + @property + def undrawn(self) -> bool: + return self.source is None + + def accepts(self, page: MessagePage, view: type[ConversationHistoryView]) -> bool: + """An initial placeholder admits its first scope; Core compares actual pages.""" + source = self.source + return source is None or view.same_display(page, source) + + def advanced(self, interval: WirePage, *, older: bool) -> WirePage: + """An empty interval advances its cut and establishes the first Core page. + + Existing fragments retain their original page evidence; a placeholder + must adopt the actual empty Core page rather than remain unobserved. + """ + source = interval.source if self.undrawn else self.source + return (replace(self, before=interval.before, has_older=interval.has_older, source=source) + if older else replace(self, after=interval.after, has_newer=interval.has_newer, + source=source)) + + +class PreparedWirePage(PreparedTranscriptPage): + """Wire intervals preserve Core's original display page when advanced empty.""" + + def advance_empty(self, prepared: PreparedTranscriptPage, older: bool) -> None: + self.page = self.page.advanced(prepared.page, older=older) + @dataclass(kw_only=True) class WireFragment(TranscriptFragment): @@ -215,19 +242,16 @@ def page(self, **bounds) -> MessagePage: return self.kind.page(self.comms, self.target, worktree=str(self.project), max_bytes=self.max_bytes, **bounds) - def observe(self, seen: WireRevision | None, through: AfterSequence, - identity: tuple | None) -> WireObservation | None: + def observe(self, seen: ConversationRevision | None) -> WireObservation | None: """What changed since `seen`, or None when the stores are as they were.""" - revision = self.comms.views.revision() + revision = self.kind.history_view.revision( + self.comms.views, self.target, worktree=str(self.project)) if revision == seen: return None high = AfterSequence(self.comms.views.latest_sequence()) - # Without new messages the viewer, peer or channel scope may still - # have been replaced: the pages drawn then belong to another reading. - replaced = seen is None or ( - not high.cursor > through.cursor and identity is not None - and revision.stores_changed_since(seen) - and self.kind.display_identity(self.page(limit=1)) != identity) + # Core's display view owns which stores can replace this reading. + # Notification/read/activity writes do not require a page to learn that. + replaced = seen is None or revision.display_changed_since(seen) return WireObservation(revision, high, replaced) def read(self, request: PageRequest, through: AfterSequence) -> PreparedTranscriptPage: @@ -243,7 +267,7 @@ def read(self, request: PageRequest, through: AfterSequence) -> PreparedTranscri if message.view_key[0] or message.seq <= through.cursor) if len(kept) < len(page.messages): page = replace(page, messages=kept, has_newer=False) - evidence = page if self.kind.display_identity(page) is not None else None + evidence = page if self.kind.history_view.has_display_evidence(page) else None shown = MessageShown(style=self.style) fragments = tuple( WireFragment.of(message, shown=shown, evidence=evidence, @@ -256,7 +280,7 @@ def read(self, request: PageRequest, through: AfterSequence) -> PreparedTranscri AfterMessage(page.newest_cursor) if page.has_newer and page.messages else through, page.has_older and bool(page.messages), page.has_newer and bool(page.messages), page, ) - return PreparedTranscriptPage.measured(wire, fragments) + return PreparedWirePage.measured(wire, fragments) def restyled(self, style: WireMessageStyle) -> WireSource: return replace(self, style=style) @@ -264,7 +288,7 @@ def restyled(self, style: WireMessageStyle) -> WireSource: @dataclass(frozen=True) class WireObservation: - revision: WireRevision + revision: ConversationRevision through: AfterSequence replaced: bool @@ -295,3 +319,7 @@ class WirePageBuffer(TranscriptPageBuffer): """The line history's page reader over a wire source.""" work = WirePageWork + + def accepts(self, loader, through) -> bool: + # Earlier wire cuts stay valid while the bus grows. + return self.loader is loader diff --git a/src/toad/workspace_chrome.py b/src/toad/workspace_chrome.py index 9274b8416..a3e67e299 100644 --- a/src/toad/workspace_chrome.py +++ b/src/toad/workspace_chrome.py @@ -6,23 +6,13 @@ from toad.widgets.channels_sidebar import ChannelsSidebar from textual.widgets import Footer from toad.session_presentation import NativeSessionSurface -from toad.core_event_carrier import CoreEventReceiver, CoreEventMessage -from toad.core.events import SidebarLayoutChanged -from agent_comms.mro_dispatch import handles if TYPE_CHECKING: from toad.app import ToadApp from toad.screens.session_view import SessionView -class WorkspaceHeader(CoreEventReceiver, Horizontal): - def on_mount(self) -> None: - self.observe_core(self.app.events) - - @handles(SidebarLayoutChanged) - async def sidebar_layout_changed(self, event: CoreEventMessage) -> None: - self.app.workspace_chrome.layout_sidebars(self.screen) - +class WorkspaceHeader(Horizontal): def compose(self) -> ComposeResult: from toad.widgets.session_tabs import SessionsTabs from toad.widgets.side_bar import TabHistoryControls diff --git a/tests/support/textual_steps.py b/tests/support/textual_steps.py index b517c7b05..2daa4b860 100644 --- a/tests/support/textual_steps.py +++ b/tests/support/textual_steps.py @@ -9,6 +9,7 @@ from contextlib import asynccontextmanager from pathlib import Path from unittest.mock import patch +from agent_comms.transcript_sources import NativeCursor, TranscriptCursor async def reveal_project_tree(ui): @@ -141,7 +142,7 @@ async def replay_with_pending_render(workspace, events=None): Yields (ui, owner tab, replay tab, release); `release()` lets the render finish. """ from agent_comms.transcript_events import AssistantTranscript - from agent_comms.transcripts import TranscriptCursor, TranscriptPage + from agent_comms.transcripts import TranscriptPage from support.agents import peer from toad.acp.agent import Agent from toad.core.events import AgentReady @@ -152,7 +153,7 @@ async def replay_with_pending_render(workspace, events=None): if events is None: events = tuple(AssistantTranscript("Paragraph.\n\n" * 100) for _ in range(20)) events += (AssistantTranscript("PREPARED_REPLAY_END"),) - page = TranscriptPage(events, TranscriptCursor("fixture", 0), TranscriptCursor("fixture", len(events)), + page = TranscriptPage(events, NativeCursor("fixture", 0), NativeCursor("fixture", len(events)), False, False) dispatch, entered, released = asyncio.Event(), asyncio.Event(), asyncio.Event() shown = [] diff --git a/tests/textual/test_block_click_scroll.py b/tests/textual/test_block_click_scroll.py index a5cc9b962..7fa9aa8ea 100644 --- a/tests/textual/test_block_click_scroll.py +++ b/tests/textual/test_block_click_scroll.py @@ -1,7 +1,8 @@ """Textual-facing: where the scroll position goes when a block is clicked (the block cursor's scroll is the frontend's).""" from agent_comms.transcript_events import AssistantTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.agent_response import AgentResponse from toad.widgets.note import Note @@ -45,7 +46,7 @@ async def test_clicking_a_block_that_fits_the_viewport_does_not_scroll(workspace async def test_clicking_saved_history_does_not_scroll(workspace): - cursor = TranscriptCursor("fixture", 1) + cursor = NativeCursor("fixture", 1) page = TranscriptPage(tuple(AssistantTranscript(f"RECORD_{i} " + "word " * 60) for i in range(60)), cursor, cursor, False, False) async with workspace.app(size=(110, 38)) as ui: diff --git a/tests/textual/test_dense_history.py b/tests/textual/test_dense_history.py index 4586a0fbc..d39926552 100644 --- a/tests/textual/test_dense_history.py +++ b/tests/textual/test_dense_history.py @@ -6,13 +6,14 @@ """ from agent_comms.transcript_events import AssistantTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.transcript_history import TranscriptHistory async def test_each_scroll_to_the_top_admits_earlier_records_and_until_the_first_record_is_reached(workspace): - cursor = TranscriptCursor("fixture", 1) + cursor = NativeCursor("fixture", 1) page = TranscriptPage(tuple( AssistantTranscript(f"Record {i}\n\n" + "\n".join(f"- item {j}" for j in range(20))) for i in range(30)), cursor, cursor, False, False) diff --git a/tests/textual/test_filtered_history_underfill.py b/tests/textual/test_filtered_history_underfill.py index 49b7e4622..11af92d6a 100644 --- a/tests/textual/test_filtered_history_underfill.py +++ b/tests/textual/test_filtered_history_underfill.py @@ -8,7 +8,8 @@ from agent_comms.message_reference import MessageReference from agent_comms.routing import MessageRoute, TurnRouting from agent_comms.transcript_events import SentTranscript, ThinkingTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.message_filter import MessageCategory, RoutedMessage, all_categories from toad.widgets.transcript_history import TranscriptHistory @@ -21,7 +22,7 @@ def hidden_tail(count: int, name: str = "HIDDEN_TAIL"): return TranscriptPage(tuple(ThinkingTranscript(f"{name}_{index}") for index in range(count)), - TranscriptCursor(FILE, 100), TranscriptCursor(FILE, 200), True, False) + NativeCursor(FILE, 100), NativeCursor(FILE, 200), True, False) def routed_shown(history) -> bool: @@ -46,7 +47,7 @@ async def test_routed_filter_reads_an_older_page_to_fill_an_empty_viewport(works async def load_page(**kwargs): calls.append(kwargs) - return TranscriptPage((OLDER,), TranscriptCursor(FILE, 0), TranscriptCursor(FILE, 100), False, True) + return TranscriptPage((OLDER,), NativeCursor(FILE, 0), NativeCursor(FILE, 100), False, True) tail = hidden_tail(90) history = await mount_then_filter(ui, tail, load_page) @@ -64,7 +65,7 @@ async def test_filter_over_more_hidden_records_than_the_budget_still_finds_the_r async def load_page(**kwargs): calls.append(kwargs) - return TranscriptPage((OLDER,), TranscriptCursor(FILE, 0), TranscriptCursor(FILE, 100), False, True) + return TranscriptPage((OLDER,), NativeCursor(FILE, 0), NativeCursor(FILE, 100), False, True) sparse = await mount_then_filter(ui, hidden_tail(340, "HIDDEN_OLDER"), load_page) await ui.until(lambda: routed_shown(sparse), why="the older routed message past 340 hidden records") @@ -78,7 +79,7 @@ async def test_clearing_the_filter_restores_the_transcript_and_setting_it_again_ conversation = ui.conversation async def load_page(**kwargs): - return TranscriptPage((OLDER,), TranscriptCursor(FILE, 0), TranscriptCursor(FILE, 100), False, True) + return TranscriptPage((OLDER,), NativeCursor(FILE, 0), NativeCursor(FILE, 100), False, True) sparse = await mount_then_filter(ui, hidden_tail(340, "HIDDEN_OLDER"), load_page) await ui.until(lambda: routed_shown(sparse), why="the routed message under the filter") @@ -97,8 +98,8 @@ async def test_history_with_no_routed_records_is_scanned_to_its_end_once(workspa async def only_hidden(**kwargs): reads.append(kwargs["before"].offset) offset = 50 if kwargs["before"].offset == 100 else 0 - return TranscriptPage((ThinkingTranscript("NO_ROUTE"),), TranscriptCursor(FILE, offset), - TranscriptCursor(FILE, kwargs["before"].offset), bool(offset), True) + return TranscriptPage((ThinkingTranscript("NO_ROUTE"),), NativeCursor(FILE, offset), + NativeCursor(FILE, kwargs["before"].offset), bool(offset), True) empty = await mount_then_filter(ui, hidden_tail(90), only_hidden) await ui.until(lambda: empty.filter.before is not None and not empty.filter.has_older, diff --git a/tests/textual/test_fragment_dividers.py b/tests/textual/test_fragment_dividers.py index e49008a4a..b19d78457 100644 --- a/tests/textual/test_fragment_dividers.py +++ b/tests/textual/test_fragment_dividers.py @@ -4,7 +4,8 @@ from agent_comms.messages import Message, MessageType from agent_comms.routing import MessageRoute, TurnRouting from agent_comms.transcript_events import IncomingTranscript, NoticeTranscript, SentTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.line_blocks import Divider from toad.widgets.agent_response import AgentResponse from toad.widgets.message_divider import MessageDivider @@ -12,7 +13,7 @@ TEXT = "## Context compacted\n\n" + "\n\n".join( f"### Section {index}\n" + "A retained fact. " * 38 for index in range(8)) -CURSOR = TranscriptCursor("", 0) +CURSOR = NativeCursor("", 0) def dividers_in(event): diff --git a/tests/textual/test_history_admission_budget.py b/tests/textual/test_history_admission_budget.py index 93a707d94..6069110f9 100644 --- a/tests/textual/test_history_admission_budget.py +++ b/tests/textual/test_history_admission_budget.py @@ -7,14 +7,15 @@ from unittest.mock import patch from agent_comms.transcript_events import AssistantTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.transcript_history import TranscriptHistory async def test_admitting_earlier_pages_repeatedly_keeps_admission_bounded_and_the_source_untruncated(workspace): events = tuple(AssistantTranscript(f"Record {index}\n\n" + "long paragraph words " * 28) for index in range(120)) - cursor = TranscriptCursor("fixture", 0) + cursor = NativeCursor("fixture", 0) async with workspace.app(size=(150, 80)) as ui: history = TranscriptHistory(TranscriptPage(events, cursor, cursor, False, False)) await ui.conversation.contents.mount(history) diff --git a/tests/textual/test_history_prefetch.py b/tests/textual/test_history_prefetch.py index cad7a679d..2f2663d91 100644 --- a/tests/textual/test_history_prefetch.py +++ b/tests/textual/test_history_prefetch.py @@ -7,7 +7,8 @@ from agent_comms.message_reference import MessageReference from agent_comms.routing import MessageRoute, TurnRouting from agent_comms.transcript_events import AssistantTranscript, SentTranscript, ThinkingTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.message_filter import MessageCategory, RoutedMessage from toad.widgets.transcript_history import TranscriptHistory @@ -23,11 +24,11 @@ async def test_older_page_is_read_before_the_viewport_reaches_the_top(workspace) async def loader(**kwargs): calls.append((window.scroll_y, kwargs)) return TranscriptPage((AssistantTranscript("OLDER_PREFETCHED_RECORD"),), - TranscriptCursor(FILE, 0), TranscriptCursor(FILE, 100), False, True) + NativeCursor(FILE, 0), NativeCursor(FILE, 100), False, True) tail = tuple(AssistantTranscript(f"Record {i}\n\n" + "Long line\n" * 5) for i in range(64)) history = TranscriptHistory(TranscriptPage( - tail, TranscriptCursor(FILE, 100), TranscriptCursor(FILE, 200), True, False), loader=loader) + tail, NativeCursor(FILE, 100), NativeCursor(FILE, 200), True, False), loader=loader) await ui.conversation.contents.mount(history) window.anchor() await ui.until(lambda: window.max_scroll_y > history.prefetch_distance, why="a scrollable history") @@ -54,13 +55,13 @@ async def loader(**kwargs): calls.append(window.scroll_y) return TranscriptPage((SentTranscript("FILTERED_PREFETCH", routing=route, source=MessageReference(2, "prefetch-filtered")),), - TranscriptCursor(FILE, 0), TranscriptCursor(FILE, 100), False, True) + NativeCursor(FILE, 0), NativeCursor(FILE, 100), False, True) tail = (*(ThinkingTranscript(f"HIDDEN_{index}") for index in range(90)), SentTranscript("VISIBLE_ROUTE\n\n" + "Long routed line\n\n" * 150, routing=route, source=MessageReference(1, "prefetch-visible"))) history = TranscriptHistory(TranscriptPage( - tail, TranscriptCursor(FILE, 100), TranscriptCursor(FILE, 200), True, False), loader=loader) + tail, NativeCursor(FILE, 100), NativeCursor(FILE, 200), True, False), loader=loader) await ui.conversation.contents.mount(history) window.anchor() await ui.until(lambda: window.max_scroll_y > history.prefetch_distance, why="a scrollable history") diff --git a/tests/textual/test_history_scroll_frames.py b/tests/textual/test_history_scroll_frames.py index 26a36c08e..9829a6e1a 100644 --- a/tests/textual/test_history_scroll_frames.py +++ b/tests/textual/test_history_scroll_frames.py @@ -10,7 +10,8 @@ from unittest.mock import patch from agent_comms.transcript_events import AssistantTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from support.textual_steps import refresh_chat from toad.transcript_preparation import PreparedTranscriptPage @@ -36,7 +37,7 @@ def record(screen, renderable): async def test_transcript_paging_keeps_the_reader_row_on_every_frame_even_with_input_during_the_read(workspace): - cursor = TranscriptCursor("fixture", 1) + cursor = NativeCursor("fixture", 1) page = TranscriptPage(tuple( AssistantTranscript(f"Record {i}\n\n" + "\n".join(f"- item {j}" for j in range(20))) for i in range(30)), cursor, cursor, False, False) @@ -90,7 +91,7 @@ async def delayed_extend(prepared, *args, **kwargs): async def test_restyled_lines_keep_the_line_the_reader_is_on_in_every_frame(workspace): """A theme change draws every line plain first and styled when ready; the reader's line stays on its row.""" - cursor = TranscriptCursor("fixture", 1) + cursor = NativeCursor("fixture", 1) text = "## Saved heading\n\n" + "\n".join(f"- saved row {row}" for row in range(120)) page = TranscriptPage((AssistantTranscript(text),), cursor, cursor, False, False) async with workspace.app(size=(110, 38)) as ui: @@ -248,7 +249,7 @@ async def roster_waits(cls, view, comms): async def test_two_screens_back_in_a_thread_of_short_records_are_refilled_by_a_few_loads(workspace): - cursor = TranscriptCursor("fixture", 1) + cursor = NativeCursor("fixture", 1) page = TranscriptPage(tuple(AssistantTranscript(f"short {i}") for i in range(400)), cursor, cursor, False, False) loads, counting = count_earlier_loads() with counting: diff --git a/tests/textual/test_message_cursor_on_line_rows.py b/tests/textual/test_message_cursor_on_line_rows.py index 7229de61b..0be861eaf 100644 --- a/tests/textual/test_message_cursor_on_line_rows.py +++ b/tests/textual/test_message_cursor_on_line_rows.py @@ -1,7 +1,8 @@ """Textual-facing: messages drawn as rows sit on their view's background, and the block cursor rests on one of them: Alt+Up/Down move it message by message, its menu copies the message and (in a channel) opens the sender or destination, and hovering shows the message's full time.""" from agent_comms.transcript_events import AssistantTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.menu import Menu from toad.widgets.message_divider import MessageClock @@ -95,7 +96,7 @@ async def test_channel_rows_have_the_view_background_and_the_cursor_acts_on_one_ async def test_saved_thread_history_rows_have_the_view_background_and_the_cursor_copies_one_message(workspace): - cursor = TranscriptCursor("fixture", 1) + cursor = NativeCursor("fixture", 1) page = TranscriptPage(tuple(AssistantTranscript(f"RECORD_{i} in *Markdown*") for i in range(6)), cursor, cursor, False, False) async with workspace.app(size=(110, 38)) as ui: diff --git a/tests/textual/test_message_filter_late_page.py b/tests/textual/test_message_filter_late_page.py index 38f87a4c9..65411fca8 100644 --- a/tests/textual/test_message_filter_late_page.py +++ b/tests/textual/test_message_filter_late_page.py @@ -5,7 +5,8 @@ from agent_comms.messages import Message, MessageType from agent_comms.routing import MessageRoute from agent_comms.transcript_events import AssistantTranscript, IncomingTranscript, ThinkingTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.message_filter import InboundCategory, ThinkingCategory from toad.widgets.transcript_history import TranscriptHistory @@ -15,7 +16,7 @@ async def test_late_older_page_from_a_previous_filter_is_not_shown(workspace): app, _ = ui.textual view = ui.conversation file = "fixture" - cursor = TranscriptCursor(file, 100) + cursor = NativeCursor(file, 100) entered, release = asyncio.Event(), asyncio.Event() message = Message("peer", "owner", "OLD_INBOUND", MessageType.INFO) incoming = IncomingTranscript(message.body, route=MessageRoute(message.sender, (message.target,)), @@ -27,10 +28,10 @@ async def older(**kwargs): calls.append(kwargs) entered.set() await release.wait() - return TranscriptPage((incoming, thinking), TranscriptCursor(file, 0), cursor, False, True) + return TranscriptPage((incoming, thinking), NativeCursor(file, 0), cursor, False, True) tail = tuple(AssistantTranscript(f"unselected {i}\n" + "long text " * 20) for i in range(24)) - pager = TranscriptHistory(TranscriptPage(tail, cursor, TranscriptCursor(file, 200), True, False), + pager = TranscriptHistory(TranscriptPage(tail, cursor, NativeCursor(file, 200), True, False), loader=older) try: await view.contents.mount(pager) diff --git a/tests/textual/test_message_filter_switch_while_mounting.py b/tests/textual/test_message_filter_switch_while_mounting.py index e3872155a..cb52a25c6 100644 --- a/tests/textual/test_message_filter_switch_while_mounting.py +++ b/tests/textual/test_message_filter_switch_while_mounting.py @@ -9,7 +9,8 @@ from agent_comms.transcript_events import ( AssistantTranscript, IncomingTranscript, ThinkingTranscript, ) -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from textual.await_complete import AwaitComplete from textual.containers import VerticalGroup @@ -39,7 +40,7 @@ async def test_filter_change_during_mount_retires_the_old_projection(workspace, events = (incoming("OLD_INBOUND"), ThinkingTranscript("NEW_THINKING")) * 8 + tuple( AssistantTranscript(f"tail {i}") for i in range(4)) pager = TranscriptHistory(TranscriptPage( - events, TranscriptCursor("mount-fixture", 0), TranscriptCursor("mount-fixture", len(events)), + events, NativeCursor("mount-fixture", 0), NativeCursor("mount-fixture", len(events)), False, False)) entered, release = asyncio.Event(), asyncio.Event() original_mount = VerticalGroup.mount @@ -119,8 +120,8 @@ async def test_filtered_history_arrives_in_batches_no_larger_than_the_rows_kept_ events = tuple(event for index in range(MATCHES) for event in ( incoming(f"INBOUND_{index}"), ThinkingTranscript(f"THINKING_{index}"), )) + tuple(AssistantTranscript(f"tail {index}") for index in range(4)) - page = TranscriptPage(events, TranscriptCursor("batch-fixture", 0), - TranscriptCursor("batch-fixture", len(events)), False, False) + page = TranscriptPage(events, NativeCursor("batch-fixture", 0), + NativeCursor("batch-fixture", len(events)), False, False) pager = TranscriptHistory(page) threads = [] prepare = TranscriptFilterWork.prepare @@ -157,16 +158,16 @@ async def test_empty_filter_reads_no_history_and_a_later_selection_finds_the_mat async with workspace.app(size=(120, 40)) as ui: view = ui.conversation view.visible_categories = frozenset() - cursor = TranscriptCursor("empty-filter", 10) + cursor = NativeCursor("empty-filter", 10) calls = [] async def earlier(**kwargs): calls.append(kwargs) - return TranscriptPage((incoming("SELECTED_INBOUND"),), TranscriptCursor("empty-filter", 0), cursor, + return TranscriptPage((incoming("SELECTED_INBOUND"),), NativeCursor("empty-filter", 0), cursor, False, True) pager = TranscriptHistory(TranscriptPage((ThinkingTranscript("TAIL"),), - cursor, TranscriptCursor("empty-filter", 20), True, False), + cursor, NativeCursor("empty-filter", 20), True, False), loader=earlier) await view.contents.mount(pager) await ui.step() diff --git a/tests/textual/test_message_menu_on_every_row.py b/tests/textual/test_message_menu_on_every_row.py index 7167569da..053da6e4b 100644 --- a/tests/textual/test_message_menu_on_every_row.py +++ b/tests/textual/test_message_menu_on_every_row.py @@ -3,7 +3,8 @@ from agent_comms.message_reference import MessageReference from agent_comms.routing import MessageRoute, TurnRouting from agent_comms.transcript_events import IncomingTranscript, SentTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.response_delivery import RoutedResponse from toad.widgets.agent_response import AgentResponse @@ -37,7 +38,7 @@ async def test_enter_on_a_saved_a_sent_and_a_routed_live_message_opens_its_sende workspace.thread("owner", "team", running=True) workspace.thread("peer", "team", running=True) wire_root = str(workspace.roots.wire) - cursor = TranscriptCursor("fixture", 1) + cursor = NativeCursor("fixture", 1) page = TranscriptPage(( IncomingTranscript("SAVED_INCOMING", route=MessageRoute("peer", ("owner",)), timestamp=1_700_000_000.0, source=MessageReference(1, "incoming-1")), diff --git a/tests/textual/test_message_names_open_from_a_thread.py b/tests/textual/test_message_names_open_from_a_thread.py index d57b1b3d2..e34501fbc 100644 --- a/tests/textual/test_message_names_open_from_a_thread.py +++ b/tests/textual/test_message_names_open_from_a_thread.py @@ -3,7 +3,8 @@ from agent_comms.message_reference import MessageReference from agent_comms.routing import MessageRoute, TurnRouting from agent_comms.transcript_events import IncomingTranscript, SentTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.line_blocks import OpenTarget from toad.widgets.comms_chat import CommsChatView @@ -20,7 +21,7 @@ async def test_names_above_saved_and_live_messages_open_the_thread_and_the_chann workspace.thread("owner", "team", running=True) workspace.thread("peer", "team", running=True) wire_root = str(workspace.roots.wire) - cursor = TranscriptCursor("fixture", 1) + cursor = NativeCursor("fixture", 1) page = TranscriptPage(( IncomingTranscript("SAVED_INCOMING", route=MessageRoute("peer", ("owner",)), timestamp=1_700_000_000.0, source=MessageReference(1, "incoming-1")), diff --git a/tests/textual/test_notice_link_in_channel.py b/tests/textual/test_notice_link_in_channel.py index ded6ddae1..ac6866c29 100644 --- a/tests/textual/test_notice_link_in_channel.py +++ b/tests/textual/test_notice_link_in_channel.py @@ -4,7 +4,8 @@ from agent_comms.messages import MessageType from agent_comms.routing import MessageRoute from agent_comms.transcript_events import IncomingTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.line_blocks import OpenUrl from toad.widgets.project_panel import FilePreview @@ -24,7 +25,7 @@ async def test_a_notice_link_to_a_file_is_drawn_as_its_label_and_opens_the_file( diagnostic.write_text('{"failure": "DIAGNOSTIC_CONTENT"}\n') text = f"Request failed. [Open diagnostic]({diagnostic.as_uri()})" workspace.comms.messaging.send("peer", "#team", text, MessageType.ALERT, notice=True) - cursor = TranscriptCursor("fixture", 1) + cursor = NativeCursor("fixture", 1) saved = TranscriptPage((IncomingTranscript(text, route=MessageRoute("peer", ("owner",)), timestamp=1_700_000_000.0, source=MessageReference(1, "incoming-1")),), cursor, cursor, False, False) async with workspace.app(size=(120, 36)) as ui: diff --git a/tests/textual/test_projected_history_budget.py b/tests/textual/test_projected_history_budget.py index cbd450384..85a273227 100644 --- a/tests/textual/test_projected_history_budget.py +++ b/tests/textual/test_projected_history_budget.py @@ -9,7 +9,8 @@ import pytest from agent_comms.transcript_events import AssistantTranscript, ThinkingTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.message_filter import ThinkingCategory, all_categories from toad.widgets.presentation_window import PresentationBudget @@ -39,8 +40,8 @@ async def load(*, before=None, after=None, through=None): else: stop = min(limit, before.offset if before is not None else limit) start = max(0, stop - 32) - return TranscriptPage(events[start:stop], TranscriptCursor(identity, start), - TranscriptCursor(identity, stop), start > 0, stop < limit) + return TranscriptPage(events[start:stop], NativeCursor(identity, start), + NativeCursor(identity, stop), start > 0, stop < limit) async with workspace.app(size=(100, 26)) as ui: view = ui.conversation diff --git a/tests/textual/test_reader_geometry.py b/tests/textual/test_reader_geometry.py index 39c579cf6..c2d9dc01e 100644 --- a/tests/textual/test_reader_geometry.py +++ b/tests/textual/test_reader_geometry.py @@ -1,7 +1,8 @@ """Textual-facing: the history window keeps a reader's position through a taller terminal and rejoins the tail only on real reader input.""" from agent_comms.transcript_events import AssistantTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.presentation_window import PresentationBudget from toad.widgets.transcript_history import TranscriptHistory @@ -11,7 +12,7 @@ async def test_reader_keeps_position_through_geometry_and_rejoins_the_tail_on_in async with workspace.app(size=(100, 35)) as ui: app, pilot = ui.textual conversation = ui.conversation - cursor = TranscriptCursor("loaded-history", 1) + cursor = NativeCursor("loaded-history", 1) page = TranscriptPage(tuple( AssistantTranscript(f"Saved record {i}\n\n" + "\n".join(f"- saved item {j}" for j in range(20))) for i in range(8)), cursor, cursor, False, False) diff --git a/tests/textual/test_reader_position.py b/tests/textual/test_reader_position.py index 7a6eaf862..5fe04cce4 100644 --- a/tests/textual/test_reader_position.py +++ b/tests/textual/test_reader_position.py @@ -23,7 +23,8 @@ import pytest from agent_comms.transcript_events import AssistantTranscript, ThinkingTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.agent_response import AgentResponse from toad.widgets.message_filter import AgentCategory, ThinkingCategory, all_categories @@ -53,7 +54,7 @@ def page(number: int) -> TranscriptPage: AssistantTranscript(f"ANSWER {number}.{index}\n\n" + "\n".join( f"- item {number}.{index}.{row}" for row in range(5)) + wide) for index in range(PER_PAGE)) - return TranscriptPage(events, TranscriptCursor(FILE, number * 100), TranscriptCursor(FILE, (number + 1) * 100), + return TranscriptPage(events, NativeCursor(FILE, number * 100), NativeCursor(FILE, (number + 1) * 100), number > 0, number < PAGES - 1) diff --git a/tests/textual/test_reflow_is_not_reader_travel.py b/tests/textual/test_reflow_is_not_reader_travel.py index 97fdb42be..7573c00ef 100644 --- a/tests/textual/test_reflow_is_not_reader_travel.py +++ b/tests/textual/test_reflow_is_not_reader_travel.py @@ -8,7 +8,8 @@ from dataclasses import replace from agent_comms.transcript_events import AssistantTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.transcript_history import TranscriptHistory @@ -21,7 +22,7 @@ async def test_trim_and_resize_reflow_are_not_travel_samples_but_a_key_is(worksp window.presentation_budget = replace(window.presentation_budget, scroll_idle_seconds=5) histories = [] for index in range(3): - cursor = TranscriptCursor(f"saved-{index}", 1) + cursor = NativeCursor(f"saved-{index}", 1) text = f"## SAVED SOURCE {index}\n\n" + "\n".join(f"- saved record {index}, row {row}" for row in range(120)) history = TranscriptHistory(TranscriptPage((AssistantTranscript(text),), cursor, cursor, False, False)) histories.append(history) diff --git a/tests/textual/test_resize_reflow.py b/tests/textual/test_resize_reflow.py index 4b66b269d..573ee34b4 100644 --- a/tests/textual/test_resize_reflow.py +++ b/tests/textual/test_resize_reflow.py @@ -1,7 +1,8 @@ """Textual-facing: resizing the terminal (fullscreen and back) reflows saved history to the new width and keeps the tail, or the reader's place, on screen.""" from agent_comms.transcript_events import AssistantTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.agent_response import AgentResponse from toad.widgets.transcript_history import TranscriptHistory @@ -10,7 +11,7 @@ async def test_resize_keeps_tail_and_reader_position_visible(workspace): text = "## Resize test\n\n" + "A paragraph that wraps to the available viewport width. " * 80 events = tuple(AssistantTranscript(f"Record {i}\n\n{text}") for i in range(20)) - page = TranscriptPage(events, TranscriptCursor("", 0), TranscriptCursor("", 0), False, False) + page = TranscriptPage(events, NativeCursor("", 0), NativeCursor("", 0), False, False) async def load(**kwargs): raise AssertionError("No preceding file history in this fixture") diff --git a/tests/textual/test_saved_history_availability.py b/tests/textual/test_saved_history_availability.py index 234fb1113..dd04bc28f 100644 --- a/tests/textual/test_saved_history_availability.py +++ b/tests/textual/test_saved_history_availability.py @@ -8,7 +8,8 @@ import re from agent_comms.transcript_events import AssistantTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from support import agents from toad.widgets.session_details import SessionDetails @@ -16,7 +17,7 @@ def saved_page(): - cursor = TranscriptCursor("saved-page-proof", 1) + cursor = NativeCursor("saved-page-proof", 1) events = tuple(AssistantTranscript( f"Response number {number} describes interesting particular observations about segmentation.\n\n" "Another paragraph with considerable additional vocabulary appears afterwards.") for number in range(6)) diff --git a/tests/textual/test_saved_tool_call_details.py b/tests/textual/test_saved_tool_call_details.py index b7ea20c56..97695dafc 100644 --- a/tests/textual/test_saved_tool_call_details.py +++ b/tests/textual/test_saved_tool_call_details.py @@ -3,14 +3,15 @@ from agent_comms.transcript_events import ( AssistantTranscript, ContextTranscript, ToolEndTranscript, ToolStartTranscript, ) -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.transcript_fragments import ToolTranscriptFragment from toad.widgets.transcript_history import TranscriptHistory def saved_page(): - cursor = TranscriptCursor("fixture", 1) + cursor = NativeCursor("fixture", 1) return TranscriptPage(( AssistantTranscript("BEFORE_THE_TOOL"), ContextTranscript("CONTEXT_GIVEN_TO_THE_AGENT"), diff --git a/tests/textual/test_sparse_history.py b/tests/textual/test_sparse_history.py index ee8b9becd..915c90dcf 100644 --- a/tests/textual/test_sparse_history.py +++ b/tests/textual/test_sparse_history.py @@ -1,14 +1,15 @@ """Textual-facing: a history of one-line pages fills the viewport by reading older pages as the window's layout asks, then goes idle.""" from agent_comms.transcript_events import AssistantTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from toad.widgets.transcript_history import TranscriptHistory def page(index): return TranscriptPage((AssistantTranscript(f"Saved activity {index}"),), - TranscriptCursor("test", index), TranscriptCursor("test", index + 1), + NativeCursor("test", index), NativeCursor("test", index + 1), index > 0, index < 29) diff --git a/tests/textual/test_thread_activation.py b/tests/textual/test_thread_activation.py index 613751313..20051ce78 100644 --- a/tests/textual/test_thread_activation.py +++ b/tests/textual/test_thread_activation.py @@ -3,7 +3,8 @@ import asyncio from agent_comms.transcript_events import AssistantTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from support.agents import detached from support.stylesheets import INSTALLED @@ -86,7 +87,7 @@ async def test_returning_while_older_history_is_loading_paints_the_latest_page(w conversation, response, window, other = await long_history_then_other_tab(ui) await ui.select_tab(owner) await conversation.contents.remove_children() - cursor = TranscriptCursor("test", 100) + cursor = NativeCursor("test", 100) page = TranscriptPage((AssistantTranscript("LATEST-PAGED-ACTIVITY"),), cursor, cursor, True, False) started, release = asyncio.Event(), asyncio.Event() @@ -116,7 +117,7 @@ async def test_a_snapshot_published_into_a_conversation_paints_at_the_latest_act agent = detached(workspace.project, "snapshot-session") ui.attach_agent(agent) await conversation.contents.remove_children() - cursor = TranscriptCursor("test", 100) + cursor = NativeCursor("test", 100) snapshot = TranscriptPage( tuple(AssistantTranscript(f"Snapshot paragraph {i}") for i in range(100)) + (AssistantTranscript("LATEST-SNAPSHOT-ACTIVITY"),), cursor, cursor, True, False) diff --git a/tests/textual/test_transcript_page_width.py b/tests/textual/test_transcript_page_width.py index 9fed6b6ed..152d03dbf 100644 --- a/tests/textual/test_transcript_page_width.py +++ b/tests/textual/test_transcript_page_width.py @@ -6,7 +6,8 @@ import unittest from agent_comms.transcript_events import AssistantTranscript, UserTranscript -from agent_comms.transcripts import TranscriptCursor, TranscriptPage +from agent_comms.transcript_sources import TranscriptCursor, NativeCursor +from agent_comms.transcripts import TranscriptPage from textual._context import active_app from textual.app import App from textual.selection import Selection @@ -17,7 +18,7 @@ def page_view(count: int) -> TranscriptPageView: - cursor = TranscriptCursor("fixture", 1) + cursor = NativeCursor("fixture", 1) events = tuple( (UserTranscript if index % 3 == 0 else AssistantTranscript)( f"Message {index} " + "word " * (index % 40) + "\n\n- item\n- another item") diff --git a/tools/performance/capture_live.py b/tools/performance/capture_live.py index 36c6c4c95..5aa642309 100644 --- a/tools/performance/capture_live.py +++ b/tools/performance/capture_live.py @@ -1,5 +1,6 @@ -"""Measure an explicitly selected live Toad process. +"""Measure an explicitly selected live Toad process on required --display. +Input controllers use this same XvfbDisplay.input owner and terminal launch identity. --profile-seconds record a py-spy speedscope profile. --frame-meter N time UI work per frame and input-to-paint for N seconds (frame_meter.py). --targets write where sidebar thread rows and session tabs are (frame_meter.targets). @@ -23,6 +24,8 @@ class XvfbDisplay(AbstractContextManager): """Own a connection to a local framebuffer, resolving X11 aliases at Xlib.""" def __init__(self, name: str): + if not name: + raise ValueError("An explicit isolated Xvfb display is required") x11 = ctypes.CDLL('libX11.so.6') x11.XOpenDisplay.argtypes = [ctypes.c_char_p] x11.XOpenDisplay.restype = ctypes.c_void_p @@ -33,23 +36,49 @@ def __init__(self, name: str): connection = x11.XOpenDisplay(name.encode()) if not connection: raise ValueError(f'Cannot connect to isolated Xvfb display: {name}') - with ExitStack() as custody: - custody.callback(x11.XCloseDisplay, connection) + with ExitStack() as close_display: + close_display.callback(x11.XCloseDisplay, connection) with socket.fromfd(x11.XConnectionNumber(connection), socket.AF_UNIX, socket.SOCK_STREAM) as peer: pid, uid, _gid = struct.unpack('3i', peer.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, 12)) address = Path(os.fsdecode(peer.getpeername()).lstrip('\0')) if address.name == 'X0' or uid != os.getuid() or Path(psutil.Process(pid).exe()).name != 'Xvfb': raise ValueError(f'Not an owned isolated Xvfb resource: {name}') self.name = name - self.custody = custody.pop_all() + self.close_display = close_display.pop_all() + + @property + def environment(self) -> dict[str, str]: + return {**os.environ, "DISPLAY": self.name} + + @classmethod + def input(cls, frontend: psutil.Process, created: float, window: int, *commands: str) -> None: + """Take the display from the actual st frontend, never the controller environment.""" + if frontend.create_time() != created or not frontend.is_running(): + raise ValueError("Frontend launch identity changed") + with cls(frontend.environ()["DISPLAY"]) as display: + env = display.environment + owner = subprocess.check_output( + ["xprop", "-display", display.name, "-id", str(window), "_NET_WM_PID"], + env=env, text=True, timeout=5, + ) + if int(owner.partition("=")[2].strip()) != frontend.pid: + raise ValueError("Window does not belong to the frontend on its display") + print(json.dumps({"at_ns": time.time_ns(), "monotonic_ns": time.monotonic_ns(), + "input_target": {"display": display.name, "window": window, + "frontend_pid": frontend.pid, "created": created}}), flush=True) + subprocess.run( + ["xdotool", "windowfocus", "--sync", str(window), *commands], + env=env, check=True, + ) def __exit__(self, *exception): - return self.custody.__exit__(*exception) + return self.close_display.__exit__(*exception) def main(): parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) parser.add_argument("--pid", type=int, required=True) + parser.add_argument("--display", required=True, help="Explicit isolated Xvfb display shared with the input controller") parser.add_argument("--output-dir", type=Path, required=True) parser.add_argument("--name", default="live-" + time.strftime("%Y%m%d-%H%M%S")) parser.add_argument("--profile-seconds", type=int, default=0) @@ -87,7 +116,7 @@ def main(): help="Arrange the screen at N new widths and time each pass (frame_meter.remeasure)") parser.add_argument("--repaint", type=int, default=0, help="Repaint the history viewport N times cold and warm, timing each (frame_meter.repaint)") - parser.add_argument("--sudo", action="store_true", help="Use non-interactive sudo for attach operations") + parser.add_argument("--sudo", action="store_true", help="Use non-interactive sudo for read-only py-spy profiling only") args = parser.parse_args() if not (args.profile_seconds > 0 or args.frame_meter > 0 or args.targets or args.profile_layout or args.trace_calls or args.trace_reflows or args.trace_rules or args.trace_exit or args.trace_garbage or args.collect_all or args.heap_components or args.heap_census or args.freeze_heap or args.trace_abc or args.trace_posts @@ -105,6 +134,9 @@ def main(): created = process.create_time() privilege = ["sudo", "-n"] if args.sudo else [] manifest = {"pid": args.pid, "created": created, "executable": process.exe(), "started_ns": time.time_ns()} + close_display = ExitStack() + display = close_display.enter_context(XvfbDisplay(args.display)) + manifest["display"] = display.name try: if args.profile_seconds: result = subprocess.run([*privilege, args.py_spy, "record", "--pid", str(args.pid), @@ -161,7 +193,6 @@ def main(): entered = str(prefix) + "-remote-entered.json" lines = [ "import importlib.util as _spec_util, json as _json, os as _os, sys as _sys, time as _time", - f"with open({entered!r}, 'w') as _out: _json.dump({{'pid': _os.getpid(), 'entered_ns': _time.time_ns()}}, _out)", # One meter module per process: snapshots record their pauses where the meter reads them. "_meter = _sys.modules.get('frame_meter')", "if _meter is None:", @@ -170,10 +201,11 @@ def main(): " _spec.loader.exec_module(_meter)", *(f"_meter.{call}(expected_pid={args.pid}, output={output + '.json'!r}{extra})" for output, call, extra in calls), + f"with open({entered!r}, 'w') as _out: _json.dump({{'pid': _os.getpid(), 'entered_ns': _time.time_ns()}}, _out)", ] script = Path(str(prefix) + "-remote.py") script.write_text("\n".join(lines) + "\n") - subprocess.run([*privilege, process.exe(), "-c", + subprocess.run([process.exe(), "-c", f"import sys; sys.remote_exec({args.pid}, {str(script)!r})"], check=True, timeout=15) deadline = time.monotonic() + max(args.frame_meter, 60 if args.profile_layout or args.remeasure or args.repaint else 0) + 20 outputs = [output + ".json" for output, _call, _extra in calls] @@ -182,6 +214,7 @@ def main(): manifest["outputs"] = {path: Path(path).exists() for path in outputs} manifest["entered"] = Path(entered).exists() finally: + close_display.close() manifest["finished_ns"] = time.time_ns() manifest_path.write_text(json.dumps(manifest, indent=2)) print(manifest_path) diff --git a/tools/performance/frame_meter.py b/tools/performance/frame_meter.py index c9387b067..db3c251d3 100644 --- a/tools/performance/frame_meter.py +++ b/tools/performance/frame_meter.py @@ -54,7 +54,8 @@ def install(*, expected_pid, seconds, output): app = _running_app(expected_pid) - inputs_types = (events.Key, events.MouseScrollUp, events.MouseScrollDown) + # Divider drags are input too; the mouse family includes wheel events. + inputs_types = (events.Key, events.MouseEvent) clock = time.monotonic_ns state = {"busy": 0, "current": None, "parts": {}, "cpu": time.thread_time_ns(), "shown": clock()} frames, inputs, slow, slow_frames, gc_pauses = [], [], [], [], [] diff --git a/tools/performance/scroll_scenario.sh b/tools/performance/scroll_scenario.sh index f2ce58797..8fe6394f9 100755 --- a/tools/performance/scroll_scenario.sh +++ b/tools/performance/scroll_scenario.sh @@ -2,7 +2,7 @@ # The fixed scroll scenario for docs/performance/scroll-series.md. # Usage: scroll_scenario.sh NAME [BIN_DIR] # Forks perf-base-20261009 (a fork of Tristan's saved agent-comms-ux session), opens it -# with the real `toad` in st on the isolated display :121, and drives real X input: +# with the real `toad` in st on explicit SCENARIO_DISPLAY, and drives real X input: # an active response, held PageUp from the input box, wheel bursts with reversal, # End, and a return to the open tab. capture_live.py --frame-meter times every frame. set -euo pipefail @@ -13,15 +13,37 @@ out=/home/ts/.cache/agent-scratch/scroll-series/$name fork=perf-$name-$(date +%H%M%S) project=/home/ts/wt/comms-post-feature-debt-audit-20260928 meter_seconds=${METER_SECONDS:-45} -export DISPLAY=${SCENARIO_DISPLAY:-:131} -rm -rf "$out" -mkdir -p "$out" +display=${SCENARIO_DISPLAY:?Set an explicit owned Xvfb display} +export DISPLAY=$display +PYTHONPATH="$tools" python3 -c ' +import sys +from capture_live import XvfbDisplay +with XvfbDisplay(sys.argv[1]): + pass +' "$display" +mkdir -p "$(dirname "$out")" +mkdir "$out" agent-comms fork --name "$fork" --parent perf-base-20261009 > "$out/fork.json" env -u NO_COLOR -u TOAD_VALIDATION st -g145x50 -t "$fork" -e "$bin/toad" acp "$bin/agent-comms-acp" \ --session "$fork" --project-dir "$project" --title "$fork" > "$out/st.log" 2>&1 & st_pid=$! +read -r st_created display < <(python3 -c ' +import psutil,sys +frontend = psutil.Process(int(sys.argv[1])) +print(frontend.create_time(), frontend.environ()["DISPLAY"]) +' "$st_pid") +# Recorder and input both take their display from the actual frontend. +xdotool() { + PYTHONPATH="$tools" python3 -c ' +import psutil,sys +from capture_live import XvfbDisplay +frontend = psutil.Process(int(sys.argv[1])) +XvfbDisplay.input(frontend, float(sys.argv[2]), int(sys.argv[3]), *sys.argv[4:]) +' "$st_pid" "$st_created" "$window" "$@" +} cleanup() { + local status=$? xdotool key --window "$window" ctrl+q 2>/dev/null || true # Wait for the app to exit by itself: killing the terminal sends SIGHUP to # the app and its agents, and a process killed inside a guarded registry @@ -30,6 +52,10 @@ cleanup() { if kill -0 "$st_pid" 2>/dev/null; then echo "app did not exit within 60 s; leaving it running rather than killing it mid-write" >&2 fi + if (( status != 0 )); then + echo "Retaining failed run and fork $fork; no stop/delete or replay" >&2 + return "$status" + fi # A fork is a full copy of the parent's 141 MB history: delete it, which # removes its registration, per-thread state and session files. agent-comms stop --name "$fork" > /dev/null 2>&1 || true @@ -39,23 +65,22 @@ trap cleanup EXIT for _ in $(seq 60); do app_pid=$(pgrep -P "$st_pid" | head -1 || true) - window=$(xdotool search --pid "$st_pid" 2>/dev/null | head -1 || true) + window=$(DISPLAY="$display" command xdotool search --pid "$st_pid" 2>/dev/null | head -1 || true) [[ -n "$app_pid" && -n "$window" ]] && break sleep 1 done -xdotool windowfocus --sync "$window" # Optional: raise the UI thread's scheduling priority (for example UI_NICE=-10) # to separate the app's own work from waiting for a CPU behind other processes. if [[ -n "${UI_NICE:-}" ]]; then sudo -n renice -n "$UI_NICE" -p "$app_pid" > "$out/renice.log" 2>&1 fi sleep "${LOAD_SECONDS:-25}" -read -r width height < <(xdotool getwindowgeometry --shell "$window" | sed -n 's/^\(WIDTH\|HEIGHT\)=//p' | paste -sd' ') +read -r width height < <(DISPLAY="$display" command xdotool getwindowgeometry --shell "$window" | sed -n 's/^\(WIDTH\|HEIGHT\)=//p' | paste -sd' ') cell() { xdotool mousemove --window "$window" $(( ($1) * width / 145 + 3 )) $(( ($2) * height / 50 + height / 100 )) click 1; } -regions() { python3 "$tools/capture_live.py" --pid "$app_pid" --output-dir "$out" --name "$1" --targets --sudo > /dev/null; } +regions() { python3 "$tools/capture_live.py" --display "$display" --pid "$app_pid" --output-dir "$out" --name "$1" --targets --sudo > /dev/null; } # Who asks the app to exit is written as it happens (the app has quit by # itself when the second tab opened, with no error recorded). -python3 "$tools/capture_live.py" --pid "$app_pid" --output-dir "$out" --name app --trace-exit --sudo > /dev/null 2>&1 & +python3 "$tools/capture_live.py" --display "$display" --pid "$app_pid" --output-dir "$out" --name app --trace-exit --sudo > /dev/null 2>&1 & # Optional: measure from here, so opening the second tab (and the closing of # the deleted thread's tab at the end) fall inside the measured window. if [[ -n "${METER_FROM_OPEN:-}" ]]; then @@ -68,7 +93,7 @@ if [[ -n "${METER_FROM_OPEN:-}" ]]; then else measure=(--frame-meter "${METER_SECONDS:-150}") fi - python3 "$tools/capture_live.py" --pid "$app_pid" --output-dir "$out" --name run "${measure[@]}" \ + python3 "$tools/capture_live.py" --display "$display" --pid "$app_pid" --output-dir "$out" --name run "${measure[@]}" \ ${TRACE_CALLS:+--trace-calls "$TRACE_CALLS"} ${TRACE_REFLOWS:+--trace-reflows} ${TRACE_RULES:+--trace-rules} \ ${TRACE_GARBAGE:+--trace-garbage} ${HEAP_CENSUS:+--heap-census "$HEAP_CENSUS"} ${COLLECT_ALL:+--collect-all "$COLLECT_ALL"} ${HEAP_COMPONENTS:+--heap-components "$HEAP_COMPONENTS"} ${FREEZE_HEAP:+--freeze-heap} ${TRACE_ABC:+--trace-abc} ${TRACE_POSTS:+--trace-posts} --sudo > "$out/capture.log" 2>&1 & capture=$! @@ -128,7 +153,7 @@ else measure=(--frame-meter "$meter_seconds") fi if [[ -z "${METER_FROM_OPEN:-}" ]]; then -python3 "$tools/capture_live.py" --pid "$app_pid" --output-dir "$out" --name run \ +python3 "$tools/capture_live.py" --display "$display" --pid "$app_pid" --output-dir "$out" --name run \ "${measure[@]}" ${TRACE_CALLS:+--trace-calls "$TRACE_CALLS"} ${TRACE_REFLOWS:+--trace-reflows} ${TRACE_RULES:+--trace-rules} --sudo > "$out/capture.log" 2>&1 & capture=$! fi @@ -212,7 +237,7 @@ python3 "$tools/screen_checks.py" "$out" | tee "$out/screen-checks.json" # Optional: the per-build paint figure for this live history viewport (after # the checks: it repaints the history window in full, cold, warm and scrolled by a page). if [[ -n "${REPAINT:-}" ]]; then - python3 "$tools/capture_live.py" --pid "$app_pid" --output-dir "$out" --name repaint \ + python3 "$tools/capture_live.py" --display "$display" --pid "$app_pid" --output-dir "$out" --name repaint \ --repaint "$REPAINT" --sudo > /dev/null python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('error') or {k: d[k] for k in ('cold', 'warm', 'scroll')})" \ "$out/repaint-repaint.json" | tee "$out/repaint.txt" @@ -221,7 +246,7 @@ fi # Optional: the per-build layout figure for this live widget tree (after the # checks, since it re-measures the screen at widths it is not shown at). if [[ -n "${REMEASURE:-}" ]]; then - python3 "$tools/capture_live.py" --pid "$app_pid" --output-dir "$out" --name remeasure \ + python3 "$tools/capture_live.py" --display "$display" --pid "$app_pid" --output-dir "$out" --name remeasure \ --remeasure "$REMEASURE" --sudo > /dev/null python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print({k: d[k] for k in ('median_ms', 'p10_ms', 'p90_ms')})" \ "$out/remeasure-remeasure.json" | tee "$out/remeasure.txt"