Skip to content

Latest commit

 

History

History
91 lines (51 loc) · 24.1 KB

web.md

File metadata and controls

91 lines (51 loc) · 24.1 KB

Web Surface

The web surface consists of authentication, HTTP, SSE, and connection primitives in src/web/common/, plus the repository viewer in src/web/viewer/ and viewer-ui/. The viewer does not use the TUI's App/ui/input; it uses session operations, runtime, and the terminal hub, so it can run together with the daemon started by nightcrow without arguments, even when no TUI is attached.

Common web boundary

  • Passwords are verified as Argon2 PHC values, and login attempts are limited process-wide to 2 per minute and 14 per hour. On success, the server issues an httpOnly, SameSite=Strict session cookie. Cookie names are distinct per server.
  • Session tokens are opaque random values stored in ~/.nightcrow/sessions and revoked server-side on logout. The configured TTL also applies to existing tokens; expired-token sweeping occurs on load/write. The store keeps at most 64 tokens: a login past the cap forgets the oldest in memory and in the persisted file, so repeated logins cannot grow the file without bound even when the TTL is zero. File writes are serialized from snapshot through replacement, so a concurrent write cannot undo an eviction or logout; a legacy file that lists a token twice or out of eviction order is normalized on load, keeping the earliest expiry per token. Unix files are owner-only (0600). The Windows permission seam is a no-op, so operators must protect access to the state directory.
  • The default bind address is loopback, and TLS is not provided. Remote access assumes an SSH tunnel or TLS reverse proxy. Connections, headers, bodies, WebSocket messages, and SSE payloads are bounded; a connection slot is returned through Drop when its handler exits.
  • SSE uses a dedicated stream that writes the head directly and flushes every event. Event names cannot contain newlines, and write errors are propagated explicitly.
  • A repository status stream sends a successfully read status as a status event and the reason for an unreadable status (such as a directory that is not a Git repository) as an unavailable event. Silently swallowing failures would leave a client waiting forever for its first status. /api/status returns the same failure as 409.

Request and repository gates

Requests follow this order:

Host → Origin → static bundle → authentication → repository lookup → path gate → handler

Host is checked before Origin to prevent DNS rebinding from exposing an internal service on a loopback bind. The static bundle is served before authentication because it contains the login form, but repository APIs, SSE, and WebSocket are opened only after authentication. Authentication occurs before repository lookup to prevent ID enumeration.

Repositories are identified by opaque IDs stable for the process lifetime, not by client-created paths. The catalog uses canonical worktree paths at open/add/close/reorder boundaries to merge duplicates, and produces the list and active ID from one catalog snapshot.

Routes that open files use resolve_in_workdir through the shared with_repo gate. Routes that pass paths only to a Git commit/pathspec use validate_commit_path through with_repo_git_path. Both gates reject traversal, absolute paths, NUL, and .git variants; the file gate also checks symlinks and containment within the worktree. Routes do not duplicate these gates. Deleted historical paths remain available to diffs because they do not go through the filesystem gate.

HTML preview sends only validated files to a sandboxed iframe. It uses both response CSP and iframe sandbox allow-scripts, with connect-src 'none' to block external connections. Navigation spoofing by a preview document remains a residual risk.

Runtime and terminal transport

Repository runtime status snapshots need only the latest payload, so byte-identical values are not published and fan-out can conflate them. Terminal output is a raw byte stream and is never conflated; a client whose queue fills has its socket closed. A WebSocket terminal binary frame contains a 4-byte little-endian pane ID followed by raw PTY bytes; control events use JSON frames. The terminal socket disables Nagle and flushes once after draining its pending frames. A connection that passed the cap check enables TCP keepalive, and the server pings a terminal socket once 15 seconds pass without a frame from the page: output written to it does not restart that clock, so a page that only watches a busy pane is still asked. Any frame back proves the path, while 30 seconds without a frame from the page closes the socket, releasing its hub client and its connection slot; a socket with a write still outstanding is never closed this way, because frames the page sent may be waiting behind that write. It ends when that write fails or the hub's queue overflows, and the silence limit applies again once the write completes. A refusal past the connection cap logs at WARN at most once per ten seconds, naming only the peer address.

Terminal connections wait for work instead of polling for it. The page's read half blocks in its own thread, the hub's frames are handed over by a forwarder thread, and one writer thread parses, frames, and writes; it sleeps until an event arrives or the next liveness decision is due. Both directions share one bounded queue, so a stalled write stops the reader at the socket rather than buffering without limit: while a write is outstanding the writer takes nothing more from the hub, so at most QUEUE_DEPTH events plus the one in hand sit outside the hub, and the hub's own bounded overflow is what ends a client that stopped reading. A blocked read cannot be cancelled portably — shutting the socket down does not wake one on Windows — so the reader rechecks its stop flag on a one-second guard interval; that guard is the whole idle cost of a connection. The writer ends both helper threads before it returns, so no connection outlives its socket. Clients that cannot keep up are closed at the session terminal-queue limit and reconnect using screen/mode/since replay. Initial replay places the replay_complete control marker in the same FIFO to identify the boundary. On the browser side a lost terminal socket is retried on a jittered doubling wait (0.5 s doubling to 15 s, reset once a connection stays up for 10 s), and a pane's emulator is kept across the reconnect and reset before the replay instead of being rebuilt: the replay is an absolute repaint, so an empty parser is all it needs. Frames wait in the page once bytes handed to xterm but not yet parsed pass a high-water mark, resume below a low-water mark, and the replay completion marker queues behind them. Panes use xterm's WebGL renderer when a context is available and stay on the DOM renderer otherwise, including after a context is lost. A per-connection/per-pane xterm reply gate is released only when all replay bytes for that pane have been parsed and term.write('', callback) runs. For a pane whose body node does not exist yet, replay bytes and the completion sentinel share one FIFO, preserving order relative to later live bytes. If the body node exists but is 0×0, xterm is opened immediately to create its parser, reply gate, and theme service; replay/live bytes are parsed at the confirmed PTY size and retained in the default 1000-line scrollback. Character metrics may not be available at that point, so when the pane body becomes visible, usePaneSizes calls term.resize(term.cols, term.rows) before fitting to retry measurement through xterm's public resize path on the same grid. If the dimensions are unchanged this does not emit a PTY resize event; subsequent fitting and size requests follow the existing size-owner rules. During replay, responses to CSI DSR/DA/DECRQM, DCS DECRQSS, and CSI 14t/16t/18t reports are suppressed, and the OSC 52 clipboard callback is not run. CSI 22t/23t title-stack commands pass through, and keyboard input and paste continue to be sent to the PTY. OSC 4/10/11/12, which can carry both queries and color sets in one OSC command, are outside this suppression scope. An authenticated client sends pane input/resize/close/reorder to the session hub; pane-specific authorization is not provided.

User key, paste, pointer down/click, and wheel input are translated into session size-owner claims. Activity by the current owner resends a confirmation to that viewer connection for manual refitting; ordinary activity does not transfer ownership or force a resize. Browser resize, focus, and reconnect do not change ownership.

The server determines canonical pane order and zoom. Clients do not apply requests optimistically; they converge from reordered/zoomed echoes and reconnect replay. Pane order and zoom are not written to disk. Created carries each pane's current size/title, and initial replay carries the exact pane count. Client-created resizes are sent to the server only after geometry settles, and only a Resized confirmed by the session owner is applied to the emulator. While the soft keyboard reduces the visual viewport, that change is not treated as geometry: no fit or resize is sent, and the pane body clips the terminal while showing the lowest window that contains the cursor line (the bottom for a full screen, the top containing the prompt for a newly opened terminal). The layout is applied once after the keyboard closes. Keyboard detection uses only the height difference between layout and visual viewports.

Wire and log contract

GET /api/repos bootstraps repositories, hot config, session accent, server clock, clone availability, and viewer arrangement. Every JSON response includes the Envelope PROTOCOL_VERSION. Rust DTOs and TypeScript interfaces are kept aligned with fixtures in viewer-ui/api.fixture.json and api.contract.test.ts. Keep fixtures for both present and absent optional fields.

viewer-ui/src/api.ts is the stable import facade over api/session.ts (authentication, preferences, and config reload), api/repository.ts (repository access, browsing, and clone), api/git.ts (history and file reads), and api/content.ts (preview and content writes). api/client.ts owns ordinary HTTP encoding and protocol checks; api/conflictMutation.ts shares optimistic-edit response handling while preserving each operation's fallback message.

POST /api/file is the only route that writes a working-tree file (POST /api/paste-image writes only outside the worktree). Like other mutations, it checks Origin and uses the SameSite=Strict cookie for CSRF protection. It passes through the same worktree gate as /api/file reads (resolve_in_workdir, which rejects traversal, symlinks, and Git directories). Authenticated users can already write files through a terminal, so this route is within the same trust boundary; it targets only existing working-tree files (commit versions are read-only history). For optimistic concurrency, if the body base_hash (the blob oid when editing began) differs from the current disk oid, the request is rejected with 409 and currentHash, preventing it from overwriting a change made underneath it; force overrides this check. On success, the response returns the saved content's blob oid for the client to use as the next save's base.

POST /api/paste-image receives an image pasted into a pane. A pane is a PTY on this machine, so the only way to give its program an image is a file it can open; the browser holding the clipboard is often on another machine, which is why CLI clipboard paste cannot reach it. The bytes therefore travel over the wire, the server writes a file, and the client types that path into the pane. The body contains raw bytes rather than base64; conn::read_request makes this possible by returning the body as bytes instead of a string.

No part of the path comes from the caller. The filename is random, the extension is determined from magic bytes (anything other than PNG/JPEG/GIF/WebP receives 415), and the directory is fixed at ~/.nightcrow/tmp; the route cannot be redirected to a caller-chosen path and needs no worktree gate. Writing outside the repository is intentional: a pasted screenshot is not a project change and should not appear in status. Old files are cleaned up on each write by age (24h) and count.

Request-body limits vary by route. Most routes carry small control payloads and use MAX_BODY_BYTES (64 KB); POST /api/file, which carries a whole document, uses MAX_FILE_WRITE_BYTES (1 MB); POST /api/paste-image, which carries one screenshot, uses MAX_PASTE_IMAGE_BYTES (10 MB). A body over its limit is rejected with 413 without reading it. Accepting a truncated body could leave a parseable fragment that a handler writes to disk as if it were a complete file. The server sends the response and closes immediately, so a client uploading at that moment may see a dropped connection instead of the response.

/api/preview/edit assembles a preview for inline editing. srcdoc inherits the parent CSP, which blocks inline scripts, so the editing agent must be delivered in a network response with its own policy. Block detection (parse5) runs on the client, and assembled HTML exceeds the 64 KB body limit; therefore the client POSTs a small insert list (one marker per block plus a head payload containing the agent), using UTF-8 byte offsets. The server rereads the file, splices inserts only if its content matches base_hash (blob oid; otherwise it returns 409 plus currentHash), stashes the result behind a one-time token, and returns the token. The frame fetches the document once through GET /api/preview/edit?token= and loads it under preview policy (sandbox plus execution CSP). The stash is temporary state inside the session trust boundary, the token is consumed, and the assembled document is never written to disk. POST /api/file handles saving edits.

/api/log uses MAX_LOG_PAGE = 100 and from=<head oid> + skip. from anchors the same revwalk so new commits between pages do not shift the offset. The cursor is not the last oid's ancestor. skip never walks past history length; the server reads page + 1 items to determine truncated. It does not automatically request more pages while filtering, and marks page failures as a retryable stalled state. When status HEAD changes, the log cache is refreshed with a new page and generation. Ref chips and upstream ahead/behind data are not included in each page; /api/log/decorations returns the repository-wide values in one response. The history described by a page does not change, but decorations do whenever refs move—push/fetch or a branch switch at the same commit—so the client reloads all decorations when status HEAD, branch, refs digest, or upstream name changes. The response is limited to 2000 ref labels (MAX_LOG_DECORATION_REFS), dropping remote branches and tags first while retaining the HEAD branch and setting truncated. Failed requests are retried after 5 seconds so changes are not lost.

Browser state and frontend

viewer.json stores the session accent, upper_pct, and each project's last view/maximize. Sidebar width is measured in pixels and belongs to one screen, so it is kept per device in browser localStorage rather than on the server. The narrow-screen Repo/Content/Terminal selection is also stored in browser localStorage and restored after refresh; it is not included in shared repository view state. Active repo is stored as an absolute worktree path and converted to an opaque ID in responses. Values are clamped on both server and client, and view path/oid/tab/face values are sanitized at storage/restore boundaries. The TUI's workspace.json and viewer preferences have separate owners.

Keyboard handling has one decision point in the document capture phase. Two listeners cannot agree on whether a key was consumed; the losing listener could either swallow a pane key or pass an app command through as an escape sequence. Commands use semantic action IDs in a registry rather than physical keys, and keyboard, help, and buttons read the same table. The registry records whether the browser preserves, reinterprets, or does not support each meaning, instead of copying the TUI Rust key table. Since terminal-panel commands are below the page, the panel registers with the intent bus and that registration is the single source of availability. The web hint line corresponding to the TUI hint bar is a pure function of the same registry and availability; the leader's armed state is rendered only from a mirror of the ref read by keystrokes. Leader preference is client-local per browser, so it belongs in browser storage rather than viewer.json or session state. React screens separate page composition, reusable components, hooks, pure lib, and API/wire modules. Terminal WebSocket decoding/encoding is validated as a discriminated union at one boundary. Large diffs/raw files keep only the viewport and overscan in the DOM; small files preserve native selection, find, and accessibility. ErrorBoundary prevents a lazy-chunk failure from appearing as a full page unmount, and stale pages are reloaded by comparing the server build ID with the content-hashed bundle. DOM hook tests use happy-dom, while pure utilities run in the node environment. Since the built viewer-ui/dist is included at runtime, cargo install must work without Node.

Images pasted or dropped onto a pane are intercepted by a capture-phase listener attached to one panel. The target pane comes from the event cell's data-pane-id: paste targets the focused terminal, while drop targets the cell under the pointer. If clipboard content includes text as well, it stays a text paste—copying from a rich editor includes both, but the user intended the text. Non-image content follows xterm's own path. On narrow screens, Attach image in the compose dialog's pencil control also opens the native picker for PNG/JPEG/GIF/WebP, calls the same POST /api/paste-image, and appends the returned path to the current pane draft. Files up to 10 MiB are accepted. While an upload is in progress or a failed attachment remains, sending is blocked until the user retries or explicitly closes the error. Upload results are discarded if the dialog closes or its target pane disappears; once an upload completes, its path is appended to the end of the draft so edits made during the upload are preserved.

The editing session is managed by hooks/git/useHtmlEditor.ts and rendered by components/content/HtmlEditor.tsx. The pencil toggle in the file panel appears only for working-tree HTML files; a committed copy is history and cannot be edited in place. The session keeps the parsed source string and its patch list in refs so a keystroke-driven frame rerender does not destroy an open edit; on save it rebuilds the result from source plus patches and sends it to POST /api/file. Editing stays closed until post-render validation completes (agent ready → applyLiveLocks → locked reply); clicking a locked block shows the reason instead of editing it. Toolbar save sends flush to the frame to commit open edits first (Ctrl+S inside the frame makes the agent commit and then send save, so it is already applied in the same channel order). If disk contents are newer, the editor asks whether to overwrite while preserving the edit. The frame uses the same sandbox as ordinary preview (never allow-same-origin) and cannot access the network directly; only the authenticated parent saves. The editing-engine chunk is lazy-loaded, so read-only sessions do not pay for it.

The inline HTML preview editing engine lives as pure logic under lib/edit/: it parses source text to find editable blocks and their byte offsets (parse), inserts markers that map rendered text back to source (markers), and applies user edits with offset splices (patch). The original remains a string and is never reserialized. Re-serializing a DOM would rewrite indentation, attribute order, and entities across the entire document, so changing one block could make the diff cover the whole file. A spec-compliant parser that provides source offsets is therefore required, so the engine uses parse5 (sourceCodeLocationInfo; htmlparser2 is faster but not spec-exact and has weaker location data). Entity encoding/decoding is delegated to the standard entities library because manually maintaining a table of over 2000 named entities would inevitably be wrong. The engine was ported from nighteditor.

Clone

Clone is delegated to the git binary, which supports credential helpers and SSH transport. Only https/http/ssh/git+ssh and scp-like user@host:path forms are allowed; ext::, file://, local paths, and git:// are rejected. The URL is checked for length/control characters, and the destination name uses only a single plain-text segment extracted from the URL. The destination directory is reserved with create_dir before execution to reduce check/use races, and remove_dir runs after failure only if the directory is empty.

Only one clone may run at a time, and a job outlives the request connection. Clients poll by job ID; a cleaned-up job ends with 404. The process uses GIT_TERMINAL_PROMPT=0, SSH liveness options, and an HTTP low-speed policy. These are limits that terminate a stalled transfer, not wall-clock deadlines guaranteeing completion.

Web Push

web::viewer::push subscribes to the session's Notices and turns each notice into a Web Push message. The session does not know the listener exists: PushService::follow registers it, holding the session weakly because the session owns the listener list. Hub workers only queue; one nightcrow-push thread encrypts (RFC 8291 aes128gcm, to the browser's own key, so the push service relays ciphertext), signs a VAPID token (RFC 8292, ES256) and posts with ureq, https only. A burst past the 64-entry queue is dropped rather than held behind a slow push service. A notice for the same repository pane inside 10 seconds is not sent again, and the Topic header lets the push service replace an undelivered one. A 404 or 410 answer removes the subscription; other failures are logged with the endpoint's host only, since the path is a per-browser capability.

The VAPID key (~/.nightcrow/push/vapid) and subscriptions (subscriptions.json, at most 32, oldest dropped) are written owner-only. A key file that exists but cannot be parsed is an error, not a reason to make a new one, because a new key silently ends every subscription. If the key cannot be loaded the viewer runs without Web Push and its /api/push/* routes answer 503.

The routes are authenticated like every other /api/ route, and everything but GET /api/push/key is a POST carrying the endpoint in its body. An endpoint must be an https URL on a public host name; addresses, localhost, .local names and userinfo are refused. There is deliberately no allowlist of push-service hosts: anyone who can register an endpoint can already run shell commands on this machine, and a list would break browsers whose push service it omits.

Accepted residual risks

  • Repository::discover searches upward from a directory opened or restored by the user. If the specified path is not a repository, an ancestor worktree may be found and the viewer root may become broader than intended. Traversal is blocked, but operators must confirm which worktree range is actually served.
  • When bound outside the default loopback address, session cookies may travel without TLS, and session_ttl_hours = 0 keeps tokens until logout. Remote use should use a TLS proxy/SSH tunnel and a restricted state directory.
  • A preview assembled for editing contains inserts supplied by the caller, so some markup originates from the caller. It is served only from a sandboxed opaque origin with connect-src 'none' and is not written to disk, so its blast radius matches previewing a repository file; however, the description "preview is exactly the repository file" does not apply to this route.
  • The editing-preview stash token is a one-time 128-bit random value and the route requires authentication, but it is not bound to the session that created the stash. The viewer has a single-trust (one password) model, so both are within the same boundary.
  • Within one authenticated session there is no isolation of pane input/resize/close between clients. Per-repository PTY and queue limits constrain process resource exhaustion, but multi-user authorization is a separate feature.

← Architecture index