agent-tui

CLI tool for AI agents to interact with TUI (Terminal User Interface) applications.

agent-tui enables AI agents to programmatically drive terminal applications by capturing screenshots and sending input—making TUI automation accessible to LLM-powered agents.

Features

• Virtual Terminal Emulation - Run TUI apps in isolated PTY sessions with full terminal emulation
• Keyboard & Text Input - Press keys, type text, or send unified input
• Wait Conditions - Wait for text or screen stability
• Output Formats - Human-readable text or JSON for automation pipelines
• Live Preview WebSocket - JSON-RPC over WebSocket for real-time UI monitoring
• Session Management - Background daemon manages multiple concurrent TUI sessions

Platform Support

agent-tui is Unix-only.

Supported runtime contract:

• OS: Linux, macOS, and other Unix-like systems with PTYs, Unix domain sockets, and POSIX signals
• Local IPC: Unix domain socket at AGENT_TUI_SOCKET, defaulting to $XDG_RUNTIME_DIR/agent-tui.sock and falling back to the system temp directory
• Persistent state: ~/.agent-tui/* by default unless overridden with AGENT_TUI_WS_STATE, AGENT_TUI_SESSION_STORE, or AGENT_TUI_UI_STATE
• Shell integration: bash, zsh, fish, and elvish
• Browser launch: $BROWSER, open on macOS, or xdg-open on other Unix desktops

Not supported:

• Native Windows runtimes
• PowerShell-specific shell integration
• Win32 path, process, or socket semantics

Installation

Quick Install

curl -fsSL https://raw.githubusercontent.com/pproenca/agent-tui/master/install.sh | bash

The installer detects your Unix platform and installs the appropriate binary to ~/.local/bin.

Package Managers

# Homebrew
brew tap pproenca/tap
brew install agent-tui
brew upgrade agent-tui

# npm
npm install -g agent-tui

# pnpm
pnpm add -g agent-tui

# bun
bun add -g agent-tui

# crates.io
cargo install agent-tui --locked

From Source

# Install from a local checkout
git clone https://github.com/pproenca/agent-tui
cd agent-tui
cargo install --path cli/crates/agent-tui --locked

# Or install directly from GitHub
cargo install --git https://github.com/pproenca/agent-tui.git --path cli/crates/agent-tui --locked

Release channel verification

Active distribution channels are GitHub Releases, install script, npm, crates.io, source install, and Homebrew. Before and after a release, verify all active channels:

just release-channel-verify 1.1.0

To check one channel at a time, run the read-only release gate from cli/:

cd cli
cargo run -p xtask -- release-channels verify --dry-run --target-version 1.1.0 --channel github-releases
cargo run -p xtask -- release-channels verify --dry-run --target-version 1.1.0 --channel install-script
cargo run -p xtask -- release-channels verify --dry-run --target-version 1.1.0 --channel npm
cargo run -p xtask -- release-channels verify --dry-run --target-version 1.1.0 --channel crates-io
cargo run -p xtask -- release-channels verify --dry-run --target-version 1.1.0 --channel source-install
cargo run -p xtask -- release-channels verify --dry-run --target-version 1.1.0 --channel homebrew

Every channel smoke should report the same agent-tui --version for the target release.

Environment Variables

Variable	Description
AGENT_TUI_INSTALL_DIR	Custom install location (default: ~/.local/bin)
AGENT_TUI_VERSION	Install specific version
AGENT_TUI_SKIP_PM	Skip package manager, use binary download
AGENT_TUI_SKIP_VERIFY	Skip checksum verification

Quick Start

# Start the daemon
agent-tui daemon start

# Run a TUI application
agent-tui run htop

# Take a screenshot
agent-tui screenshot

# Send keyboard input
agent-tui press Enter
agent-tui type "hello world"
agent-tui scroll down

# Wait for conditions
agent-tui wait "Loading complete" --assert

# Stop the session
agent-tui kill --yes

CLI Reference

For the full CLI reference (auto-generated from clap), see docs/cli/agent-tui.md.

You can also run:

• agent-tui --help
• agent-tui <command> --help

Legacy compatibility commands still parse for older automation, but they emit deprecation notices on stderr, preserve JSON stdout validity, and are planned for removal in the next major release. Prefer current commands in new scripts; see skills/agent-tui/SKILL.md for the migration table covering input, action, screenshot -e, screenshot -a, wait -e, and scroll-into-view.

Output Formats

Text (default)

Human-readable output for interactive use:

Screenshot:
<screen contents here>

JSON

Machine-readable output for automation:

agent-tui screenshot --json

{
  "session_id": "abc123",
  "screenshot": "..."
}

Configuration

Environment Variables

Variable	Description	Default
AGENT_TUI_SOCKET	IPC socket path	$XDG_RUNTIME_DIR/agent-tui.sock or temp dir fallback
AGENT_TUI_TRANSPORT	CLI transport (unix or ws)	unix
AGENT_TUI_WS_ADDR	Remote WS-RPC URL when transport is ws	-
AGENT_TUI_WS_LISTEN	Daemon WS bind address	127.0.0.1:0
AGENT_TUI_WS_ALLOW_REMOTE	Allow non-loopback WS bind	false
AGENT_TUI_WS_STATE	WS state file path	~/.agent-tui/api.json
AGENT_TUI_WS_DISABLED	Disable daemon WS server	false
AGENT_TUI_WS_MAX_CONNECTIONS	Max WS connections	32
AGENT_TUI_WS_QUEUE	WS outbound queue size	128
AGENT_TUI_SESSION_STORE	Session metadata log path	~/.agent-tui/sessions.jsonl
AGENT_TUI_UI_URL	External UI URL	-
AGENT_TUI_DETACH_KEYS	Attach detach key sequence	Ctrl-P Ctrl-B
AGENT_TUI_LOG	Log file path (optional)	-
AGENT_TUI_LOG_FORMAT	Log format (text or json)	text
AGENT_TUI_LOG_STREAM	Log output stream (stderr or stdout)	stderr
PORT	Standalone Bun web server port (web/server.ts)	-
NO_COLOR	Disable colored output	-

Architecture

agent-tui/
├── cli/                    # Rust workspace
│   └── crates/agent-tui/   # Main binary
│       ├── app/            # Application layer (CLI, handlers)
│       ├── adapters/       # Infrastructure adapters (IPC, RPC)
│       ├── domain/         # Domain models (screen, snapshot, style)
│       ├── usecases/       # Business logic (snapshot, input, wait)
│       └── infra/          # Infrastructure (daemon, terminal)
├── web/                    # Bun-based web UI
├── scripts/                # Automation scripts
└── docs/                   # Documentation

The tool follows Clean Architecture principles:

• Domain - Core models (Screen, Snapshot, Style)
• Use Cases - Business logic (screenshot, input, wait conditions)
• Adapters - External interfaces (CLI, RPC, WebSocket)
• Infrastructure - Terminal emulation, daemon runtime

See docs/ops/process-model.md for process types and deployment guidance.

Development

Prerequisites

• Rust stable (1.88+)
• Bun (for web UI)
• just (task runner)
• cargo-nextest (Rust test runner)

Build Commands

just build           # Build Rust crate
just build-release   # Optimized release build
just web-build       # Build web UI
just test            # Run the fast Rust suite with cargo-nextest
just test-core-e2e   # Run ignored real-daemon E2E tests with cargo-nextest
just ready           # Full CI checks (fmt, clippy, tests)
just lint            # Run Clippy
just format          # Format code
just doc             # Build and open docs

just test is the fast Rust lane. It uses cargo-nextest for library tests and the mock-daemon CLI contracts that should stay quick enough for normal development. just test-core-e2e is the explicit real-daemon lane: it runs the ignored system_e2e tests against real PTYs and real POSIX processes, with nextest owning retries, slow-test timeouts, and serial execution for the process-heavy group. just ready runs both Rust lanes plus the existing bash CLI smoke suite.

Future real E2E expansion should stress workflows that mocks cannot prove: shell interrupt/recovery, vi/vim editing with file persistence, top-style live redraw, resize/reflow, attach/detach interaction, multi-session isolation, and failure recovery/cleanup.

Running Locally

just dev             # Run daemon in dev mode

License

MIT

Links

• Repository
• Issues