feat(input): route key encoding to the backend, and honor DECCKM

[Ayman Bagabas (aymanbagabas)]

Routes key encoding to a backend's own encoder where one exists, and fixes a bug that finding out about surfaced.

The bug: no DECCKM at all

keys.rs modelled the Kitty protocol flags and nothing else. CSI ?1h (DECCKM) moves the cursor keys from CSI A onto SS3 A, and readline, vim, and less all set it — so key press Up has been sending CSI A to children that asked for SS3 A.

It reaches exactly the keys CSI ... A-D, H, and F spell: the four arrows plus Home and End. Only an unmodified press changes, which is already what ss3_unmodified means for F1-F4, so it reuses that path rather than adding a second rule that could drift from it.

Two cases deliberately do not change:

• A modified cursor key stays CSI. Ctrl+Up is CSI 1;5A, and SS3 has nowhere to put the modifier parameter.
• Kitty outranks it. The Kitty protocol replaces the legacy encoding outright and its cursor keys are CSI with a parameter, so DECCKM does not apply.

The routing

Emulator::encode_key(&KeyPress) -> Option<Vec<u8>>, defaulting to None. A backend that has an encoder uses it; None falls back to the shared encoder.

backend	own encoder?	why
ghostty	yes	ghostty_vt::key::Encoder
alacritty	no	encoding lives in the alacritty GUI crate, not alacritty_terminal
rio	no	same split; rio-vt has only hex and graphics encoders
xtermjs	no	evaluateKeyboardEvent is in browser/, absent from the headless bundle

Preferring the backend matters because encoding depends on more terminal state than the shared encoder models. Ghostty's set_options_from_terminal reads the live terminal and applies keypad application mode, modifyOtherKeys, and the alt-escape prefix — none of which are visible through the Emulator trait, and none of which keys.rs has ever handled.

Declining is a feature. Ghostty's modifier bitmask has no Hyper or Meta, which the Kitty protocol does define. Rather than encode without them, encode_key returns None and the event falls back, so a modifier is never silently dropped. Same for a key ghostty has no physical code for.

Fallback is all-or-nothing per token. A token half-encoded by the backend and half by the fallback would interleave two encodings of the same keypress.

How the two halves fit

They meet rather than overlap. Ghostty encodes its own DECCKM inside its encoder; the shared encoder's new DECCKM is what the other three backends get. Without the second half, this PR would have made ghostty correct in application cursor mode and left the other three wrong — a divergence introduced by a fix. With both, all four agree.

Shape of the change

InputModes groups the terminal state encoding branches on:

pub struct InputModes {
    pub keyboard: KeyboardMode,
    pub cursor_key_application: bool,
}

Public entry points take impl Into<InputModes> with From<KeyboardMode>, so all existing callers and the 32 existing test call sites compile untouched.

Emulator::cursor_key_application() is new and implemented by all four backends: TermMode::APP_CURSOR (alacritty), Mode::APP_CURSOR (rio), Mode::DECCKM (ghostty), and modes.applicationCursorKeysMode (xterm.js, public API).

Tests

Five DECCKM cases in keys.rs: application mode uses SS3, normal mode is unchanged, a modified cursor key stays CSI, Kitty outranks the mode, and the mode reaches only the cursor keys (F1 stays SS3 either way, F5 and PageUp are untouched).

Four routing cases against a real ghostty terminal, which is the only way to show the routing is live rather than merely wired: the encoder follows CSI ?1h and CSI ?1l, it follows negotiated Kitty flags, Hyper/Meta decline, and an unmapped key declines.

cargo test --workspace --features tui-test-rs/ghostty,tui-test-rs/rio,tui-test-rs/xtermjs gives 419 passed, 0 failed.

Related

Stacked on #180. It started out independent, but the two features interact: set_options_from_terminal reads ghostty's Kitty flags straight off the live terminal, which is the reason to route encoding there at all and also goes around the profile. A session with kitty_keyboard = false reported no flags and then encoded Escape as CSI 27u anyway. Neither branch could show that alone. fix(ghostty): honor a disabled Kitty profile in the backend encoder clears the flags out of the encoder after the terminal has had its say, with a regression test.

Independent of #181. #183 stacks on this one.

---

Update: four bugs found by the follow-up oracle

The differential test in #183 was written against this branch and immediately found four bugs in it, now fixed here in fix(input): carry the text a key produces to the backend encoder:

symptom	cause
key press A sent a	tokens were lowercased wholesale before reaching the backend — right for a named key, wrong for a character
key press Space sent nothing	ghostty encodes a text-bearing key with no utf8 set as the empty string, and only single-character tokens were setting it
key press Alt+a sent a	set_options_from_terminal resets macos_option_as_alt to False; with it off Alt is Option and composes text instead of prefixing ESC
Shift+a sent CSI 97;2u instead of A	consumed_mods was never set, so ghostty reported a Shift that had been spent producing the text

KeyPress now carries key as the unshifted key plus the text it produces, so ! arrives as "1" with shift and text "!". Text is carried rather than derived because deriving it needs a keyboard layout — Shift+1 is ! on a US layout and " on a UK one. token_to_presses now classifies a token exactly as token_to_seq_for_action_with_mode does, so both encoders receive the same events rather than two readings of one token.

These are a good argument for reviewing #183 alongside this PR: without it, the first three would have shipped.

Update 2: Shift+a reported a spent modifier

Shift that goes into producing a character is consumed: it made A out of a and is not a modifier to report alongside it. Without saying so via set_consumed_mods, Shift+a encoded as CSI 97;2u where the Kitty spec has a text-producing key still send its text when only disambiguation is on. Also found by #183.