Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions apps/mobile/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,8 +154,8 @@ The shell is set by [ADR-0211](../../docs/adr/0211-the-mobile-app-has-two-tabs-w
- Opening a conversation marks it read on the Host (`channelMarkRead`) whenever the Human is at the newest message, so the chip clears on the phone and on desktop.
- Swipe for pin and mute; long-press opens a native menu.
- On iOS a long-press on a Host conversation row peeks at the chat, like Messages: the row lifts into the system context-menu preview (expo-router `Link.Preview`) showing the conversation itself, and tapping the preview opens it. A plain tap opens the conversation as before.
- A long-press on the app icon lists the three chats opened most recently (Home Screen quick actions on iOS, app shortcuts on Android), newest first, and tapping one opens that chat on its Host. The list updates whenever a chat opens and is kept with the device's saved Host list.
- **Share to a Bot.** DeepSeekBot is a target in the system share sheet (an iOS Share Extension, Android `SEND` / `SEND_MULTIPLE`). Sharing text, a link, photos, videos or files opens **Send to**: a preview of what arrived above the paired Host's chats, in the Chats order. Picking a chat opens it with the text added below any unsent draft and the files waiting as attachment tiles; nothing is sent until the Human taps Send. Cloud channels are not targets yet, and with no paired Host the screen says to pair first.
- A long-press on the app icon lists the three chats opened most recently (Home Screen quick actions on iOS, app shortcuts on Android), newest first, and tapping one opens that chat on its Host. iOS marks each with a chat-bubble symbol; Android, which has no SF Symbols, shows the app icon. The list updates whenever a chat opens and is kept with the device's saved Host list.
- **Share to a Bot.** DeepSeekBot is a target in the system share sheet (an iOS Share Extension, Android `SEND` / `SEND_MULTIPLE`). Sharing text, a link, photos, videos or files opens **Send to**: a preview of what arrived (the text, a thumbnail per image, a file tile per other file, and the count) above the paired Host's chats, in the Chats order. Opening it hides the keyboard, and it hides the keyboard again if the system brings it back while the screen is open, so the list is never covered. Files keep the order the sharing app sends them in. Picking a chat opens it with the text added below any unsent draft and the files waiting as attachment tiles; nothing is sent until the Human taps Send. Cloud channels are not targets yet, and with no paired Host the screen says to pair first.

**Count chip**

Expand All @@ -175,7 +175,7 @@ The shell is set by [ADR-0211](../../docs/adr/0211-the-mobile-app-has-two-tabs-w
- Activity rows do not navigate on tap. Long-press (or the accessibility details action) opens the message sheet with the complete plain-text content in a scrollable, selectable block capped at `messageSheet.textHeightRatio` of the window height, then Copy/Share and an explicit Open conversation, Open memory or Open new Bot action when a destination exists; there are no reactions. Consecutive posts by the same Bot into the same Channel on the same day fold into a counted row whose menu opens the latest post. Memory opens the Bot's memory files, not a commit diff. Bot-to-Bot DMs remain read-only. Keep full actor/action/detail text in the accessibility label and menu even when the compact memory line uses only the icon and commit subject.
- A top-to-bottom list (not inverted), oldest first, on one code path for iOS and Android. It opens hidden and is shown once it has been scrolled to the end and its content has settled for a frame, so it never flashes the oldest messages first. Short conversations sit at the bottom, above the composer. Older history loads when the Human nears the top (`onStartReached`), and `maintainVisibleContentPosition` keeps the messages on screen where they were while it is prepended. Messages from the same author within 5 minutes form a group, and a time separator appears after a 30-minute gap.
- At the newest message the list follows a streaming reply: content growth pins it to the end with a non-animated scroll, at most once per frame. Once the Human begins a drag or scrolls up into history, nothing moves the list under them: the visual playback cursor holds while received text continues to queue and persist. Only finishing a return gesture near the bottom or choosing the jump button resumes playback; a layout-driven scroll callback can neither rearm nor release following. Only a drag, the status-bar scroll to top or a jump to a linked message leaves the newest message, so the inset adjustment, content-size reports and `maintainVisibleContentPosition` corrections while the list first fills cannot strand it at the oldest message. A round jump button with a ⌄ then floats at the trailing edge `space.x2` above the composer, aligned with its trailing edge, like Grok (a 36pt Liquid Glass circle like **+** on iOS 26 with Reduce Transparency off, otherwise `raised`), with an accent badge counting the Bot messages that arrived since they left (99+ at most), like the Web Client's "Return to latest messages". Tapping it, sending a message, or scrolling back down to the newest message follows again and hides the button.
- **Pull up to type (iOS).** At the newest message, pulling the list further up past its end (the rubber-band bounce) and letting go focuses the composer and raises the keyboard, like pull-to-refresh in reverse. A drag counts only when it begins at the newest message (within the follow slop), so reading history never triggers it. A capsule with a keyboard symbol fades in above the composer as the pull grows, reading "Pull up to type" and switching to "Release to type" with one light haptic once the pull passes `pullToCompose.threshold` (40pt of offset, about 95pt of finger travel against the rubber band); letting go follows the newest message again first, so the keyboard does not cover it; easing back below it before release cancels. It is off where there is no composer (a Bot-to-Bot DM) or the Host is not connected. Android's overscroll stretches without moving the offset, so there is nothing to measure while the finger pulls: on Android an upward flick at the newest message (finger speed at least `pullToCompose.flick`, 1 dp/ms, with the list still at the newest message on release) focuses the composer with the same haptic, and no hint is drawn.
- **Pull up to type (iOS).** At the newest message, pulling the list further up past its end (the rubber-band bounce) and letting go focuses the composer and raises the keyboard, like pull-to-refresh in reverse. A drag counts only when it begins at the newest message (within the follow slop), so reading history never triggers it. A capsule with a keyboard symbol fades in above the composer as the pull grows, reading "Pull up to type" and switching to "Release to type" with one light haptic once the pull passes `pullToCompose.threshold` (40pt of offset, about 95pt of finger travel against the rubber band); letting go follows the newest message again first, so the keyboard does not cover it; easing back below it before release cancels. It is off where there is no composer (a Bot-to-Bot DM) or the Host is not connected. Android's overscroll stretches without moving the offset, so there is nothing to measure while the finger pulls: on Android an upward flick at the newest message (finger speed at least `pullToCompose.flick`, 0.3 dp/ms, so a calm upward swipe counts as well as a quick flick, with the list still at the newest message on release) focuses the composer with the same haptic, and no hint is drawn.
- Settings → **Message display** offers **Streaming** and **Complete bubbles**, saved on this device. Streaming uses a local playback queue with a short prebuffer and reserve, releasing graphemes on a bounded clock. Rate changes follow queue depth gradually; clauses, sentences and paragraphs add progressively longer pauses. Android fades the active prose tail on the UI thread (180ms fade, up to 72ms stagger, at most 48 animated spans, with up to eight new spans per update); earlier text stays visible. Source commitment preserves the displayed cursor and row while the rest drains, then enables message actions. Historical messages open directly. Reduce Motion skips the fade. Code/table/native-only Markdown keep their established renderer; their content still follows the playback cursor. Complete bubbles keeps the working indicator and reveals each committed Channel Message independently. Switching modes changes presentation only; received drafts, committed messages and interruption semantics remain owned by Conversation and the Host. A draft-only append updates its row subscription while the timeline snapshot and historical message references stay stable. The bottom fade belongs to the keyboard-sticky composer.
- Android Markdown uses synchronous native text/layout for prose, headings, lists, quotes, code and tables. Completed source blocks retain their keys and avoid rerendering when only the tail changes. Images and native-only extensions retain the enriched renderer. Drafts and committed messages keep the same renderer to avoid a settlement-only layout switch; iOS keeps its existing enriched implementation.
- Append-only Android Markdown reparses the changing suffix with boundary lookback, retaining completed block identities. Blocks retain the lexer result with its exact source; rendering and window-support checks reuse it only while that source matches. A repaired streaming tail or a source-only block uses a fresh parse. Corrections and document-wide references rebuild the document. Prose reveal segmentation likewise retains a grapheme boundary and revisits the changing suffix, including a joined emoji or combining mark; it must not repeatedly segment the complete paragraph.
Expand Down Expand Up @@ -222,7 +222,7 @@ The shell is set by [ADR-0211](../../docs/adr/0211-the-mobile-app-has-two-tabs-w
- **+** sits outside the capsule, bottom-aligned as the field grows. While dictating it becomes **×** (cancel), and while the mic is held, a red dot. With one trailing button (the mic or Send) the capsule is one row: the field and the button. When the text no longer fits that one row, or with more (attached files, or the mic next to Send), the capsule becomes a rounded rectangle with the same corner radius and stacks like the web composer: the field on top at full width, and a row below with expand on the left and the buttons on the right.
- **+** opens Photos, Camera and File below the composer. Picked files show as removable tiles inside the capsule, up to 10 per message. Send appears once there is text or a file.
- While the field is focused and the clipboard holds an image, a panel above the composer, styled like the @ suggestions (Liquid Glass on iOS 26, otherwise the `sheet` layer), reads "Image on the clipboard" with a paste control. On iOS it is the system Paste button (`UIPasteControl`, through `expo-clipboard`'s `ClipboardPasteButton`, image only, icon only at accessibility text sizes), so no paste-permission prompt appears. On Android it is a text `Button` "Paste image" that reads the clipboard; Android only shows its own brief "pasted from your clipboard" notice. The image attaches as a JPEG tile and the row hides until something new is copied. The @ suggestions take that slot first. A text field cannot take an image paste on its own in React Native, so this is the paste path.
- With a hardware keyboard, **⌘↩** sends on iOS and iPadOS (listed in the ⌘-hold shortcut overlay as "Send"), and on Android **Enter** sends while **Shift+Enter** adds a line. **Ctrl+Enter** also sends on Android. In the full-screen draft editor plain Enter adds a line, and only ⌘↩ or Ctrl+Enter sends. The on-screen keyboard never sends; its return key adds a line as before. The local `hardware-send` module listens on the view around the field, so it only acts while the field is focused and the draft can be sent.
- With a hardware keyboard, **⌘↩** sends on iOS and iPadOS (listed in the ⌘-hold shortcut overlay as "Send"), and on Android **Enter** sends while **Shift+Enter** adds a line. **Ctrl+Enter** also sends on Android. In the full-screen draft editor plain Enter adds a line, and only ⌘↩ or Ctrl+Enter sends. The on-screen keyboard never sends; its return key adds a line as before. Input methods that take a hardware Enter and send it again as their own key, such as WeType, still send. The local `hardware-send` module listens on the view around the field, so it only acts while the field is focused and the draft can be sent.
- Sending uploads each file to the Host first. A retry keeps the uploads that already finished.
- Nothing sits under the composer, so the chat stays clean. A DM's Model and Workspace settings live in the Bot profile, in a **Bot settings** group under the profile actions: a **Model** row (the Bot's current model name, or "Loading…", "No model set" or "Unavailable") and a **Workspace** row (the folder count, "N folders"), each with a chevron.
- **Model** opens the native Models and presets page inside the profile drawer: Orchestrator, Assignment default, allowed Assignment models, reasoning effort, restore global default, save as preset and apply a shared preset. The searchable chooser pushes inside that same drawer and returns only after an accepted save and canonical refresh. Editing only the Orchestrator preserves Assignment policy. Assignment edits retain other allowed models, efforts and notes; the current default cannot be removed until another is chosen. Existing preset copies and per-Bot plans retain the Host's revision checks. Global default updates use the Host's existing profile-default contract; mobile does not create a separate global Assignment authority.
Expand Down
11 changes: 10 additions & 1 deletion apps/mobile/modules/hardware-send/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,16 @@ draft and the send action stay in React Native, and the view only reports
`EditText`. Enter sends when `enter` is set, Shift+Enter always adds a line,
and Ctrl+Enter or Meta+Enter always sends. Key events flagged as coming from
the on-screen keyboard, or from a virtual input device, pass through
unchanged. Like `stream-reveal`, the view keeps the size Yoga gives it and
unchanged, with one exception. Some input methods, such as WeType, take a
hardware Enter and send it again as their own key event. The view notes each
hardware Enter in `dispatchKeyEventPreIme`, before the input method sees it,
and treats an input-method Enter that follows within 500 ms as that key, with
its original modifiers. Other input methods, including WeType on HyperOS,
type a newline into the field instead of sending a key. The view watches the
focused field inside it, and a single newline typed within 500 ms of a
hardware Enter that would send is removed and sends instead. When the input
method uses the Enter to confirm text it is composing, it types that text and
no newline, so nothing is sent. Like `stream-reveal`, the view keeps the size Yoga gives it and
leaves child layout to React Native.

When `enabled` is false the view does nothing, so an empty draft or a
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
package expo.modules.hardwaresend

import android.content.Context
import android.os.SystemClock
import android.text.Editable
import android.text.TextWatcher
import android.view.KeyEvent
import android.view.View
import android.view.ViewTreeObserver
import android.widget.EditText
import expo.modules.kotlin.AppContext
import expo.modules.kotlin.viewevent.EventDispatcher
import expo.modules.kotlin.views.ExpoView
Expand All @@ -17,22 +23,115 @@ internal fun sendsOnKey(keyCode: Int, meta: Int, enter: Boolean, soft: Boolean):
internal fun fromSoftKeyboard(event: KeyEvent): Boolean =
event.flags and KeyEvent.FLAG_SOFT_KEYBOARD != 0 || event.device?.isVirtual == true

internal fun insertedNewline(text: CharSequence, start: Int, before: Int, count: Int): Boolean =
before == 0 && count == 1 && start < text.length && text[start] == '\n'

internal class ImeEcho(private val window: Long) {
private var meta = 0
private var at = Long.MIN_VALUE

fun pressed(keyCode: Int, meta: Int, at: Long) {
if (keyCode != KeyEvent.KEYCODE_ENTER && keyCode != KeyEvent.KEYCODE_NUMPAD_ENTER) return
this.meta = meta
this.at = at
}

fun echoed(keyCode: Int, now: Long): Int? {
if (keyCode != KeyEvent.KEYCODE_ENTER && keyCode != KeyEvent.KEYCODE_NUMPAD_ENTER) return null
if (at == Long.MIN_VALUE || now - at > window) return null
return meta
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

fun clear() {
at = Long.MIN_VALUE
}
}

class HardwareSendView(context: Context, appContext: AppContext) : ExpoView(context, appContext) {
var sending = false
var enter = false
private val onSend by EventDispatcher()

private val echo = ImeEcho(ECHO_WINDOW)

override fun dispatchKeyEventPreIme(event: KeyEvent): Boolean {
if (event.action == KeyEvent.ACTION_DOWN && !fromSoftKeyboard(event)) {
echo.pressed(event.keyCode, event.metaState, event.eventTime)
}
return super.dispatchKeyEventPreIme(event)
}

override fun dispatchKeyEvent(event: KeyEvent): Boolean {
if (!sending || !sendsOnKey(event.keyCode, event.metaState, enter, fromSoftKeyboard(event))) {
val soft = fromSoftKeyboard(event)
val meta = if (soft) echo.echoed(event.keyCode, SystemClock.uptimeMillis()) else event.metaState
if (!sending || meta == null || !sendsOnKey(event.keyCode, meta, enter, false)) {
return super.dispatchKeyEvent(event)
}
if (event.action == KeyEvent.ACTION_DOWN && event.repeatCount == 0) onSend(mapOf<String, Any>())
if (event.action == KeyEvent.ACTION_UP) echo.clear()
return true
}

private var watched: EditText? = null
private var newline = -1

private val watcher = object : TextWatcher {
override fun beforeTextChanged(text: CharSequence, start: Int, count: Int, after: Int) = Unit

override fun onTextChanged(text: CharSequence, start: Int, before: Int, count: Int) {
if (!sending || !insertedNewline(text, start, before, count)) return
val meta = echo.echoed(KeyEvent.KEYCODE_ENTER, SystemClock.uptimeMillis()) ?: return
if (sendsOnKey(KeyEvent.KEYCODE_ENTER, meta, enter, false)) newline = start
}

override fun afterTextChanged(text: Editable) {
val at = newline
if (at < 0) return
newline = -1
echo.clear()
if (at < text.length && text[at] == '\n') text.delete(at, at + 1)
onSend(mapOf<String, Any>())
}
}

private val focus = ViewTreeObserver.OnGlobalFocusChangeListener { _, next -> watch(next) }

private fun watch(next: View?) {
val field = (next as? EditText)?.takeIf { isAncestorOf(it) }
if (field === watched) return
watched?.removeTextChangedListener(watcher)
watched = field
field?.addTextChangedListener(watcher)
}

private fun isAncestorOf(view: View): Boolean {
var parent = view.parent
while (parent != null) {
if (parent === this) return true
parent = parent.parent
}
return false
}

override fun onAttachedToWindow() {
super.onAttachedToWindow()
viewTreeObserver.addOnGlobalFocusChangeListener(focus)
watch(findFocus())
}

override fun onDetachedFromWindow() {
viewTreeObserver.removeOnGlobalFocusChangeListener(focus)
watch(null)
super.onDetachedFromWindow()
}

override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
setMeasuredDimension(MeasureSpec.getSize(widthMeasureSpec), MeasureSpec.getSize(heightMeasureSpec))
}

private companion object {
const val ECHO_WINDOW = 500L
}

override fun onLayout(changed: Boolean, left: Int, top: Int, right: Int, bottom: Int) = Unit
}
Loading
Loading