Skip to content

Android: voice mode is a control, and the only time the mic listens (#341) - #351

Merged
Khoality-dev merged 4 commits into
mainfrom
feat/341-android-voice-mode
Sep 28, 2026
Merged

Khoality-dev merged 4 commits into
mainfrom
feat/341-android-voice-mode

Conversation

@Khoality-dev

Copy link
Copy Markdown
Owner

Closes #341

The Android half of the Claude Design mockup "Kurisu - Voice Mode v1" (section 3), and the desktop's voice model (#253) with the same words: voice mode, the wake word, an interaction.

The model

  • Voice mode is a control: the headset button in the chat's top bar, or the Chats screen's strip. This device remembers it (kurisu_voice_mode, the desktop's key). It is the only time the mic listens: CoreService records only in voice mode (ACTION_SET_VOICE_MODE, ACTION_RETRY_MIC), and with it off nothing said is used.
  • It replaced "Always listen", which is gone from the ⋮ menu. AlwaysListenRemoval, a DataStore migration, deletes kurisu_asr_always_listen from the store rather than leaving it unread. Dictation is gone: speech without the wake word is ignored and never lands in the composer.
  • The wake word starts an interaction, and only in voice mode.
  • Each interaction is a new conversation. Its first message goes out with newConversation. CoreState.startNewInteraction clears the chat, the persona's cached conversation is forgotten as /clear does, and the transcript opens on a marker ("New conversation — You said “Kurisu” at 7:42 PM. The last one is in Chats.") that stays with the new conversation once the server creates it. The Chats screen opens the chat when an interaction starts, where it used to match raw transcripts itself.
  • It ends 30 s after the last reply, finished streaming and speaking. Talking holds that window shut (setUserTalking, from the VAD's start of speech until what was said is transcribed), and it reopens full.

The UI (section 3)

  • Top bar: the voice mode button is outlined when off and a filled blue circle with a ring when on, with an amber dot on a problem. The screen-reader state for the dot is "Needs attention". Under the model it says "· voice mode on", replacing "· mic …".
  • Voice bar (VoiceBar), in place of the composer with the keyboard hidden. One state at a time from voiceBarPhase, in the desktop's order:
    • waiting (Say “Kurisu” to start);
    • listening, transcribing (placeholder lines), thinking, speaking (rings on the persona);
    • the 30-second window: the top line drains, with "Ends soon unless you say something". There is no countdown; the old "idle timeout in Ns" line and secondsUntil are gone.
  • Problems, each with its fix:
    • no wake word set → Open Assistant;
    • microphone unavailable → Try again;
    • microphone access is off → Open settings, the app's Android settings page;
    • speech recognition didn't load → Retry.
  • End voice mode is a 44 dp tonal button in the bottom-right in every state. A snackbar after ending says "Voice mode is off. The mic isn't listening."
  • Colours: the voice colours are the design's, blue in dark mode too, where the theme's primary is grey.

Found on the way

  • Turning voice mode on without RECORD_AUDIO would start a microphone foreground service without the permission, which Android 14+ refuses with a SecurityException. CoreService.requestVoiceMode checks first. Without access, the bar says "Microphone access is off" and the service is not touched. The chat retries when it comes back to the foreground (onResume), e.g. from Android settings.
  • The voice bar's text had no colour of its own, so in dark mode outside a Scaffold it was near-black. It now sets LocalContentColor. Found in the emulator screenshots.

Tests

Written first (61e963c, d8fcb64, and the permission case), red before the implementation:

  • VoiceInteractionManagerTest (16, rewritten): voice mode on and off; off, nothing is used; the wake word starts an interaction in a new conversation; no dictation; the window opens only after the reply, and talking holds it.
  • VoiceBarPhaseTest (8), AlwaysListenRemovalTest (3).
  • ChatViewModelVoiceModeTest (5): the control and its memory, the off notice, the new conversation's marker, and no mic access.
  • ConversationsViewModelTest: an interaction opens the chat; a transcript alone does not.
  • VoiceBarTest (instrumented, 14): the design's copy for every state, each problem's action, End voice mode in every state, the top-bar control and its dot, the marker. ChatInputTest loses its two voice cases, since the bar is no longer inside the composer.

Locally: testDevDebugUnitTest green; VoiceBarTest + ChatInputTest green on the thesys API 35 emulator. All twenty states were captured in light and dark and checked against the mockup.

Not covered

  • The real microphone, VAD and wake word are still checked by hand, as CLAUDE.md says. No emulator test drives speech.
  • docs/screens.md's screenshots predate this and still show the old strip; the text says so.

🤖 Generated with Claude Code

https://claude.ai/code/session_014L9pxjp1MTSJe5DZG9A9sv

akn-sfu and others added 4 commits September 28, 2026 02:07
…tens (#341)

Red first:
- VoiceInteractionManager: voice mode on/off; off, nothing said is used;
  the wake word starts an interaction in a new conversation; anything
  else is ignored, never dictation; the 30-second window opens once the
  reply is done and talking holds it shut.
- voiceBarPhase: one state at a time, in the desktop's order.
- AlwaysListenRemoval: the old setting leaves the store in a migration;
  voice mode lives under the desktop's key.
- ChatViewModel: the control turns voice mode on and off and remembers
  it; turning it off says the mic is off; an interaction opens a new
  conversation on a marker.
- VoiceBar (instrumented): the design's copy, problems and their fixes,
  End voice mode in every state, the top-bar control and its dot, the
  new-conversation marker. The countdown and its test go: the design
  has none.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014L9pxjp1MTSJe5DZG9A9sv
…arts (#341)

Red first: an interaction starting (voice mode heard the wake word) opens
the chat; a transcript by itself opens nothing. The strip used to match
transcripts itself.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014L9pxjp1MTSJe5DZG9A9sv
…ervice (#341)

Red first: turning voice mode on without RECORD_AUDIO must show
"access is off" and start nothing, since a microphone foreground service
may not start without it; coming back with access allowed tries again.
The top-bar dot is said as well as drawn ("Needs attention"). The e2e
base and a persona test move to the voice mode API.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014L9pxjp1MTSJe5DZG9A9sv
…tens (#341)

The desktop's model (#253) and section 3 of the Claude Design mockup
"Kurisu - Voice Mode v1":

- A top-bar button (and the Chats strip) turns voice mode on and off;
  the device remembers it under the desktop's key. The service records
  only in voice mode. "Always listen" is gone, deleted from the store by
  a DataStore migration, and so is dictation.
- The wake word starts an interaction only in voice mode; each is a new
  conversation, opened on a marker that points to Chats.
- The voice bar replaces the composer: one state at a time
  (voiceBarPhase), what was last said, a top line that drains over the
  30-second window (no countdown), problems with their fix, and End
  voice mode in every state; a snackbar after ending.
- Talking holds the window shut; it reopens full.
- Without mic access the service is not started (Android 14+ refuses a
  microphone foreground service without it); the bar says access is
  off and the chat retries on resume.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014L9pxjp1MTSJe5DZG9A9sv
@Khoality-dev
Khoality-dev marked this pull request as ready for review September 28, 2026 09:21
@Khoality-dev
Khoality-dev merged commit f7981de into main Sep 28, 2026
5 checks passed
@Khoality-dev
Khoality-dev deleted the feat/341-android-voice-mode branch September 28, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Android: voice mode is turned on with a control, and is the only time the mic listens

2 participants