Skip to content

Add HandyVoiceParser to delegate voice input to @cjpais/Handy - #6

Merged
tathagata merged 2 commits into
mainfrom
copilot/delegate-voice-input-handling
Mar 5, 2026
Merged

tathagata merged 2 commits into
mainfrom
copilot/delegate-voice-input-handling

Conversation

Copilot AI commented Mar 3, 2026 •

Copy link
Copy Markdown
Contributor

@cjpais/Handy doesn't expose an HTTP API, but it does provide two integration points: a CLI flag (handy --toggle-transcription) to control recording, and clipboard output (Handy pastes the transcript into the active text field when done). This PR wires those together into a HandyVoiceParser that satisfies the same speech_to_text() contract as the existing VoiceParser, making it a drop-in replacement in Orchestrator.

New: HandyVoiceParser (src/voice_parser.py)

  • trigger() — runs handy --toggle-transcription via subprocess; raises VoiceParserError if the executable is missing or exits non-zero
  • wait_for_transcript(previous_clipboard) — polls the clipboard via pyperclip until the content changes (Handy pasted the result), with a configurable timeout
  • speech_to_text() — composes the above: snapshot clipboard → trigger Handy → wait → sanitize → return
from voice_parser import HandyVoiceParser

# Drop-in replacement for VoiceParser; requires Handy running in background
parser = HandyVoiceParser()          # or HandyVoiceParser(handy_executable="/path/to/handy")
text = parser.speech_to_text()      # triggers recording, blocks until transcript arrives

# Can be passed directly to Orchestrator
orchestrator = Orchestrator(parser, claude_api, guardrails, tts_bridge)

Dependencies

  • Added pyperclip to requirements.txt for cross-platform clipboard access.

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: tathagata <68151+tathagata@users.noreply.github.com>
Copilot AI changed the title [WIP] Investigate API availability for voice input handling Add HandyVoiceParser to delegate voice input to @cjpais/Handy Mar 3, 2026
@tathagata
tathagata requested a review from Copilot March 5, 2026 03:04
@tathagata
tathagata marked this pull request as ready for review March 5, 2026 03:04
@tathagata
tathagata merged commit 3e20269 into main Mar 5, 2026
5 checks passed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces a new HandyVoiceParser implementation that integrates with the external handy CLI + clipboard to provide the same speech_to_text() interface as the existing VoiceParser, enabling it to be used as a drop-in parser in the orchestration flow.

Changes:

  • Added HandyVoiceParser to trigger Handy transcription via subprocess and retrieve transcripts by polling the clipboard.
  • Added unit tests covering HandyVoiceParser trigger, clipboard polling, and speech_to_text() behavior.
  • Added pyperclip dependency for cross-platform clipboard access.

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.

File Description
src/voice_parser.py Adds HandyVoiceParser implementation (subprocess trigger + clipboard polling + sanitization).
tests/test_voice_parser.py Adds test suite for HandyVoiceParser behavior and error cases.
requirements.txt Adds pyperclip dependency.
Comments suppressed due to low confidence (2)

src/voice_parser.py:175

  • The speech_to_text docstring says it "reuses :meth:VoiceParser.sanitize logic", but the implementation duplicates the sanitization inline. This can drift over time (and currently produces different error messaging than VoiceParser.sanitize). Consider calling VoiceParser.sanitize directly (e.g., by factoring sanitization into a shared helper/staticmethod) so both parsers stay consistent.
    def speech_to_text(self):
        """Delegate voice capture to Handy and return the transcribed text.

        Captures the current clipboard content, triggers Handy transcription,
        waits for the clipboard to be updated with the transcript, sanitizes
        the result, and returns it.

        The sanitization step reuses :meth:`VoiceParser.sanitize` logic:
        collapses internal whitespace and raises :class:`VoiceParserError` for
        empty results.
        """
        if self._pyperclip is None:
            raise VoiceParserError(
                "pyperclip is required for HandyVoiceParser. "
                "Install it with: pip install pyperclip"
            )
        try:
            previous = self._pyperclip.paste()
        except Exception as e:
            raise VoiceParserError(f"Failed to read clipboard: {e}") from e

        self.trigger()
        raw_text = self.wait_for_transcript(previous_clipboard=previous)

        if not raw_text:
            raise VoiceParserError("Handy returned an empty transcript.")
        return re.sub(r"\s+", " ", raw_text.strip())

tests/test_voice_parser.py:2

  • Unused import: call is imported from unittest.mock but not used anywhere in this test module. Removing it will keep the tests tidy and avoid future lint noise if a linter is added to CI.
from unittest.mock import MagicMock, patch, call

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/voice_parser.py
Comment on lines +133 to +142
deadline = time.monotonic() + self._timeout
while time.monotonic() < deadline:
try:
current = self._pyperclip.paste()
except Exception as e:
raise VoiceParserError(f"Failed to read clipboard: {e}") from e
if current != previous_clipboard:
logger.info("Handy transcript received from clipboard.")
return current.strip()
time.sleep(self._poll_interval)

Copilot AI Mar 5, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait_for_transcript relies on current != previous_clipboard to detect completion. If the user speaks the same command as what's already on the clipboard (or Handy produces identical text), this will never be observed as a change and will time out even though transcription succeeded. Consider using a sentinel strategy (e.g., optionally write a unique marker / clear clipboard before triggering and optionally restore), or track changes via an additional signal rather than strict inequality with the previous clipboard contents.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants