Repository navigation
feat: play quoted Lark voice messages in Channel bubbles #3354
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| --- | ||
| section: Added | ||
| --- | ||
|
|
||
| ## en | ||
|
|
||
| - Prepare and play a Lark voice message quoted by a Bot-mentioned request in its Channel bubble: the Host decodes the Opus original to WAV only on explicit request, shows the platform duration and keeps the original downloadable ([#1023](https://github.com/BotHarness/DeepSeekBot/issues/1023)). | ||
|
|
||
| ## zh | ||
|
|
||
| - 在 Channel 请求气泡内准备并播放被 @ Bot 请求引用的 Lark 语音：Host 仅在明确操作后将 Opus 原件解码为 WAV，显示平台时长，原件仍可下载（[#1023](https://github.com/BotHarness/DeepSeekBot/issues/1023)）。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # Lark parent-voice tracer — #1023 audio slice | ||
|
|
||
| ## Scope | ||
|
|
||
| The audio slice of [#1023](https://github.com/BotHarness/DeepSeekBot/issues/1023), after the video slice ([#1950](https://github.com/BotHarness/DeepSeekBot/pull/1950)). A native text request that mentions the receiving App and replies to a native Lark `audio` message carries that parent's exact voice descriptor. Direct native audio messages (without a quoting request) and speech recognition remain outside this slice. | ||
|
|
||
| Provider: [dsh-im#35](https://github.com/DoodleBears/dsh-im/pull/35), source `bb0c7956896f5f5845950a0707837cd7938a11b3`, 425 runtime files with SHA-256 `b1df502e022b25e474a691b8c99517f7f6a7222b308c7f3fe2aff23232540751`. | ||
|
|
||
| ## Implemented path | ||
|
|
||
| Provider `sourceVoiceAudio` opt-in → one `audio/unknown` attachment `voice.opus` bound to the parent message and `voice: { transcript: 'unavailable', itemId, durationMs? }` → Core accepts it only as the exact quoted parent (`isLarkVoice`) and with `source-voice-audio-checked` → Channel projection exposes an `audio` item and the platform duration → explicit **Prepare playback** → existing `prepareAudio` owner reads the original under current source authority and decodes Ogg Opus to 24 kHz mono WAV in a bounded worker (`opus-decoder`, 1 MiB input, 12 MiB PCM, 10 s) → browser playback. The original stays downloadable; other containers are refused with `audio-codec-unsupported`. | ||
|
|
||
| ## Native qualification — 2026-11-03 JST | ||
|
|
||
| Isolated Host on DSH `0.2.1-rc.3`, model DeepSeek V4.1 Flash, app `BotHarness Lark QA1`, group "BotHarness QA1 Test" (test account and Bot only). A 4.9 s synthetic Opus voice was sent as a native `audio` message through the Bot API; a real Human reply in the Lark web client quoted it and mentioned the Bot through the native @ picker. | ||
|
|
||
| - Intake: the Channel message carried one `audio` item `voice.opus` and `voice: { transcript: 'unavailable', durationMs: 4916 }`. No attachment file existed on the Host before preparation, and the Web Client sent no attachment request before **Prepare playback**. | ||
| - Playback: after preparation Chromium played the WAV (`duration` 4.907 s, time advanced 1.3 s in 1.5 s, seek to 3.5 s, no media error) in light and dark. | ||
| - CLI: `channel-media-download` returned the original byte-identical to the sent file (`156874cb…`, 43,237 bytes); with `--representation playback` it returned `audio/wav` (235,580 bytes). | ||
| - The Bot replied once in Lark and said no transcript was available. | ||
|
|
||
| The run found one defect, now fixed: the Client parser accepted bridged voice only from QQ, so the Lark bubble showed text without the audio card. | ||
|
|
||
| Not exercised: a voice recorded by a Human in a Lark client (the browser tool cannot record audio), the installed product and the mobile app. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| import { MessagingError } from '../messaging/provider.js'; | ||
| import { decodeInWorker, MAX_VOICE_INPUT_BYTES } from './voice-audio.js'; | ||
|
|
||
| const MAX_PACKETS = 20000; | ||
|
There was a problem hiding this comment.
 |
||
|
|
||
| interface OggOpus { | ||
| channels: number; | ||
| preSkip: number; | ||
| packets: Uint8Array[]; | ||
| } | ||
|
|
||
| export function oggOpusPackets(bytes: Uint8Array): OggOpus { | ||
| const ascii = (offset: number, length: number) => | ||
| new TextDecoder().decode(bytes.subarray(offset, offset + length)); | ||
| if (bytes.length < 28 || ascii(0, 4) !== 'OggS') | ||
| throw new MessagingError('audio-codec-unsupported'); | ||
| const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); | ||
| const packets: Uint8Array[] = []; | ||
| let pending: Uint8Array[] = []; | ||
| let serial: number | undefined; | ||
| let offset = 0; | ||
| while (offset < bytes.length) { | ||
| if (offset + 27 > bytes.length || ascii(offset, 4) !== 'OggS' || bytes[offset + 4] !== 0) | ||
| throw new MessagingError('audio-decode-failed'); | ||
| const pageSerial = view.getUint32(offset + 14, true); | ||
| if (serial !== undefined && pageSerial !== serial) | ||
| throw new MessagingError('audio-codec-unsupported'); | ||
| serial = pageSerial; | ||
| const segments = bytes[offset + 26]!; | ||
| let data = offset + 27 + segments; | ||
| if (data > bytes.length) throw new MessagingError('audio-decode-failed'); | ||
| for (let index = 0; index < segments; index++) { | ||
| const size = bytes[offset + 27 + index]!; | ||
| if (data + size > bytes.length) throw new MessagingError('audio-decode-failed'); | ||
| pending.push(bytes.subarray(data, data + size)); | ||
| data += size; | ||
| if (size < 255) { | ||
| const packet = new Uint8Array(pending.reduce((total, part) => total + part.length, 0)); | ||
| let at = 0; | ||
| for (const part of pending) { | ||
| packet.set(part, at); | ||
| at += part.length; | ||
| } | ||
| pending = []; | ||
| if (packet.length) packets.push(packet); | ||
| if (packets.length > MAX_PACKETS + 2) throw new MessagingError('audio-too-large'); | ||
| } | ||
| } | ||
| offset = data; | ||
| } | ||
| const head = packets[0]; | ||
| if ( | ||
| !head || | ||
| head.length < 19 || | ||
| new TextDecoder().decode(head.subarray(0, 8)) !== 'OpusHead' || | ||
| head[18] !== 0 || | ||
| (head[9] !== 1 && head[9] !== 2) | ||
| ) | ||
| throw new MessagingError('audio-codec-unsupported'); | ||
| const tags = packets[1]; | ||
| if (!tags || new TextDecoder().decode(tags.subarray(0, 8)) !== 'OpusTags' || packets.length < 3) | ||
| throw new MessagingError('audio-decode-failed'); | ||
| return { | ||
| channels: head[9]!, | ||
| preSkip: head[10]! | (head[11]! << 8), | ||
| packets: packets.slice(2), | ||
| }; | ||
| } | ||
|
|
||
| const DECODE_WORKER = ` | ||
| const { parentPort, workerData } = require('node:worker_threads'); | ||
| (async () => { | ||
| try { | ||
| const { OpusDecoder } = await import(workerData.moduleUrl); | ||
| const decoder = new OpusDecoder({ | ||
| channels: workerData.channels, preSkip: workerData.preSkip, sampleRate: 24000, | ||
| }); | ||
| await decoder.ready; | ||
| const result = decoder.decodeFrames(workerData.packets); | ||
| decoder.free(); | ||
| const frames = result.samplesDecoded; | ||
| if (result.errors.length || !frames || frames * 2 > workerData.maxBytes) | ||
| throw new Error('bounded output'); | ||
| const pcm = new Uint8Array(frames * 2); | ||
| const view = new DataView(pcm.buffer); | ||
| for (let frame = 0; frame < frames; frame++) { | ||
| let sample = 0; | ||
| for (const channel of result.channelData) sample += channel[frame]; | ||
| sample = Math.max(-1, Math.min(1, sample / result.channelData.length)); | ||
| view.setInt16(frame * 2, Math.round(sample * 32767), true); | ||
| } | ||
| parentPort.postMessage({ pcm }); | ||
| } catch { parentPort.postMessage({ failed: true }); } | ||
| })();`; | ||
|
|
||
| export async function decodeLarkVoice(bytes: Uint8Array, signal: AbortSignal): Promise<Uint8Array> { | ||
| signal.throwIfAborted(); | ||
| if (!bytes.length || bytes.length > MAX_VOICE_INPUT_BYTES) | ||
| throw new MessagingError('audio-too-large'); | ||
| const { channels, preSkip, packets } = oggOpusPackets(bytes); | ||
| return decodeInWorker( | ||
| DECODE_WORKER, | ||
| { module: 'opus-decoder', channels, preSkip, packets }, | ||
| signal, | ||
| ); | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct the preparation download limit.
Preparation can acquire the original under the attachment store’s 25 MiB default limit before
prepareAudiochecks the 1 MiB decoding limit. A larger voice can therefore be downloaded and retained even though playback preparation fails. Update this guidance and the equivalent text indocs/lark-connection.zh.md:478. (raw.githubusercontent.com)