diff --git a/.github/workflows/README.md b/.github/workflows/README.md index efc8091..b2e2869 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -120,3 +120,9 @@ recordings, private trust setup, source-bound artifacts and attended hardware wo CI establishes source/build behavior, not audible playback or hardware recovery. See the [dated device results](../../docs/development/feasibility-report.md). + +The device job also runs `tools/check_quran.py --arduinojson` against its +resolved ArduinoJson headers. This compiles the production metadata worker and +shared redirect policy under UBSan with deterministic HTTP/task adapters. It +checks cancellation, Athan priority, URL restrictions, paging/search, malformed +responses and unavailable recordings; it does not establish physical streaming. diff --git a/.github/workflows/firmware.yml b/.github/workflows/firmware.yml index a97cf2d..a969ac6 100644 --- a/.github/workflows/firmware.yml +++ b/.github/workflows/firmware.yml @@ -128,6 +128,7 @@ jobs: cmake --build build --target openathan_core --parallel 2 python -m unittest discover -s tests -p test_settings_adapter.py -v python tools/check_upgrade_runtime.py --arduinojson "$OPENATHAN_ARDUINOJSON_INCLUDE" + python tools/check_quran.py --arduinojson "$OPENATHAN_ARDUINOJSON_INCLUDE" - name: Test update request formatting with the resolved SDK if: matrix.configuration == 'reference' && (steps.build_policy.outcome != 'success' || steps.build_policy.outputs.build_required != 'false') env: diff --git a/CMakeLists.txt b/CMakeLists.txt index 4b5b247..859ba91 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -32,6 +32,7 @@ add_executable(audio_validator tests/audio_validator.cpp) target_include_directories(audio_validator PRIVATE lib/openathan-core/include) add_executable(audio_adapter_tests tests/audio_adapter_tests.cpp firmware/esphome/components/openathan_audio/partition_audio.cpp) +target_compile_definitions(audio_adapter_tests PRIVATE OPENATHAN_QURAN) target_include_directories(audio_adapter_tests PRIVATE tests/stubs lib/openathan-core/include firmware/esphome/components/openathan_audio) if(NOT APPLE) diff --git a/PRODUCT.md b/PRODUCT.md index 2b3ad08..4179337 100644 --- a/PRODUCT.md +++ b/PRODUCT.md @@ -25,7 +25,8 @@ OpenAthan brings scheduled Athan playback into the home through an open-source, low-cost DIY speaker. The immediate product is standalone Athan: calculate prayer times locally, play the appropriate normal or Fajr recording, and let the household review and change its settings through a local browser interface. -Quran playback and adhkar are future capabilities. +Development firmware adds owner-requested MP3Quran surah streaming through the +local interface, pending physical qualification. Adhkar remains future scope. Success means a household can configure its location and prayer conventions, understand when announcements will play, and control playback with confidence. @@ -93,8 +94,10 @@ alone cannot establish the current time after a cold boot. [current release evidence](docs/development/release-validation-2026-10-03.md). The optional GC9107 status display is a post-v0.2.1 development candidate; [bounded isolated hardware acceptance](docs/development/display-validation-2026-10-03.md) - is recorded; successor release qualification remains separate. Quran, adhkar, - touch adapters, + is recorded; successor release qualification remains separate. + [MP3Quran streaming](docs/development/quran-streaming.md) is also a development + candidate, with single-surah playback and Athan interruption without automatic + resume. Physical playback and runtime memory remain unqualified. Adhkar, touch adapters, expanded storage and Home Assistant integration remain future scope in the [roadmap](docs/development/roadmap.md). - Software, documentation, and original hardware designs retain their respective diff --git a/README.md b/README.md index c0e7ce2..3688e2f 100644 --- a/README.md +++ b/README.md @@ -107,7 +107,12 @@ separates shipped features from remaining work and validation limits. Displays and their saved brightness/time-format controls are optional capabilities; touch controls remain future work. None is required by the reusable scheduler. -Quran streaming, reciter and passage selection, resume position, adhkar, offline Quran storage, RTC expansion for other hardware, and optional Home Assistant integration are later work. Streaming audio should use PSRAM for buffering; Quran audio is not expected to fit in the AtomS3R's flash. +Development firmware adds [MP3Quran streaming](docs/development/quran-streaming.md) +with reciter, recording edition and surah selection in the local UI. One surah +plays directly on the device; Athan interrupts it without automatic resume. +Physical playback and runtime memory qualification remain release gates. +Resume positions, ayah selection, adhkar, offline Quran storage, RTC expansion +for other hardware and optional Home Assistant integration remain later work. ## Development safety diff --git a/docs/development/architecture.md b/docs/development/architecture.md index 83e1fc5..d17da7b 100644 --- a/docs/development/architecture.md +++ b/docs/development/architecture.md @@ -140,6 +140,17 @@ for serial ownership, authentication, persistence, and hardware acceptance limit The generic ESPHome web UI may be useful during development but is not the intended permanent product interface. +### Quran streaming + +The product layer's optional Quran service resolves MP3Quran reciter, edition and +surah identifiers on a bounded background task. Playback adapters expose optional +streaming, source/state and session-epoch methods; the portable scheduler has no +provider or network dependency. The existing announcement pipeline plays both +stored Athan and streamed Quran, with Athan taking priority and no automatic +Quran resume. The shared-audio partition is unchanged. See the +[streaming contract](quran-streaming.md) for HTTPS restrictions, local APIs, +the pinned reader patch and separate hardware qualification gates. + ### Upgrade qualification boundary The production updater remains the single update engine. The separately selected diff --git a/docs/development/quran-streaming-validation-2026-10-10.md b/docs/development/quran-streaming-validation-2026-10-10.md new file mode 100644 index 0000000..8104d93 --- /dev/null +++ b/docs/development/quran-streaming-validation-2026-10-10.md @@ -0,0 +1,302 @@ +# Quran streaming implementation validation — 2026-10-10 + +MP3Quran single-surah streaming is a development implementation. Automated checks +cover metadata resolution, cancellation, Athan priority and local controls. +Physical streaming, audible output and concurrent runtime memory remain +unqualified. This report does not establish release readiness. + +## Verification of playback request race fixes — 2026-10-11 + +These fixes compare PR #43 at `aebc1d094ddf6dfb2287c790a5062a4c03c508dc` +with measured firmware, UI and regression source +`7e1d505c5d9b9151875a6f85085ab8dbdad9774a`, rebased onto unchanged main +`e4304465de98c925c34820b945b2549854a4bd63`. All 409 tracked measured source +files were verified against that measured revision. This report update changes +documentation only; the earlier review and original implementation results below +remain historical comparisons. + +Production firmware and UI code are unchanged after the measured revision. The +final browser fixture source is +`36478f31fb735863387a0aaeb0474ef8f549131c`: its prayer-occurrence cases explicitly +gate Stop readback and await confirmed Stop before releasing an older Skip/Restore +reply. The initial CI run passed 531/532 cases and exposed the fixture's earlier +assumption that an updated prayer name already implied completed Stop. All state +and feedback assertions are retained. The complete local browser results below +were recorded on the measured source; 28/28 affected cases passed with the final +fixture. Current-head complete suites are enforced by +[PR #43's checks](https://github.com/OpenAthan-Project/openathan/pull/43/checks). + +Play now requires the transient `quran.playback_token` observed from the device. +The per-boot identifier and playback counter reject delayed requests invalidated +by Stop, a new Athan or reboot before altering audio or allocating work. Missing +or stale tokens return HTTP 409; malformed payloads return HTTP 400. Older cached +clients must reload. `/api/stop` and durable storage are unchanged. + +The browser owns playback intent separately from catalog loading. Stop stays +reachable in Today, Settings and Quran while Play is pending or unconfirmed, and +supersedes Play callbacks. Every current Play/Stop outcome triggers a new status +read; earlier polls cannot confirm the command or replace its later playback +observation. Concurrent settings and Skip observations retain their own ordering. +An uncertain Play is never repeated, and unchanged-token idle cannot resolve it. +Stop confirmation requires inactive playback and invalidation of a pending Play +token. Missing playback fields keep Play pending until fresh state or Stop resolves +it. Browser storage remembers selection IDs only. + +| Check | Result | +| --- | --- | +| CMake/CTest host suite under UBSan | 18/18 passed. | +| Complete pinned Python suite | 113/113 passed without skips. | +| Complete browser suites | Chromium 266/266 and WebKit 266/266 passed with Node 24.19.0, Playwright 1.62.1 and the production Digest bridge. | +| Focused browser regressions | 82/82 passed on measured source; 28/28 final fixture and playback-ordering cases passed across Chromium and WebKit. | +| Production Quran worker and HTTPS helper under UBSan | Passed token validation, simulated physical/other-client Stop, reboot invalidation, counter wrap, pending/active Athan priority and existing worker cancellation checks. | +| Production and isolated local API tests | Passed token forwarding/status exposure, HTTP 409 propagation and durable-record preservation. | +| ESPHome configuration checks | All 13 CI configurations passed. | +| Pinned installer contract | Both hardware profiles passed without device access. | +| Resolved ESP-IDF HTTP formatter | Complete request-line/header transmission regressions passed. | + +Browser regressions cover Stop arriving before a delayed Play from all three +views, the actual five-second idle poll before Play acknowledgement, delayed +pre-acknowledgement reads, obsolete successful replies, malformed/lost Play and +Stop replies, incomplete or failed readback and reconnect. They assert visible +controls and feedback, final simulated device state, no automatic repeat Play, +and no corrective Stop that could interrupt a newly due Athan. + +Both original race scenarios fail against exact baseline UI assets in Chromium +and WebKit. The temporary comparison fixture accepts the baseline three-ID Play +API: a delayed Play is admitted after Stop, and an idle poll removes Stop while +Play remains pending. Bypassing only that early visibility assertion also proves +the later acknowledgement cannot replace the baseline's idle observation. The +permanent regression assertions remain intact on the fixed source. + +### OTA comparison for playback request race fixes + +All ten before/after variants use identical Python 3.13.5, ESPHome 2026.9.0, +tzdata 2026.4, ESP-IDF 5.5.5, source/build paths, configurations, substitutions, +compile-only settings and resolved dependencies. Each actual `firmware.ota.bin` +passed the existing capacity checker. Both 2,097,152-byte application slots and +the separate 3,670,016-byte shared-audio partition are retained. Synthetic audio +fixtures were used only for compilation and must not be installed or played. + +| Variant | Unfixed OTA bytes | Fixed OTA bytes | Fix delta | Budget remaining | Slot free | Static RAM bytes (fix delta) | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| Atom reference | 1,299,248 | 1,301,456 | +2,208 | 271,408 | 795,696 | 115,675 (+24) | +| Waveshare production | 1,307,232 | 1,309,696 | +2,464 | 263,168 | 787,456 | 116,171 (+24) | +| Scheduler device | 1,121,520 | 1,121,520 | 0 | 451,344 | 975,632 | 113,231 (0) | +| Scheduler validation | 1,139,056 | 1,139,056 | 0 | 433,808 | 958,096 | 113,943 (0) | +| Provisioning validation | 1,300,688 | 1,302,992 | +2,304 | 269,872 | 794,160 | 115,675 (+24) | +| Upgrade qualification | 1,305,536 | 1,307,840 | +2,304 | 265,024 | 789,312 | 115,747 (+24) | +| Upgrade startup failure | 1,305,536 | 1,307,840 | +2,304 | 265,024 | 789,312 | 115,747 (+24) | +| Waveshare audio | 1,235,904 | 1,238,208 | +2,304 | 334,656 | 858,944 | 113,747 (+24) | +| Waveshare display | 1,285,632 | 1,287,920 | +2,288 | 284,944 | 809,232 | 116,171 (+24) | +| Waveshare diagnostics | 1,372,464 | 1,374,768 | +2,304 | 198,096 | 722,384 | 116,831 (+24) | + +The smallest remaining application budget is 198,096 bytes against the existing +1,572,864-byte limit. Quran-enabled OTA images grow by 2,208–2,464 bytes; the two +generic scheduler images are unchanged. Embedded gzip assets total 26,342 bytes +versus 25,494 bytes on the unfixed baseline, +848 bytes. No dependencies or +recordings were added. Static RAM grows by 24 bytes on affected variants; this +does not establish runtime heap, fragmentation, PSRAM or stack headroom. + +No firmware was installed and no hardware was accessed. Physical streaming, +audible output, real device TLS/decoding, long-surah behavior, physical Stop/Athan +interruption and concurrent runtime-memory measurements remain separate release +gates. These automated checks do not establish release readiness. + +## Verification of earlier review fixes — 2026-10-11 + +The baseline for these fixes is PR #43 at +`36fd0a5db639a4ee10ddb3be7e415ae609c6dab5`. Final firmware, UI and regression +sources are `2eaa4436df07a96bc2589dfc0b014b9c4baa9248`, rebased onto unchanged +main `e4304465de98c925c34820b945b2549854a4bd63`. This report update changes +documentation only. All 409 tracked files in each exported source snapshot were +verified against their respective Git revision before compilation. + +| Check | Result | +| --- | --- | +| CMake/CTest host suite under UBSan | 18/18 passed. | +| Complete pinned Python suite | 113/113 passed without skips; final audio adapter regressions also passed the seven audio-image tests after adding stored-start cancellation cases. | +| Complete browser suites | Chromium 255/255 and WebKit 255/255 passed with Node 24.19.0, Playwright 1.62.1 and the production Digest bridge. | +| Final Quran browser cases | 22/22 passed across both browsers after bounding test response waits; application/UI sources were unchanged from the complete run. | +| Production Quran worker and HTTPS helper under UBSan | Passed, including rejection during pending and active Athan ownership. | +| ESPHome configuration checks | All 13 CI configurations passed. | +| Pinned installer contract | Both hardware profiles passed the real producer/consumer checks without device access. | +| Pinned ESPHome mirror verification | All 20 upstream manifest hashes match ESPHome 2026.9.0; only the two documented patched files differ locally. Both copied license files are unchanged. | + +Audio adapter regressions cover outgoing playback, draining one command at a time, +independently delayed pipeline stops, new playback and natural completion. They +also cover direct stored starts, rapid replacements, invalid-request preservation, +pending stored/stream cancellation, speaker failure and recovery, reader/decoder +errors before and after submission, and the startup deadline including Stop wait. +The production worker rejects Quran Play while Athan owns a pending start even +when the speaker reports idle, and continues rejecting it during active playback. + +Browser regressions cover editable search during delayed or failed edition +loading, multiple edits while busy, stale responses after returning to the same +query, pending/uncertain Play, fresh status and Athan gating, changed-query +pagination, explicit failure recovery and appending only to an accepted query. +The permanent pagination regression fails on the preserved baseline in both +browsers: it requests offset 20 for the new query instead of zero. It passes on +the candidate. These checks use simulated pipelines and HTTP responses, not +physical decoding, TLS or listening. + +### OTA comparison for review fixes + +All ten baseline and candidate variants were compiled with identical Python +3.13.5, ESPHome 2026.9.0, tzdata 2026.4, ESP-IDF 5.5.5, source/build paths, +configurations, substitutions and compile-only settings. Capacity checks passed +for every actual `firmware.ota.bin`; resolved dependency pins, both 2 MiB slots +and the separate 3.5 MiB shared-audio partition remain unchanged. Fixtures are +synthetic and must not be installed or played. + +| Variant | PR baseline OTA bytes | Fixed OTA bytes | Fix delta | Budget remaining | Slot free | Static RAM bytes (fix delta) | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| Atom reference | 1,298,640 | 1,299,248 | +608 | 273,616 | 797,904 | 115,651 (+32) | +| Waveshare production | 1,306,640 | 1,307,232 | +592 | 265,632 | 789,920 | 116,147 (+32) | +| Scheduler device | 1,121,520 | 1,121,520 | 0 | 451,344 | 975,632 | 113,231 (0) | +| Scheduler validation | 1,139,056 | 1,139,056 | 0 | 433,808 | 958,096 | 113,943 (0) | +| Provisioning validation | 1,300,096 | 1,300,688 | +592 | 272,176 | 796,464 | 115,651 (+32) | +| Upgrade qualification | 1,304,912 | 1,305,536 | +624 | 267,328 | 791,616 | 115,723 (+24) | +| Upgrade startup failure | 1,304,912 | 1,305,536 | +624 | 267,328 | 791,616 | 115,723 (+24) | +| Waveshare audio | 1,235,296 | 1,235,904 | +608 | 336,960 | 861,248 | 113,723 (+24) | +| Waveshare display | 1,285,024 | 1,285,632 | +608 | 287,232 | 811,520 | 116,147 (+32) | +| Waveshare diagnostics | 1,371,888 | 1,372,464 | +576 | 200,400 | 724,688 | 116,807 (+24) | + +The smallest remaining budget against the 1,572,864-byte limit is 200,400 bytes. +Embedded gzip assets total 25,494 bytes versus 25,104 on the PR baseline, +390 +bytes. There are no new dependencies or bundled recordings. A relocated cached +bootloader CMake project initially prevented the baseline startup-failure build; +regenerating that disposable cache resolved it, and the table uses the successful +run. The generic firmware images have no byte growth. + +Static RAM increases of 24 or 32 bytes do not establish runtime heap, +fragmentation, PSRAM or stack headroom. The owned pending URL is bounded, but its +runtime allocation and concurrent streaming still need physical qualification. +No firmware was installed, no hardware was accessed, and no merge or release was +performed for these fixes. HTTP APIs, durable settings, prayer history and +shared-audio storage are unchanged. The sole added interface is the guarded +internal speaker observation helper. The streaming contract now correctly +attributes ESPHome's C++ runtime to GPLv3 and Python/other material to MIT. + +## Original implementation scope — 2026-10-10 + +The following measurements retain the original implementation evidence. The +verification section above gives the final automated results and firmware sizes. + +The comparison baseline is main `e4304465de98c925c34820b945b2549854a4bd63`. +Candidate firmware and embedded UI match `16d7da50436dc294d8d23dc975edc6f9a875460d`. +Both use ESPHome 2026.9.0, Python 3.13, tzdata 2026.4 and ESP-IDF 5.5.5, +the same configuration and public compile-only settings. Generated shared-audio +fixtures are synthetic, outside application images and unsuitable for playback +or installation. No hardware was accessed for these results. + +See [CI reproduction](../../.github/workflows/README.md), +[browser prerequisites](../../web/device-ui/README.md#browser-validation) and +the [streaming contract](quran-streaming.md). + +| Check | Result | +| --- | --- | +| CMake/CTest host suite under UBSan | 18/18 passed, including audio, scheduler, display, Waveshare and RTC adapters. | +| Python suite with pinned ESPHome and resolved ArduinoJson | 113/113 passed; production and isolated local API routing preserve durable records. | +| Complete browser suites | Chromium 247/247 and WebKit 247/247 passed with the production Digest bridge. | +| Final Quran browser cases | 8/8 passed across both browsers, including two additional uncertain-Play/Settings-Stop cases. | +| Existing Stop browser cases | 100/100 passed across Chromium and WebKit after the final loading-state change. | +| Production Quran worker and HTTPS helper under UBSan | Passed deterministic HTTP/task checks. | +| Pinned ESPHome mirror verification | Only the two documented upstream source files differ; all other manifest hashes match. | + +The complete browser suites cover existing preferences, drafts, recovery, +authentication and response ordering. Focused final checks cover catalog retry, +edition availability, distinct catalog/playback feedback, playback completion, +uncertain Play without repetition and Stop during loading from Settings. An +initial mixed-source full run had a WebKit playback-observation timeout; both +complete suites passed against stable rebased source, followed by final affected +checks. Simulated responses do not establish device network or speaker behavior. + +The worker tests use production `quran.cpp` and the shared HTTP policy. They +cover whole-catalog search and paging, all 114 surah names, malformed/truncated +objects, object-size limits, unavailable surahs, task allocation failure, Stop, +Athan, disconnect, speaker failure, maintenance and shutdown cancellation. +Catalog browsing leaves active audio untouched. Redirect tests reject insecure +or foreign destinations before connecting and verify detached callback lifetime. +The audio adapter checks decoder failure, request timeout and epoch revocation. +These tests use deterministic adapters, without real TLS or a physical decoder. + +## Original OTA capacity + +All ten variants are compared using actual `firmware.ota.bin` payloads and the +capacity checker. The 1,572,864-byte application budget, dual 2,097,152-byte slots +and separate 3,670,016-byte shared-audio partition are retained. Generic scheduler +profiles have no Quran metadata service or product network-reader patch. + +| Variant | Baseline OTA bytes | Candidate OTA bytes | Delta | Budget remaining | Slot free | Static RAM bytes (delta) | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| Atom reference | 1,277,360 | 1,298,640 | +21,280 | 274,224 | 798,512 | 115,619 (+152) | +| Waveshare production | 1,284,752 | 1,306,640 | +21,888 | 266,224 | 790,512 | 116,115 (+152) | +| Scheduler device | 1,121,168 | 1,121,520 | +352 | 451,344 | 975,632 | 113,231 (+16) | +| Scheduler validation | 1,138,704 | 1,139,056 | +352 | 433,808 | 958,096 | 113,943 (+16) | +| Provisioning validation | 1,278,800 | 1,300,096 | +21,296 | 272,768 | 797,056 | 115,619 (+152) | +| Upgrade qualification | 1,284,336 | 1,304,912 | +20,576 | 267,952 | 792,240 | 115,699 (+160) | +| Upgrade startup failure | 1,284,336 | 1,304,912 | +20,576 | 267,952 | 792,240 | 115,699 (+160) | +| Waveshare audio | 1,213,344 | 1,235,296 | +21,952 | 337,568 | 861,856 | 113,699 (+160) | +| Waveshare display | 1,263,312 | 1,285,024 | +21,712 | 287,840 | 812,128 | 116,115 (+152) | +| Waveshare diagnostics | 1,350,224 | 1,371,888 | +21,664 | 200,976 | 725,264 | 116,783 (+160) | + +A task-local disk-space failure occurred after the display OTA was generated, +before the ELF copy completed. Generated intermediates were cleared, the compile +was rerun successfully and its capacity result above uses the successful run. + +Static RAM from the linker is a separate measurement. It does not establish +available heap, fragmentation, PSRAM, TLS headroom or task-stack margin. The +8 KiB metadata task and bounded JSON allocations need concurrent hardware +measurement before release. There is no added UI framework, font service or +bundled Quran recording. Embedded gzip assets total 25,104 bytes versus 22,015 +bytes on the baseline, an increase of 3,089 bytes. All 150 tracked binary-producing +firmware, UI and core files in the candidate build tree match the reviewed source. + +## Provider observations + +The live English reciter response contained 243 entries in 161,432 bytes; the +largest raw entry was 5,329 bytes, below the 16 KiB object limit. Its observed +audio hosts were `cdn.mp3quran.net` and `server15.mp3quran.net`. These are dated +provider observations, not a fixed catalog count or a guarantee of future hosts. + +Bounded byte-range downloads, without playing or bundling media, returned +`audio/mpeg` over certificate-verified HTTPS. FFprobe identified these formats: + +| Requested recording | Observed format and redirect | +| --- | --- | +| Ibrahim Al-Akhdar, surah 1 | MP3, 22,050 Hz stereo, 48 kbit/s; no redirect. | +| Alhusayni Al-Azazi, surah 1 | Provider redirected to Muhammad Minshawi's `r3/001.mp3` on the same approved CDN; MP3, 44,100 Hz stereo, 128 kbit/s. | +| Othman Al-Ansary, surah 1 | MP3, 44,100 Hz stereo, 256 kbit/s; no redirect. | + +These range requests returned HTTP 206; the production reader requests a complete +recording and requires HTTP 200. They establish provider reachability, sample +formats and an observed redirect, without proving the device's TLS, decode, +mono output, clock handling or long-stream behavior. The pinned decoder supports +16-bit PCM with one or two channels and supplies decoded stream information to +the existing I2S speaker; actual output across these formats remains untested. + +## UI evidence + +Chromium captures use a synthetic reciter, catalog and playback state at 1440 px +desktop and 390 px phone widths. They show interface layout and feedback only. +The existing palette, native controls and local asset design remain intact. +Review fixes separately resolved stale playback feedback, catalog errors hidden +during playback, and Settings Stop hidden during loading. + +![Quran controls on desktop](quran-streaming/desktop.png) + +![Quran controls on a phone](quran-streaming/mobile.png) + +The loading-state Stop bar is visible in both +[desktop](quran-streaming/settings-loading-desktop.png) and +[phone](quran-streaming/settings-loading-mobile.png) captures. + +## Remaining qualification + +Before release, follow the [attended hardware qualification list](quran-streaming.md#hardware-qualification-before-release). +Measure long-surah and repeated-start memory behavior while streaming, parsing +catalogs, polling the local UI and updating optional displays. Verify audible +formats, physical Stop, due Athan priority, Wi-Fi stalls/loss, cold restart and +firmware-maintenance interruption. Confirm preserved prayer settings, consumption +history, shared recordings and recovery. Existing Athan hardware evidence does +not qualify this new Quran path. diff --git a/docs/development/quran-streaming.md b/docs/development/quran-streaming.md new file mode 100644 index 0000000..0f4de1a --- /dev/null +++ b/docs/development/quran-streaming.md @@ -0,0 +1,178 @@ +# Quran streaming + +The product development firmware can play one MP3Quran surah on the speaker. +The local Quran page loads the live reciter catalog, searches across it, pages +through matching reciters, and offers each selected recording edition's available +surahs. This feature requires separate physical qualification before release; +see the [implementation validation](quran-streaming-validation-2026-10-10.md). + +## Playback contract + +Audio travels directly from MP3Quran to the device over verified HTTPS. The browser +only selects numeric reciter, edition and surah identifiers. Closing it does not +stop playback. One surah plays to completion, without a queue or automatic next +surah. Stop remains available while resolving or loading a recording. + +Scheduled Athan replaces Quran using the existing single announcement pipeline. +Quran never resumes automatically after Athan, Stop, reboot, loss of Wi-Fi, +speaker failure or firmware maintenance. A late metadata result cannot restart a +session invalidated by Stop or Athan. A playback token also rejects a Play request +that reaches the device after Stop, Athan or a restart. Another Play replaces the previous Quran +session; Play is refused during Athan or another outstanding metadata request. +The physical Stop control uses the same playback adapter. + +The UI distinguishes catalog feedback from authoritative playback feedback. +`loading` means the device is resolving or starting a request; `playing` observes +the player pipeline and is not proof of audible output. Decoder/reader failures +report an error; a queued request that never starts times out after 30 seconds. +Every Play or Stop response is followed by a new status read. Earlier polls cannot +settle that command or replace its subsequent playback observation. An uncertain +Play is never automatically repeated: idle with its original token still valid +does not prove cancellation. Stop remains available from Today, Settings and +Quran, including before Play reaches the device. Another Play stays disabled until +the current request is resolved. A Stop is confirmed only after both active +playback and Quran loading are absent from fresh accepted status and any pending +Play token has been invalidated. Obsolete Play replies cannot undo Stop feedback +or send a second Stop that could interrupt a newly due Athan. + +Browser storage remembers selection identifiers, without writing prayer settings, +consumption history or shared audio. Pause, seek, bookmarks, schedules, offline +downloads, automatic retries and additional providers are outside this version. + +## Boundaries and implementation + +The portable `Playback` capability supplies optional streaming, source, state and +session-epoch methods. Existing adapters default to unsupported streaming. The +core scheduler has no network or provider dependency. `openathan_device/Quran` +owns MP3Quran metadata and identifier resolution; `PartitionAudio` owns playback. +Waveshare's adapter retains its codec readiness and health gates. The optional +screen identifies Quran separately from Athan. + +The authenticated local API queues one background metadata task. It parses one +reciter/surah object at a time, rather than retaining the complete raw catalog. +The ESPHome main loop alone starts playback and consumes completed results. +Cancellation revokes results immediately; a task already in a blocking network +call finishes before accepting another metadata request. Metadata browsing leaves +current audio running. Streaming requires completed setup, healthy settings, +Wi-Fi and a valid clock for TLS. Updates, USB maintenance and shutdown cancel it. + +Accepted starts reserve their source immediately. The audio adapter retains one +replacement and submits it only after the speaker command queue is empty and +both pipelines report stopped. It ignores outgoing playback and errors during +that wait; only the submitted session can complete the new reservation. Later +starts replace the pending recording, and Stop or speaker failure cancels it. +Quran's 30-second startup deadline includes the wait for Stop. Athan ownership +blocks Quran Play throughout the handoff and playback; canceled Quran never +automatically resumes. + +Catalog search stays editable with a 400 ms debounce. Edits during catalog, +edition or Play requests coalesce into the latest search, starting from offset +zero after the current operation finishes. Following Play, a fresh playback +observation must no longer report Quran loading or Athan ownership. Obsolete +catalog/edition results are ignored, and pagination appends only to results for +the accepted query. Failed API requests require an explicit retry; Play is never +automatically repeated. + +Bounds are 1 MiB per metadata response, 16 KiB per object, 1,024 reciters, +20 reciters per returned page, 32 editions per reciter, 114 distinct surahs, +256 bytes per name, 128 bytes per search and 32 KiB per normalized response. +Metadata requests have four-second I/O timeouts and a 30-second parsing/read-loop +deadline. Redirect opening and an in-flight blocking call finish within their +I/O timeouts before cancellation takes effect. Names and IDs come from the +provider; malformed or oversized responses +fail without substituting another recording. Search uses the English catalog and +ASCII case folding. An edition is playable only if its selected surah is listed. + +Only `https://www.mp3quran.net/api/v3/` metadata URLs and audio URLs on +`cdn.mp3quran.net` or `server.mp3quran.net` (one to three decimal digits) are +allowed. Audio paths reject credentials, explicit ports, queries, fragments, +escaped characters and traversal. The shared HTTP helper disables automatic +redirects, checks each absolute redirect before connecting, permits at most four, +and uses the ESP-IDF certificate bundle. Foreign hosts, insecure URLs, relative +redirects and newly introduced provider hosts fail explicitly. Playback requires +HTTP 200 and MP3 content type. No user-entered URL or provider credential is used. + +The product package mirrors the pinned ESPHome 2026.9.0 `audio` and `speaker` +components because the original network reader followed redirects without the +provider restriction, and the player lacked a public pipeline-state observation. +Only `audio/audio_reader.cpp` and `speaker/media_player/speaker_media_player.h` +change upstream behavior, under `OPENATHAN_QURAN`. Stored-file playback retains +the upstream path. The generic scheduler profiles retain upstream components and +do not enable this feature. The guarded speaker observation also checks queue +drain and both stopped pipeline states. Each mirror includes ESPHome's upstream +license: its C++ runtime is GPLv3, while Python and other material use MIT. The +[upstream manifest](../../firmware/esphome/components/audio/upstream.json) and +vendoring regression pin the unmodified files. An ESPHome upgrade must refresh +both mirrors, reconsider these two patches and rerun adapter and firmware checks. + +## Local API + +These routes use the existing device password, Digest authentication, origin +checks and main-loop exchange. Errors use the existing JSON `error` field. + +| Route | Input | Result | +| --- | --- | --- | +| `POST /api/quran/catalog` | `{"kind":"reciters","offset":0,"query":""}` | Catalog state and request ID; starts bounded catalog/search loading. | +| `POST /api/quran/catalog` | `{"kind":"editions","reciter":1}` | Catalog state and request ID; loads that reciter's editions. | +| `GET /api/quran/catalog` | None | `idle`, `loading`, `ready` or `error`, request ID, error and completed data when available. | +| `POST /api/quran/play` | `{"reciter":1,"edition":9,"surah":18,"expected_playback_token":""}` | Full device status; starts fresh identifier resolution only if the token still matches. The example IDs illustrate the shape only. | +| `POST /api/stop` | `{}` | Full status after invalidating pending and active playback. | + +Reciter data includes `reciters:[{id,name}]`, `surahs:[{id,name}]`, `total`, +`offset` and nullable `next_offset`. Edition data includes +`editions:[{id,name,surahs:[1,18]}]`. Pollers must match the returned request ID; +another client's catalog request must not be accepted as their own result. +No complete catalog is cached across requests or reboots. + +`GET /api/status` adds `playback_source` (`none`, `athan`, `quran`) and optional +`quran:{supported,state,error,reciter,edition,surah,playback_token}`. The Quran state is `idle`, +`loading`, `playing` or `error`. IDs describe the selected session, including +after it ends. These fields are transient observations, without durable revision +numbers; clients must preserve newer request observations. `playback_token` is an +opaque, transient string of at most 59 bytes, combining a per-boot identifier and +the adapter's playback counter. Stop and new starts advance that counter; reboot +changes the identifier. Clients use exact equality, never interpret or persist +the token, and obtain a new observation after each command settles. It is a Play +precondition, not a complete playback-state revision or authentication credential. +Missing or stale Play tokens return HTTP 409 before altering audio or starting a +worker. Invalid payload types, unexpected keys or oversized tokens return HTTP +400. This development API requires the precondition immediately: cached older +pages must reload, and pages without a token cannot submit Play. A successful +Play acknowledgment does not confirm playback. Unsupported builds can omit `quran`. + +## Provider and licensing + +Catalog and recording links come from [MP3Quran's API](https://www.mp3quran.net/eng/api). +The local page attributes MP3Quran. Its published +[developer-use policy](https://www.mp3quran.net/eng/privacy) permits application +use of its materials and links; recordings retain their own terms. This change +bundles no Quran recording, rehosts no provider media and establishes no right to +redistribute a future offline library. Project-owned software/documentation +licensing remains unchanged; copied ESPHome C++ runtime sources retain GPLv3 +attribution and Python/other material retains MIT attribution. Both copied +license files remain intact. + +## Hardware qualification before release + +Use the applicable [scheduler runbook](../../firmware/esphome/scheduler/VALIDATION.md) +or [provisioning runbook](../../firmware/esphome/provisioning/HARDWARE_TEST.md), +recheck the actual device and preserve settings, prayer history, shared recordings +and application recovery. CI fixtures are synthetic and must never be installed +or played. Qualification requires attended testing with source-bound application +images and real provider recordings: + +- Verify several reciters, editions, MP3 sample rates/channel counts and complete + playback, including a long surah and closing the browser. +- Interrupt resolving, loading and playing Quran with local and physical Stop, + and with a due Athan. Verify exactly one Athan and no subsequent Quran restart. +- Exercise Wi-Fi loss, stalls, TLS/redirect rejection, malformed metadata, decode + failure, reboot and firmware maintenance. Verify truthful status and explicit + recovery through a new owner Play. +- Measure minimum internal heap, largest free block, fragmentation, PSRAM usage + and metadata-task stack headroom during TLS, catalog parsing, streaming, + status/UI polling and display activity. Compare long runs and repeated starts. + Flash budget, static RAM totals and successful builds do not establish this. +- Confirm normal/Fajr playback, schedule/settings/history integrity, volume, + optional displays and recovery still work after streaming tests. + +Record observed hardware results separately from host, browser and build evidence. diff --git a/docs/development/quran-streaming/desktop.png b/docs/development/quran-streaming/desktop.png new file mode 100644 index 0000000..45343a3 Binary files /dev/null and b/docs/development/quran-streaming/desktop.png differ diff --git a/docs/development/quran-streaming/mobile.png b/docs/development/quran-streaming/mobile.png new file mode 100644 index 0000000..49020b5 Binary files /dev/null and b/docs/development/quran-streaming/mobile.png differ diff --git a/docs/development/quran-streaming/settings-loading-desktop.png b/docs/development/quran-streaming/settings-loading-desktop.png new file mode 100644 index 0000000..8b57a3d Binary files /dev/null and b/docs/development/quran-streaming/settings-loading-desktop.png differ diff --git a/docs/development/quran-streaming/settings-loading-mobile.png b/docs/development/quran-streaming/settings-loading-mobile.png new file mode 100644 index 0000000..4f276a6 Binary files /dev/null and b/docs/development/quran-streaming/settings-loading-mobile.png differ diff --git a/docs/development/roadmap.md b/docs/development/roadmap.md index cb20cf3..7074436 100644 --- a/docs/development/roadmap.md +++ b/docs/development/roadmap.md @@ -86,10 +86,12 @@ discovery is separate from public OTA application-transfer acceptance. ## Phase 3 — Quran and adhkar -- [ ] Quran streaming with selectable reciter/source. -- [ ] Surah/ayah playback controls. +- [x] Development implementation: MP3Quran streaming with reciter, edition and + available-surah selection; see the [contract](quran-streaming.md). +- [ ] Physical playback and runtime memory qualification before release. +- [ ] Additional streaming providers and ayah playback controls. - [ ] Resume/bookmark behavior. -- [ ] Athan interruption/resume policy. +- [x] Athan interrupts Quran; this version requires an explicit Play to restart it. - [ ] Morning/evening adhkar support. ## Phase 4 — Offline resilience and expansion diff --git a/firmware/esphome/components/audio/LICENSE b/firmware/esphome/components/audio/LICENSE new file mode 100644 index 0000000..215460a --- /dev/null +++ b/firmware/esphome/components/audio/LICENSE @@ -0,0 +1,709 @@ +# ESPHome License + +Copyright (c) 2019 ESPHome + +The ESPHome License is made up of two base licenses: MIT and the GNU GENERAL PUBLIC LICENSE. +The C++/runtime codebase of the ESPHome project (file extensions .c, .cpp, .h, .hpp, .tcc, .ino) are +published under the GPLv3 license. The python codebase and all other parts of this codebase are +published under the MIT license. + +Both MIT and GPLv3 licenses are attached to this document. + +## MIT License + +Copyright (c) 2019 ESPHome + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +## GPLv3 License + + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. diff --git a/firmware/esphome/components/audio/__init__.py b/firmware/esphome/components/audio/__init__.py new file mode 100644 index 0000000..2a5304b --- /dev/null +++ b/firmware/esphome/components/audio/__init__.py @@ -0,0 +1,441 @@ +from collections.abc import Callable +from dataclasses import dataclass, field +from typing import Any + +import esphome.codegen as cg +from esphome.components.esp32 import ( + add_idf_component, + add_idf_sdkconfig_option, + include_builtin_idf_component, + require_certificate_bundle, +) +import esphome.config_validation as cv +from esphome.const import ( + CONF_BITS_PER_SAMPLE, + CONF_NUM_CHANNELS, + CONF_SAMPLE_RATE, + CONF_SIZE, +) +from esphome.core import CORE +import esphome.final_validate as fv +from esphome.types import ConfigType + +AUTO_LOAD = ["ring_buffer"] +CODEOWNERS = ["@kahrendt"] +DOMAIN = "audio" +audio_ns = cg.esphome_ns.namespace("audio") + +AudioFile = audio_ns.struct("AudioFile") +AudioFileType = audio_ns.enum("AudioFileType", is_class=True) +AUDIO_FILE_TYPE_ENUM = { + "NONE": AudioFileType.NONE, + "WAV": AudioFileType.WAV, + "MP3": AudioFileType.MP3, + "FLAC": AudioFileType.FLAC, + "OPUS": AudioFileType.OPUS, +} + +MEMORY_PSRAM = "psram" +MEMORY_INTERNAL = "internal" +MEMORY_LOCATIONS = [MEMORY_PSRAM, MEMORY_INTERNAL] + + +@dataclass +class FlacOptions: + buffer_memory: str | None = None + + +@dataclass +class Mp3Options: + buffer_memory: str | None = None + + +@dataclass +class OpusPseudostackOptions: + threadsafe: bool | None = None + buffer_memory: str | None = None + size: int | None = None + + +@dataclass +class OpusOptions: + floating_point: bool | None = None + state_memory: str | None = None + pseudostack: OpusPseudostackOptions = field(default_factory=OpusPseudostackOptions) + + +@dataclass +class AudioData: + flac_support: bool = False + mp3_support: bool = False + opus_support: bool = False + wav_support: bool = False + micro_decoder_support: bool = False + flac: FlacOptions = field(default_factory=FlacOptions) + mp3: Mp3Options = field(default_factory=Mp3Options) + opus: OpusOptions = field(default_factory=OpusOptions) + + +def _get_data() -> AudioData: + if DOMAIN not in CORE.data: + CORE.data[DOMAIN] = AudioData() + return CORE.data[DOMAIN] + + +def request_flac_support() -> None: + """Request FLAC codec support for audio decoding.""" + _get_data().flac_support = True + + +def request_mp3_support() -> None: + """Request MP3 codec support for audio decoding.""" + _get_data().mp3_support = True + + +def request_opus_support() -> None: + """Request Opus codec support for audio decoding.""" + _get_data().opus_support = True + + +def request_wav_support() -> None: + """Request WAV codec support for audio decoding.""" + _get_data().wav_support = True + + +def request_micro_decoder_support() -> None: + """Request micro-decoder library support for audio decoding.""" + _get_data().micro_decoder_support = True + + +CONF_MIN_BITS_PER_SAMPLE = "min_bits_per_sample" +CONF_MAX_BITS_PER_SAMPLE = "max_bits_per_sample" +CONF_MIN_CHANNELS = "min_channels" +CONF_MAX_CHANNELS = "max_channels" +CONF_MIN_SAMPLE_RATE = "min_sample_rate" +CONF_MAX_SAMPLE_RATE = "max_sample_rate" + +CONF_CODECS = "codecs" +CONF_WAV = "wav" +CONF_FLAC = "flac" +CONF_MP3 = "mp3" +CONF_OPUS = "opus" +CONF_BUFFER_MEMORY = "buffer_memory" +CONF_FLOATING_POINT = "floating_point" +CONF_STATE_MEMORY = "state_memory" +CONF_PSEUDOSTACK = "pseudostack" +CONF_THREADSAFE = "threadsafe" + + +_MEMORY_LOCATION_VALIDATOR = cv.one_of(*MEMORY_LOCATIONS, lower=True) + + +def _maybe_empty_codec(schema: cv.Schema) -> Callable[[Any], Any]: + """Wrap a codec dict schema so that a bare key (None value) is treated as an empty dict.""" + + def validator(value: Any) -> Any: + if value is None: + value = {} + return schema(value) + + return validator + + +CODEC_FLAC_SCHEMA = cv.Schema( + { + cv.Optional(CONF_BUFFER_MEMORY): _MEMORY_LOCATION_VALIDATOR, + } +) + +CODEC_MP3_SCHEMA = cv.Schema( + { + cv.Optional(CONF_BUFFER_MEMORY): _MEMORY_LOCATION_VALIDATOR, + } +) + +OPUS_PSEUDOSTACK_SCHEMA = cv.Schema( + { + cv.Optional(CONF_THREADSAFE): cv.boolean, + cv.Optional(CONF_BUFFER_MEMORY): _MEMORY_LOCATION_VALIDATOR, + cv.Optional(CONF_SIZE): cv.int_range(60000, 240000), + } +) + +CODEC_OPUS_SCHEMA = cv.Schema( + { + cv.Optional(CONF_FLOATING_POINT): cv.boolean, + cv.Optional(CONF_STATE_MEMORY): _MEMORY_LOCATION_VALIDATOR, + cv.Optional(CONF_PSEUDOSTACK): _maybe_empty_codec(OPUS_PSEUDOSTACK_SCHEMA), + } +) + +CODEC_WAV_SCHEMA = cv.Schema({}) + +CODECS_SCHEMA = cv.Schema( + { + cv.Optional(CONF_FLAC): _maybe_empty_codec(CODEC_FLAC_SCHEMA), + cv.Optional(CONF_MP3): _maybe_empty_codec(CODEC_MP3_SCHEMA), + cv.Optional(CONF_OPUS): _maybe_empty_codec(CODEC_OPUS_SCHEMA), + cv.Optional(CONF_WAV): _maybe_empty_codec(CODEC_WAV_SCHEMA), + } +) + +CONFIG_SCHEMA = cv.All( + cv.Schema( + { + cv.Optional(CONF_CODECS): _maybe_empty_codec(CODECS_SCHEMA), + } + ), + cv.only_on_esp32, +) + +AUDIO_COMPONENT_SCHEMA = cv.Schema( + { + cv.Optional(CONF_BITS_PER_SAMPLE): cv.int_range(8, 32), + cv.Optional(CONF_NUM_CHANNELS): cv.int_range(1, 2), + cv.Optional(CONF_SAMPLE_RATE): cv.int_range(8000, 48000), + } +) + + +def set_stream_limits( + min_bits_per_sample: int = cv.UNDEFINED, + max_bits_per_sample: int = cv.UNDEFINED, + min_channels: int = cv.UNDEFINED, + max_channels: int = cv.UNDEFINED, + min_sample_rate: int = cv.UNDEFINED, + max_sample_rate: int = cv.UNDEFINED, +) -> Callable[[ConfigType], None]: + """Sets the limits for the audio stream that audio component can handle + + When the component sinks audio (e.g., a speaker), these indicate the limits to the audio it can receive. + When the component sources audio (e.g., a microphone), these indicate the limits to the audio it can send. + """ + + def set_limits_in_config(config: ConfigType) -> None: + if min_bits_per_sample is not cv.UNDEFINED: + config[CONF_MIN_BITS_PER_SAMPLE] = min_bits_per_sample + if max_bits_per_sample is not cv.UNDEFINED: + config[CONF_MAX_BITS_PER_SAMPLE] = max_bits_per_sample + if min_channels is not cv.UNDEFINED: + config[CONF_MIN_CHANNELS] = min_channels + if max_channels is not cv.UNDEFINED: + config[CONF_MAX_CHANNELS] = max_channels + if min_sample_rate is not cv.UNDEFINED: + config[CONF_MIN_SAMPLE_RATE] = min_sample_rate + if max_sample_rate is not cv.UNDEFINED: + config[CONF_MAX_SAMPLE_RATE] = max_sample_rate + + return set_limits_in_config + + +def final_validate_audio_schema( + name: str, + *, + audio_device: str, + bits_per_sample: int = cv.UNDEFINED, + channels: int = cv.UNDEFINED, + sample_rate: int = cv.UNDEFINED, + enabled_channels: list[int] = cv.UNDEFINED, + audio_device_issue: bool = False, +) -> cv.Schema: + """Validates audio compatibility when passed between different components. + + The component derived from ``AUDIO_COMPONENT_SCHEMA`` should call ``set_stream_limits`` in a validator to specify its compatible settings + + - If audio_device_issue is True, then the error message indicates the user should adjust the AUDIO_COMPONENT_SCHEMA derived component's configuration to match the values passed to this function + - If audio_device_issue is False, then the error message indicates the user should adjust the configuration of the component calling this function, as it falls out of the valid stream limits + + Args: + name (str): Friendly name of the component calling this function with an audio component to validate + audio_device (str): The configuration parameter name that contains the ID of an AUDIO_COMPONENT_SCHEMA derived component to validate against + bits_per_sample (int, optional): The desired bits per sample + channels (int, optional): The desired number of channels + sample_rate (int, optional): The desired sample rate + enabled_channels (list[int], optional): The desired enabled channels + audio_device_issue (bool, optional): Format the error message to indicate the problem is in the configuration for the ``audio_device`` component. Defaults to False. + """ + + def validate_audio_compatiblity(audio_config: ConfigType) -> ConfigType: + audio_schema = {} + + if bits_per_sample is not cv.UNDEFINED: + try: + cv.int_range( + min=audio_config.get(CONF_MIN_BITS_PER_SAMPLE), + max=audio_config.get(CONF_MAX_BITS_PER_SAMPLE), + )(bits_per_sample) + except cv.Invalid as exc: + if audio_device_issue: + error_string = f"Invalid configuration for the specified {audio_device}. The {name} component requires {bits_per_sample} bits per sample." + else: + error_string = f"Invalid configuration for the {name} component. The {CONF_BITS_PER_SAMPLE} {str(exc)}" + raise cv.Invalid(error_string) from exc + + if channels is not cv.UNDEFINED: + try: + cv.int_range( + min=audio_config.get(CONF_MIN_CHANNELS), + max=audio_config.get(CONF_MAX_CHANNELS), + )(channels) + except cv.Invalid as exc: + if audio_device_issue: + error_string = f"Invalid configuration for the specified {audio_device}. The {name} component requires {channels} channels." + else: + error_string = f"Invalid configuration for the {name} component. The {CONF_NUM_CHANNELS} {str(exc)}" + raise cv.Invalid(error_string) from exc + + if sample_rate is not cv.UNDEFINED: + try: + cv.int_range( + min=audio_config.get(CONF_MIN_SAMPLE_RATE), + max=audio_config.get(CONF_MAX_SAMPLE_RATE), + )(sample_rate) + except cv.Invalid as exc: + if audio_device_issue: + error_string = f"Invalid configuration for the specified {audio_device}. The {name} component requires a {sample_rate} sample rate." + else: + error_string = f"Invalid configuration for the {name} component. The {CONF_SAMPLE_RATE} {str(exc)}" + raise cv.Invalid(error_string) from exc + + if enabled_channels is not cv.UNDEFINED: + for channel in enabled_channels: + try: + # Channels are 0-indexed + cv.int_range( + min=0, + max=audio_config.get(CONF_MAX_CHANNELS) - 1, + )(channel) + except cv.Invalid as exc: + if audio_device_issue: + error_string = f"Invalid configuration for the specified {audio_device}. The {name} component requires channel {channel}." + else: + error_string = f"Invalid configuration for the {name} component. Enabled channel {channel} {str(exc)}" + raise cv.Invalid(error_string) from exc + + return cv.Schema(audio_schema, extra=cv.ALLOW_EXTRA)(audio_config) + + return cv.Schema( + { + cv.Required(audio_device): fv.id_declaration_match_schema( + validate_audio_compatiblity + ) + }, + extra=cv.ALLOW_EXTRA, + ) + + +def _emit_memory_pair(value: str | None, psram_key: str, internal_key: str) -> None: + if value == MEMORY_PSRAM: + add_idf_sdkconfig_option(psram_key, True) + add_idf_sdkconfig_option(internal_key, False) + elif value == MEMORY_INTERNAL: + add_idf_sdkconfig_option(psram_key, False) + add_idf_sdkconfig_option(internal_key, True) + + +async def to_code(config: ConfigType) -> None: + # Re-enable ESP-IDF's HTTP client (excluded by default to save compile time) + include_builtin_idf_component("esp_http_client") + # HTTPS streams verify the server against the root certificate bundle + require_certificate_bundle() + + add_idf_component( + name="esphome/esp-audio-libs", + ref="3.2.1", + ) + + data = _get_data() + + # Merge user-supplied codec configuration (additive: presence enables the codec) + if codecs_config := config.get(CONF_CODECS): + if (flac_config := codecs_config.get(CONF_FLAC)) is not None: + data.flac_support = True + if (buffer_memory := flac_config.get(CONF_BUFFER_MEMORY)) is not None: + data.flac.buffer_memory = buffer_memory + if (mp3_config := codecs_config.get(CONF_MP3)) is not None: + data.mp3_support = True + if (buffer_memory := mp3_config.get(CONF_BUFFER_MEMORY)) is not None: + data.mp3.buffer_memory = buffer_memory + if (opus_config := codecs_config.get(CONF_OPUS)) is not None: + data.opus_support = True + floating_point = opus_config.get(CONF_FLOATING_POINT) + if floating_point is not None: + data.opus.floating_point = floating_point + if (state_memory := opus_config.get(CONF_STATE_MEMORY)) is not None: + data.opus.state_memory = state_memory + if (pseudostack_config := opus_config.get(CONF_PSEUDOSTACK)) is not None: + threadsafe = pseudostack_config.get(CONF_THREADSAFE) + if threadsafe is not None: + data.opus.pseudostack.threadsafe = threadsafe + if ( + buffer_memory := pseudostack_config.get(CONF_BUFFER_MEMORY) + ) is not None: + data.opus.pseudostack.buffer_memory = buffer_memory + if (size := pseudostack_config.get(CONF_SIZE)) is not None: + data.opus.pseudostack.size = size + if CONF_WAV in codecs_config: + data.wav_support = True + + if data.micro_decoder_support: + add_idf_component(name="esphome/micro-decoder", ref="0.4.0") + + # All codecs are enabled by default in micro-decoder, so disable the ones that aren't requested to save flash + if not data.flac_support: + add_idf_sdkconfig_option("CONFIG_MICRO_DECODER_CODEC_FLAC", False) + if not data.mp3_support: + add_idf_sdkconfig_option("CONFIG_MICRO_DECODER_CODEC_MP3", False) + if not data.opus_support: + add_idf_sdkconfig_option("CONFIG_MICRO_DECODER_CODEC_OPUS", False) + # Vorbis is unsupported in ESPHome, so always disable it + add_idf_sdkconfig_option("CONFIG_MICRO_DECODER_CODEC_VORBIS", False) + if not data.wav_support: + add_idf_sdkconfig_option("CONFIG_MICRO_DECODER_CODEC_WAV", False) + + # Configure each codec library. + # Adds a define and IDF component for legacy `audio_decoder.cpp`. + if data.flac_support: + cg.add_define("USE_AUDIO_FLAC_SUPPORT") + add_idf_component(name="esphome/micro-flac", ref="0.2.0") + _emit_memory_pair( + data.flac.buffer_memory, + "CONFIG_MICRO_FLAC_PREFER_PSRAM", + "CONFIG_MICRO_FLAC_PREFER_INTERNAL", + ) + if data.mp3_support: + cg.add_define("USE_AUDIO_MP3_SUPPORT") + add_idf_component(name="esphome/micro-mp3", ref="0.4.0") + _emit_memory_pair( + data.mp3.buffer_memory, + "CONFIG_MICRO_MP3_PREFER_PSRAM", + "CONFIG_MICRO_MP3_PREFER_INTERNAL", + ) + if data.opus_support: + cg.add_define("USE_AUDIO_OPUS_SUPPORT") + add_idf_component(name="esphome/micro-opus", ref="0.4.1") + if data.opus.floating_point is not None: + add_idf_sdkconfig_option( + "CONFIG_OPUS_FLOATING_POINT", data.opus.floating_point + ) + _emit_memory_pair( + data.opus.state_memory, + "CONFIG_OPUS_STATE_PREFER_PSRAM", + "CONFIG_OPUS_STATE_PREFER_INTERNAL", + ) + if data.opus.pseudostack.threadsafe is True: + add_idf_sdkconfig_option("CONFIG_OPUS_THREADSAFE_PSEUDOSTACK", True) + add_idf_sdkconfig_option("CONFIG_OPUS_NONTHREADSAFE_PSEUDOSTACK", False) + elif data.opus.pseudostack.threadsafe is False: + add_idf_sdkconfig_option("CONFIG_OPUS_THREADSAFE_PSEUDOSTACK", False) + add_idf_sdkconfig_option("CONFIG_OPUS_NONTHREADSAFE_PSEUDOSTACK", True) + _emit_memory_pair( + data.opus.pseudostack.buffer_memory, + "CONFIG_OPUS_PSEUDOSTACK_PREFER_PSRAM", + "CONFIG_OPUS_PSEUDOSTACK_PREFER_INTERNAL", + ) + if data.opus.pseudostack.size is not None: + add_idf_sdkconfig_option( + "CONFIG_OPUS_PSEUDOSTACK_SIZE", data.opus.pseudostack.size + ) + if data.wav_support: + cg.add_define("USE_AUDIO_WAV_SUPPORT") + add_idf_component(name="esphome/micro-wav", ref="0.2.0") diff --git a/firmware/esphome/components/audio/audio.cpp b/firmware/esphome/components/audio/audio.cpp new file mode 100644 index 0000000..402e741 --- /dev/null +++ b/firmware/esphome/components/audio/audio.cpp @@ -0,0 +1,131 @@ +#include "audio.h" + +#include "esphome/core/helpers.h" + +#include + +namespace esphome::audio { + +// Euclidean's algorithm for finding the greatest common divisor +static uint32_t gcd(uint32_t a, uint32_t b) { + while (b != 0) { + uint32_t t = b; + b = a % b; + a = t; + } + return a; +} + +AudioStreamInfo::AudioStreamInfo(uint8_t bits_per_sample, uint8_t channels, uint32_t sample_rate) + : bits_per_sample_(bits_per_sample), channels_(channels), sample_rate_(sample_rate) { + this->ms_sample_rate_gcd_ = gcd(1000, this->sample_rate_); + this->bytes_per_sample_ = (this->bits_per_sample_ + 7) / 8; +} + +uint32_t AudioStreamInfo::frames_to_microseconds(uint32_t frames) const { + return (frames * 1000000 + (this->sample_rate_ >> 1)) / this->sample_rate_; +} + +uint32_t AudioStreamInfo::frames_to_milliseconds_with_remainder(uint32_t *total_frames) const { + uint32_t unprocessable_frames = *total_frames % (this->sample_rate_ / this->ms_sample_rate_gcd_); + uint32_t frames_for_ms_calculation = *total_frames - unprocessable_frames; + + uint32_t playback_ms = (frames_for_ms_calculation * 1000) / this->sample_rate_; + *total_frames = unprocessable_frames; + return playback_ms; +} + +bool AudioStreamInfo::operator==(const AudioStreamInfo &rhs) const { + return (this->bits_per_sample_ == rhs.get_bits_per_sample()) && (this->channels_ == rhs.get_channels()) && + (this->sample_rate_ == rhs.get_sample_rate()); +} + +const char *audio_file_type_to_string(AudioFileType file_type) { + switch (file_type) { +#ifdef USE_AUDIO_FLAC_SUPPORT + case AudioFileType::FLAC: + return "FLAC"; +#endif +#ifdef USE_AUDIO_MP3_SUPPORT + case AudioFileType::MP3: + return "MP3"; +#endif +#ifdef USE_AUDIO_OPUS_SUPPORT + case AudioFileType::OPUS: + return "OPUS"; +#endif +#ifdef USE_AUDIO_WAV_SUPPORT + case AudioFileType::WAV: + return "WAV"; +#endif + default: + return "unknown"; + } +} + +AudioFileType detect_audio_file_type(const char *content_type, const char *url) { + // Try Content-Type header first + if (content_type != nullptr && content_type[0] != '\0') { +#ifdef USE_AUDIO_MP3_SUPPORT + if (strcasecmp(content_type, "mp3") == 0 || strcasecmp(content_type, "audio/mp3") == 0 || + strcasecmp(content_type, "audio/mpeg") == 0) { + return AudioFileType::MP3; + } +#endif +#ifdef USE_AUDIO_WAV_SUPPORT + if (strcasecmp(content_type, "audio/wav") == 0) { + return AudioFileType::WAV; + } +#endif +#ifdef USE_AUDIO_FLAC_SUPPORT + if (strcasecmp(content_type, "audio/flac") == 0 || strcasecmp(content_type, "audio/x-flac") == 0) { + return AudioFileType::FLAC; + } +#endif +#ifdef USE_AUDIO_OPUS_SUPPORT + // Match "audio/ogg" with a codecs parameter containing "opus" + // Valid forms: audio/ogg;codecs=opus, audio/ogg; codecs="opus", etc. + // Plain "audio/ogg" without opus is not matched (almost always Ogg Vorbis) + if (strncasecmp(content_type, "audio/ogg", 9) == 0 && str_contains_ignore_case(content_type + 9, "opus")) { + return AudioFileType::OPUS; + } +#endif + } + + // Fallback to URL extension + if (url != nullptr && url[0] != '\0') { +#ifdef USE_AUDIO_WAV_SUPPORT + if (str_endswith_ignore_case(url, ".wav")) { + return AudioFileType::WAV; + } +#endif +#ifdef USE_AUDIO_MP3_SUPPORT + if (str_endswith_ignore_case(url, ".mp3")) { + return AudioFileType::MP3; + } +#endif +#ifdef USE_AUDIO_FLAC_SUPPORT + if (str_endswith_ignore_case(url, ".flac")) { + return AudioFileType::FLAC; + } +#endif +#ifdef USE_AUDIO_OPUS_SUPPORT + if (str_endswith_ignore_case(url, ".opus")) { + return AudioFileType::OPUS; + } +#endif + } + + return AudioFileType::NONE; +} + +void scale_audio_samples(const int16_t *audio_samples, int16_t *output_buffer, int16_t scale_factor, + size_t samples_to_scale) { + // Note the assembly dsps_mulc function has audio glitches if the input and output buffers are the same. + for (size_t i = 0; i < samples_to_scale; i++) { + int32_t acc = (int32_t) audio_samples[i] * (int32_t) scale_factor; + output_buffer[i] = (int16_t) (acc >> 15); + } +} + +} // namespace esphome::audio diff --git a/firmware/esphome/components/audio/audio.h b/firmware/esphome/components/audio/audio.h new file mode 100644 index 0000000..36780a3 --- /dev/null +++ b/firmware/esphome/components/audio/audio.h @@ -0,0 +1,200 @@ +#pragma once + +#include "esphome/core/defines.h" +#include "esphome/core/helpers.h" // for ESPDEPRECATED + +#include +#include + +namespace esphome::audio { + +class AudioStreamInfo { + /* Class to respresent important parameters of the audio stream that also provides helper function to convert between + * various audio related units. + * + * - An audio sample represents a unit of audio for one channel. + * - A frame represents a unit of audio with a sample for every channel. + * + * In general, converting between bytes, samples, and frames shouldn't result in rounding errors so long as frames + * are used as the main unit when transferring audio data. Durations may result in rounding for certain sample rates; + * e.g., 44.1 KHz. The ``frames_to_milliseconds_with_remainder`` function should be used for accuracy, as it takes + * into account the remainder rather than just ignoring any rounding. + */ + public: + AudioStreamInfo() + : AudioStreamInfo(16, 1, 16000){}; // Default values represent ESPHome's audio components historical values + AudioStreamInfo(uint8_t bits_per_sample, uint8_t channels, uint32_t sample_rate); + + uint8_t get_bits_per_sample() const { return this->bits_per_sample_; } + uint8_t get_channels() const { return this->channels_; } + uint32_t get_sample_rate() const { return this->sample_rate_; } + + /// @brief Convert bytes to duration in milliseconds. + /// @param bytes Number of bytes to convert + /// @return Duration in milliseconds that will store `bytes` bytes of audio. May round down for certain sample rates + /// or values of `bytes`. + uint32_t bytes_to_ms(size_t bytes) const { + return bytes * 1000 / (this->sample_rate_ * this->bytes_per_sample_ * this->channels_); + } + + /// @brief Convert bytes to frames. + /// @param bytes Number of bytes to convert + /// @return Audio frames that will store `bytes` bytes. + uint32_t bytes_to_frames(size_t bytes) const { return (bytes / (this->bytes_per_sample_ * this->channels_)); } + + /// @brief Convert bytes to samples. + /// @param bytes Number of bytes to convert + /// @return Audio samples that will store `bytes` bytes. + uint32_t bytes_to_samples(size_t bytes) const { return (bytes / this->bytes_per_sample_); } + + /// @brief Converts frames to bytes. + /// @param frames Number of frames to convert. + /// @return Number of bytes that will store `frames` frames of audio. + size_t frames_to_bytes(uint32_t frames) const { return frames * this->bytes_per_sample_ * this->channels_; } + + /// @brief Converts samples to bytes. + /// @param samples Number of samples to convert. + /// @return Number of bytes that will store `samples` samples of audio. + size_t samples_to_bytes(uint32_t samples) const { return samples * this->bytes_per_sample_; } + + /// @brief Converts duration to frames. + /// @param ms Duration in milliseconds + /// @return Audio frames that will store `ms` milliseconds of audio. May round down for certain sample rates. + uint32_t ms_to_frames(uint32_t ms) const { return (ms * this->sample_rate_) / 1000; } + + /// @brief Converts duration to samples. + /// @param ms Duration in milliseconds + /// @return Audio samples that will store `ms` milliseconds of audio. May round down for certain sample rates. + uint32_t ms_to_samples(uint32_t ms) const { return (ms * this->channels_ * this->sample_rate_) / 1000; } + + /// @brief Converts duration to bytes. May round down for certain sample rates. + /// @param ms Duration in milliseconds + /// @return Bytes that will store `ms` milliseconds of audio. May round down for certain sample rates. + size_t ms_to_bytes(uint32_t ms) const { + return (ms * this->bytes_per_sample_ * this->channels_ * this->sample_rate_) / 1000; + } + + /// @brief Computes the duration, in microseconds, the given amount of frames represents. + /// @param frames Number of audio frames + /// @return Duration in microseconds `frames` represents. May be slightly inaccurate due to integer division rounding + /// for certain sample rates. + uint32_t frames_to_microseconds(uint32_t frames) const; + + /// @brief Computes the duration, in milliseconds, the given amount of frames represents. Avoids + /// accumulating rounding errors by updating `frames` with the remainder after converting. + /// @param frames Pointer to uint32_t with the number of audio frames. Replaced with the remainder. + /// @return Duration in milliseconds `frames` represents. Always less than or equal to the actual value due to + /// rounding. + uint32_t frames_to_milliseconds_with_remainder(uint32_t *frames) const; + + // Class comparison operators + bool operator==(const AudioStreamInfo &rhs) const; + bool operator!=(const AudioStreamInfo &rhs) const { return !operator==(rhs); } + + protected: + uint8_t bits_per_sample_; + uint8_t channels_; + uint32_t sample_rate_; + + // The greatest common divisor between 1000 ms = 1 second and the sample rate. Used to avoid accumulating error when + // converting from frames to duration. Computed at construction. + uint32_t ms_sample_rate_gcd_; + + // Conversion factor derived from the number of bits per sample. Assumes audio data is aligned to the byte. Computed + // at construction. + size_t bytes_per_sample_; +}; + +enum class AudioFileType : uint8_t { + NONE = 0, +#ifdef USE_AUDIO_FLAC_SUPPORT + FLAC, +#endif +#ifdef USE_AUDIO_MP3_SUPPORT + MP3, +#endif +#ifdef USE_AUDIO_OPUS_SUPPORT + OPUS, +#endif +#ifdef USE_AUDIO_WAV_SUPPORT + WAV, +#endif +}; + +struct AudioFile { + const uint8_t *data; + size_t length; + AudioFileType file_type; +}; + +/// @brief Helper function to convert file type to a const char string +/// @param file_type +/// @return const char pointer to the readable file type +const char *audio_file_type_to_string(AudioFileType file_type); + +/// @brief Detect audio file type from a Content-Type header value and/or URL extension. +/// Tries Content-Type first, then falls back to URL extension. Either parameter may be null. +/// @param content_type Content-Type header value (may be null or empty) +/// @param url URL to inspect for file extension (may be null or empty) +/// @return The detected AudioFileType, or NONE if unknown +AudioFileType detect_audio_file_type(const char *content_type, const char *url); + +/// @brief Scales Q15 fixed point audio samples. Scales in place if audio_samples == output_buffer. +/// @param audio_samples PCM int16 audio samples +/// @param output_buffer Buffer to store the scaled samples +/// @param scale_factor Q15 fixed point scaling factor +/// @param samples_to_scale Number of samples to scale +// Remove before 2026.12.0 +ESPDEPRECATED("Use esp_audio_libs::gain::apply() (from ) instead. Removed in 2026.12.0.", "2026.6.0") +void scale_audio_samples(const int16_t *audio_samples, int16_t *output_buffer, int16_t scale_factor, + size_t samples_to_scale); + +/// @brief Unpacks a quantized audio sample into a Q31 fixed-point number. +/// @param data Pointer to uint8_t array containing the audio sample +/// @param bytes_per_sample The number of bytes per sample +/// @return Q31 sample +inline int32_t unpack_audio_sample_to_q31(const uint8_t *data, size_t bytes_per_sample) { + int32_t sample = 0; + if (bytes_per_sample == 1) { + sample |= data[0] << 24; + } else if (bytes_per_sample == 2) { + sample |= data[0] << 16; + sample |= data[1] << 24; + } else if (bytes_per_sample == 3) { + sample |= data[0] << 8; + sample |= data[1] << 16; + sample |= data[2] << 24; + } else if (bytes_per_sample == 4) { + sample |= data[0]; + sample |= data[1] << 8; + sample |= data[2] << 16; + sample |= data[3] << 24; + } + + return sample; +} + +/// @brief Packs a Q31 fixed-point number as an audio sample with the specified number of bytes per sample. +/// Packs the most significant bits - no dithering is applied. +/// @param sample Q31 fixed-point number to pack +/// @param data Pointer to data array to store +/// @param bytes_per_sample The audio data's bytes per sample +inline void pack_q31_as_audio_sample(int32_t sample, uint8_t *data, size_t bytes_per_sample) { + if (bytes_per_sample == 1) { + data[0] = static_cast(sample >> 24); + } else if (bytes_per_sample == 2) { + data[0] = static_cast(sample >> 16); + data[1] = static_cast(sample >> 24); + } else if (bytes_per_sample == 3) { + data[0] = static_cast(sample >> 8); + data[1] = static_cast(sample >> 16); + data[2] = static_cast(sample >> 24); + } else if (bytes_per_sample == 4) { + data[0] = static_cast(sample); + data[1] = static_cast(sample >> 8); + data[2] = static_cast(sample >> 16); + data[3] = static_cast(sample >> 24); + } +} + +} // namespace esphome::audio diff --git a/firmware/esphome/components/audio/audio_decoder.cpp b/firmware/esphome/components/audio/audio_decoder.cpp new file mode 100644 index 0000000..0513956 --- /dev/null +++ b/firmware/esphome/components/audio/audio_decoder.cpp @@ -0,0 +1,426 @@ +#include "audio_decoder.h" + +#ifdef USE_ESP32 + +#include "esphome/core/hal.h" +#include "esphome/core/log.h" + +namespace esphome::audio { + +static const char *const TAG = "audio.decoder"; + +static const uint32_t READ_WRITE_TIMEOUT_MS = 20; // Timeout for transferring audio data + +// Max consecutive decode iterations that consume input but produce no output; e.g., skipping a large metadata block, +// before yielding and returning. +static const uint8_t MAX_NO_OUTPUT_ITERATIONS = 32; + +static const uint32_t MAX_POTENTIALLY_FAILED_COUNT = 10; + +AudioDecoder::AudioDecoder(size_t input_buffer_size, size_t output_buffer_size) + : input_buffer_size_(input_buffer_size) { + this->output_transfer_buffer_ = AudioSinkTransferBuffer::create(output_buffer_size); +} + +esp_err_t AudioDecoder::add_source(std::weak_ptr &input_ring_buffer) { + // Zero-copy source reading directly from the ring buffer's internal storage. Raw file data is byte + // aligned, so no frame alignment is required. + auto source = RingBufferAudioSource::create(input_ring_buffer.lock(), this->input_buffer_size_); + if (source == nullptr) { + // create() only returns nullptr for invalid arguments (expired ring buffer or zero buffer size) + return ESP_ERR_INVALID_ARG; + } + this->input_buffer_ = std::move(source); + return ESP_OK; +} + +esp_err_t AudioDecoder::add_source(const uint8_t *data_pointer, size_t length) { + auto source = make_unique(); + source->set_data(data_pointer, length); + this->input_buffer_ = std::move(source); + return ESP_OK; +} + +esp_err_t AudioDecoder::add_sink(std::weak_ptr &output_ring_buffer) { + if (this->output_transfer_buffer_ != nullptr) { + this->output_transfer_buffer_->set_sink(output_ring_buffer); + return ESP_OK; + } + return ESP_ERR_NO_MEM; +} + +#ifdef USE_SPEAKER +esp_err_t AudioDecoder::add_sink(speaker::Speaker *speaker) { + if (this->output_transfer_buffer_ != nullptr) { + this->output_transfer_buffer_->set_sink(speaker); + return ESP_OK; + } + return ESP_ERR_NO_MEM; +} +#endif + +esp_err_t AudioDecoder::add_sink(AudioSinkCallback *callback) { + if (this->output_transfer_buffer_ != nullptr) { + this->output_transfer_buffer_->set_sink(callback); + return ESP_OK; + } + return ESP_ERR_NO_MEM; +} + +esp_err_t AudioDecoder::start(AudioFileType audio_file_type) { + if (this->output_transfer_buffer_ == nullptr) { + return ESP_ERR_NO_MEM; + } + + this->audio_file_type_ = audio_file_type; + + this->potentially_failed_count_ = 0; + this->end_of_file_ = false; + + switch (this->audio_file_type_) { +#ifdef USE_AUDIO_FLAC_SUPPORT + case AudioFileType::FLAC: + this->flac_decoder_ = make_unique(); + this->free_buffer_required_ = + this->output_transfer_buffer_->capacity(); // Adjusted and reallocated after reading the header + break; +#endif +#ifdef USE_AUDIO_MP3_SUPPORT + case AudioFileType::MP3: + this->mp3_decoder_ = make_unique(); + this->free_buffer_required_ = + this->output_transfer_buffer_->capacity(); // Adjusted and reallocated after reading the header + break; +#endif +#ifdef USE_AUDIO_OPUS_SUPPORT + case AudioFileType::OPUS: + this->opus_decoder_ = make_unique(); + this->free_buffer_required_ = + this->output_transfer_buffer_->capacity(); // Adjusted and reallocated after reading the header + break; +#endif +#ifdef USE_AUDIO_WAV_SUPPORT + case AudioFileType::WAV: + this->wav_decoder_ = make_unique(); + // 1 KiB suffices to always make progress while avoiding excessive CPU spinning for decoding + this->free_buffer_required_ = 1024; + if (this->output_transfer_buffer_->capacity() < this->free_buffer_required_) { + this->output_transfer_buffer_->reallocate(this->free_buffer_required_); + } + break; +#endif + case AudioFileType::NONE: + default: + return ESP_ERR_NOT_SUPPORTED; + break; + } + + return ESP_OK; +} + +AudioDecoderState AudioDecoder::decode(bool stop_gracefully) { + if (this->input_buffer_ == nullptr) { + return AudioDecoderState::FAILED; + } + + if (stop_gracefully) { + if (this->output_transfer_buffer_->available() == 0) { + if (this->end_of_file_) { + // The file decoder indicates it reached the end of file + return AudioDecoderState::FINISHED; + } + + if (!this->input_buffer_->has_buffered_data()) { + // If all the internal buffers are empty, the decoding is done + return AudioDecoderState::FINISHED; + } + } + } + + if (this->potentially_failed_count_ > MAX_POTENTIALLY_FAILED_COUNT) { + if (stop_gracefully) { + // No more new data is going to come in, so decoding is done + return AudioDecoderState::FINISHED; + } + return AudioDecoderState::FAILED; + } + + FileDecoderState state = FileDecoderState::MORE_TO_PROCESS; + uint8_t no_output_iterations = 0; + + while (state == FileDecoderState::MORE_TO_PROCESS) { + // Transfer decoded out + if (!this->pause_output_) { + // Never shift the data in the output transfer buffer to avoid unnecessary, slow data moves + size_t bytes_written = + this->output_transfer_buffer_->transfer_data_to_sink(pdMS_TO_TICKS(READ_WRITE_TIMEOUT_MS), false); + + if (this->audio_stream_info_.has_value()) { + this->accumulated_frames_written_ += this->audio_stream_info_.value().bytes_to_frames(bytes_written); + this->playback_ms_ += + this->audio_stream_info_.value().frames_to_milliseconds_with_remainder(&this->accumulated_frames_written_); + } + + if ((bytes_written > 0) && (this->output_transfer_buffer_->available() == 0)) { + // All decoded audio has been flushed to the sink; return so the caller can react to stop/pause before + // decoding the next batch + return AudioDecoderState::DECODING; + } + } else { + // If paused, block to avoid wasting CPU resources + delay(READ_WRITE_TIMEOUT_MS); + } + + if (this->output_transfer_buffer_->available() > 0) { + // Output transfer buffer indicates backpressure, return so caller can handle other events; + // e.g., stop/pause, before trying again + return AudioDecoderState::DECODING; + } + + // Reaching here means no decoded output is pending (any would have returned above). Bounds long no-output + // stretches; e.g., skipping a large metadata block, so a source that keeps the ring buffer full can't spin this + // loop without yielding and trip the watchdog. The delay yields allowing other tasks to feed the watchdog and + // the return keeps stop/pause responsive. + if (++no_output_iterations >= MAX_NO_OUTPUT_ITERATIONS) { + delay(1); + return AudioDecoderState::DECODING; + } + + // Expose the next chunk of file data. Every decoder buffers internally and consumes only what it + // processed, so the source does not need to accumulate or stitch chunks across fill() calls. + this->input_buffer_->fill(pdMS_TO_TICKS(READ_WRITE_TIMEOUT_MS), false); + + const size_t available_before_decode = this->input_buffer_->available(); + + if (available_before_decode == 0) { + // No data to decode, attempt to get more data next time + state = FileDecoderState::IDLE; + } else { + switch (this->audio_file_type_) { +#ifdef USE_AUDIO_FLAC_SUPPORT + case AudioFileType::FLAC: + state = this->decode_flac_(); + break; +#endif +#ifdef USE_AUDIO_MP3_SUPPORT + case AudioFileType::MP3: + state = this->decode_mp3_(); + break; +#endif +#ifdef USE_AUDIO_OPUS_SUPPORT + case AudioFileType::OPUS: + state = this->decode_opus_(); + break; +#endif +#ifdef USE_AUDIO_WAV_SUPPORT + case AudioFileType::WAV: + state = this->decode_wav_(); + break; +#endif + case AudioFileType::NONE: + default: + state = FileDecoderState::IDLE; + break; + } + } + + if (state == FileDecoderState::POTENTIALLY_FAILED) { + ++this->potentially_failed_count_; + } else if (state == FileDecoderState::END_OF_FILE) { + this->end_of_file_ = true; + } else if (state == FileDecoderState::FAILED) { + return AudioDecoderState::FAILED; + } else if (state == FileDecoderState::MORE_TO_PROCESS) { + // Reset the failsafe only when the iteration made forward progress: input was consumed or output was + // produced (output_transfer_buffer_ is drained empty above, so any available bytes are new). A + // MORE_TO_PROCESS that neither consumes input nor produces output means the decoder is stalled; count it + // toward the failsafe so a stuck stream eventually surfaces as FAILED instead of looping forever. + if ((this->input_buffer_->available() < available_before_decode) || + (this->output_transfer_buffer_->available() > 0)) { + this->potentially_failed_count_ = 0; + } else { + ++this->potentially_failed_count_; + } + } + } + return AudioDecoderState::DECODING; +} + +#ifdef USE_AUDIO_FLAC_SUPPORT +FileDecoderState AudioDecoder::decode_flac_() { + size_t bytes_consumed, samples_decoded; + + micro_flac::FLACDecoderResult result = this->flac_decoder_->decode( + this->input_buffer_->data(), this->input_buffer_->available(), this->output_transfer_buffer_->get_buffer_end(), + this->output_transfer_buffer_->free(), bytes_consumed, samples_decoded); + + if (result == micro_flac::FLAC_DECODER_SUCCESS) { + if (samples_decoded > 0 && this->audio_stream_info_.has_value()) { + this->output_transfer_buffer_->increase_buffer_length( + this->audio_stream_info_.value().samples_to_bytes(samples_decoded)); + } + this->input_buffer_->consume(bytes_consumed); + } else if (result == micro_flac::FLAC_DECODER_HEADER_READY) { + // Header just parsed, stream info now available + const auto &info = this->flac_decoder_->get_stream_info(); + this->audio_stream_info_ = audio::AudioStreamInfo(info.bits_per_sample(), info.num_channels(), info.sample_rate()); + + // Reallocate the output transfer buffer to the required size + this->free_buffer_required_ = this->flac_decoder_->get_output_buffer_size_samples() * info.bytes_per_sample(); + if (!this->output_transfer_buffer_->reallocate(this->free_buffer_required_)) { + return FileDecoderState::FAILED; + } + this->input_buffer_->consume(bytes_consumed); + } else if (result == micro_flac::FLAC_DECODER_END_OF_STREAM) { + this->input_buffer_->consume(bytes_consumed); + return FileDecoderState::END_OF_FILE; + } else if (result == micro_flac::FLAC_DECODER_NEED_MORE_DATA) { + this->input_buffer_->consume(bytes_consumed); + return FileDecoderState::MORE_TO_PROCESS; + } else if (result == micro_flac::FLAC_DECODER_ERROR_OUTPUT_TOO_SMALL) { + // Reallocate to decode the frame on the next call + const auto &info = this->flac_decoder_->get_stream_info(); + this->free_buffer_required_ = this->flac_decoder_->get_output_buffer_size_samples() * info.bytes_per_sample(); + if (!this->output_transfer_buffer_->reallocate(this->free_buffer_required_)) { + return FileDecoderState::FAILED; + } + } else { + ESP_LOGE(TAG, "FLAC decoder failed: %d", static_cast(result)); + return FileDecoderState::POTENTIALLY_FAILED; + } + + return FileDecoderState::MORE_TO_PROCESS; +} +#endif + +#ifdef USE_AUDIO_MP3_SUPPORT +FileDecoderState AudioDecoder::decode_mp3_() { + // microMP3's samples_decoded value is samples per channel; e.g., what ESPHome typically calls an audio frame. + // microMP3 uses the term frame to refer to an MP3 frame: an encoded packet that contains multiple audio frames. + size_t bytes_consumed = 0; + size_t samples_decoded = 0; + + // microMP3 buffers internally: it consumes from our input buffer at its own pace, emits MP3_STREAM_INFO_READY once + // the first frame header is parsed, and only then produces PCM. It handles sync-word search and ID3v2 tag skipping. + micro_mp3::Mp3Result result = this->mp3_decoder_->decode( + this->input_buffer_->data(), this->input_buffer_->available(), this->output_transfer_buffer_->get_buffer_end(), + this->output_transfer_buffer_->free(), bytes_consumed, samples_decoded); + + this->input_buffer_->consume(bytes_consumed); + + if (result == micro_mp3::MP3_OK) { + if (samples_decoded > 0 && this->audio_stream_info_.has_value()) { + this->output_transfer_buffer_->increase_buffer_length( + this->audio_stream_info_.value().frames_to_bytes(samples_decoded)); + } + } else if (result == micro_mp3::MP3_STREAM_INFO_READY || result == micro_mp3::MP3_STREAM_INFO_CHANGED) { + // Header parsed: capture stream info and resize the output buffer to fit one full frame. + // microMP3 always outputs 16-bit PCM. MP3_STREAM_INFO_CHANGED is handled identically: despite its + // negative value it is documented as recoverable, so it must not reach the catch-all below. + this->audio_stream_info_ = + audio::AudioStreamInfo(16, this->mp3_decoder_->get_channels(), this->mp3_decoder_->get_sample_rate()); + this->free_buffer_required_ = + this->mp3_decoder_->get_samples_per_frame() * this->mp3_decoder_->get_channels() * sizeof(int16_t); + if (!this->output_transfer_buffer_->reallocate(this->free_buffer_required_)) { + return FileDecoderState::FAILED; + } + } else if (result == micro_mp3::MP3_NEED_MORE_DATA) { + return FileDecoderState::MORE_TO_PROCESS; + } else if (result == micro_mp3::MP3_OUTPUT_BUFFER_TOO_SMALL) { + // Fallback to worst-case size + this->free_buffer_required_ = this->mp3_decoder_->get_min_output_buffer_bytes(); + if (!this->output_transfer_buffer_->reallocate(this->free_buffer_required_)) { + return FileDecoderState::FAILED; + } + } else if (result == micro_mp3::MP3_DECODE_ERROR) { + // Corrupt frame skipped; recoverable, retry on next call + ESP_LOGW(TAG, "MP3 decoder skipped a corrupt frame"); + return FileDecoderState::POTENTIALLY_FAILED; + } else { + // MP3_ALLOCATION_FAILED, MP3_INPUT_INVALID, or any future error -- not recoverable + ESP_LOGE(TAG, "MP3 decoder failed: %d", static_cast(result)); + return FileDecoderState::FAILED; + } + + return FileDecoderState::MORE_TO_PROCESS; +} +#endif + +#ifdef USE_AUDIO_OPUS_SUPPORT +FileDecoderState AudioDecoder::decode_opus_() { + bool processed_header = this->opus_decoder_->is_initialized(); + + size_t bytes_consumed, samples_decoded; + + micro_opus::OggOpusResult result = this->opus_decoder_->decode( + this->input_buffer_->data(), this->input_buffer_->available(), this->output_transfer_buffer_->get_buffer_end(), + this->output_transfer_buffer_->free(), bytes_consumed, samples_decoded); + + if (result == micro_opus::OGG_OPUS_OK) { + if (!processed_header && this->opus_decoder_->is_initialized()) { + // Header processed and stream info is available + this->audio_stream_info_ = + audio::AudioStreamInfo(this->opus_decoder_->get_bit_depth(), this->opus_decoder_->get_channels(), + this->opus_decoder_->get_sample_rate()); + } + if (samples_decoded > 0 && this->audio_stream_info_.has_value()) { + // Some audio was processed + this->output_transfer_buffer_->increase_buffer_length( + this->audio_stream_info_.value().frames_to_bytes(samples_decoded)); + } + this->input_buffer_->consume(bytes_consumed); + } else if (result == micro_opus::OGG_OPUS_OUTPUT_BUFFER_TOO_SMALL) { + // Reallocate to decode the packet on the next call + this->free_buffer_required_ = this->opus_decoder_->get_required_output_buffer_size(); + if (!this->output_transfer_buffer_->reallocate(this->free_buffer_required_)) { + // Couldn't reallocate output buffer + return FileDecoderState::FAILED; + } + } else { + ESP_LOGE(TAG, "Opus decoder failed: %" PRId8, result); + return FileDecoderState::POTENTIALLY_FAILED; + } + return FileDecoderState::MORE_TO_PROCESS; +} +#endif + +#ifdef USE_AUDIO_WAV_SUPPORT +FileDecoderState AudioDecoder::decode_wav_() { + // microWAV's samples_decoded counts individual channel samples; e.g., for + // 16-bit stereo, 4 input bytes results in 2 samples_decoded. + size_t bytes_consumed = 0; + size_t samples_decoded = 0; + + micro_wav::WAVDecoderResult result = this->wav_decoder_->decode( + this->input_buffer_->data(), this->input_buffer_->available(), this->output_transfer_buffer_->get_buffer_end(), + this->output_transfer_buffer_->free(), bytes_consumed, samples_decoded); + + this->input_buffer_->consume(bytes_consumed); + + if (result == micro_wav::WAV_DECODER_SUCCESS) { + if (samples_decoded > 0 && this->audio_stream_info_.has_value()) { + this->output_transfer_buffer_->increase_buffer_length( + this->audio_stream_info_.value().samples_to_bytes(samples_decoded)); + } + } else if (result == micro_wav::WAV_DECODER_HEADER_READY) { + // After HEADER_READY, get_bits_per_sample() returns the output bit depth + // (16 for A-law/mu-law, 32 for IEEE float, original value for PCM). + this->audio_stream_info_ = + audio::AudioStreamInfo(this->wav_decoder_->get_bits_per_sample(), this->wav_decoder_->get_channels(), + this->wav_decoder_->get_sample_rate()); + } else if (result == micro_wav::WAV_DECODER_NEED_MORE_DATA) { + return FileDecoderState::MORE_TO_PROCESS; + } else if (result == micro_wav::WAV_DECODER_END_OF_STREAM) { + return FileDecoderState::END_OF_FILE; + } else { + ESP_LOGE(TAG, "WAV decoder failed: %d", static_cast(result)); + return FileDecoderState::FAILED; + } + + return FileDecoderState::MORE_TO_PROCESS; +} +#endif + +} // namespace esphome::audio + +#endif diff --git a/firmware/esphome/components/audio/audio_decoder.h b/firmware/esphome/components/audio/audio_decoder.h new file mode 100644 index 0000000..e772b7e --- /dev/null +++ b/firmware/esphome/components/audio/audio_decoder.h @@ -0,0 +1,160 @@ +#pragma once + +#ifdef USE_ESP32 + +#include "audio.h" +#include "audio_transfer_buffer.h" + +#include "esphome/components/ring_buffer/ring_buffer.h" +#include "esphome/core/defines.h" +#include "esphome/core/helpers.h" + +#ifdef USE_SPEAKER +#include "esphome/components/speaker/speaker.h" +#endif + +#include "esp_err.h" + +// micro-flac +#ifdef USE_AUDIO_FLAC_SUPPORT +#include +#endif + +// micro-mp3 +#ifdef USE_AUDIO_MP3_SUPPORT +#include +#endif + +// micro-opus +#ifdef USE_AUDIO_OPUS_SUPPORT +#include +#endif + +// micro-wav +#ifdef USE_AUDIO_WAV_SUPPORT +#include +#endif + +namespace esphome::audio { + +enum class AudioDecoderState : uint8_t { + DECODING = 0, // More data is available to decode + FINISHED, // All file data has been decoded and transferred + FAILED, // Encountered an error +}; + +// Only used within the AudioDecoder class; conveys the state of the particular file type decoder +enum class FileDecoderState : uint8_t { + MORE_TO_PROCESS, // Successsfully read a file chunk and more data is available to decode + IDLE, // Not enough data to decode, waiting for more to be transferred + POTENTIALLY_FAILED, // Decoder encountered a potentially recoverable error if more file data is available + FAILED, // Decoder encoutnered an uncrecoverable error + END_OF_FILE, // The specific file decoder knows its the end of the file +}; + +class AudioDecoder { + /* + * @brief Class that facilitates decoding an audio file. + * The audio file is read from a source (ring buffer or const data pointer), decoded, and sent to an audio sink + * (ring buffer, speaker component, or callback). + * Supports flac, mp3, ogg opus, and wav formats (each enabled independently at compile time). + */ + public: + /// @brief Allocates the output transfer buffer and stores the input buffer size for later use by add_source() + /// @param input_buffer_size Soft cap on the bytes a ring buffer source exposes per fill, in bytes. + /// @param output_buffer_size Size of the output transfer buffer in bytes. + AudioDecoder(size_t input_buffer_size, size_t output_buffer_size); + + ~AudioDecoder() = default; + + /// @brief Adds a source ring buffer for raw file data. Shares ownership of the ring buffer via a shared_ptr. + /// The decoder reads directly from the ring buffer's internal storage with a zero-copy RingBufferAudioSource. + /// @param input_ring_buffer weak_ptr of the source ring buffer to read from + /// @return ESP_OK if successful, ESP_ERR_INVALID_ARG if the ring buffer is expired or the buffer size is zero + esp_err_t add_source(std::weak_ptr &input_ring_buffer); + + /// @brief Adds a sink ring buffer for decoded audio. Takes ownership of the ring buffer in a shared_ptr. + /// @param output_ring_buffer weak_ptr of a shared_ptr of the sink ring buffer to transfer ownership + /// @return ESP_OK if successsful, ESP_ERR_NO_MEM if the transfer buffer wasn't allocated + esp_err_t add_sink(std::weak_ptr &output_ring_buffer); + +#ifdef USE_SPEAKER + /// @brief Adds a sink speaker for decoded audio. + /// @param speaker pointer to speaker component + /// @return ESP_OK if successsful, ESP_ERR_NO_MEM if the transfer buffer wasn't allocated + esp_err_t add_sink(speaker::Speaker *speaker); +#endif + + /// @brief Adds a const data pointer as the source for raw file data. Does not allocate a transfer buffer. + /// @param data_pointer Pointer to the const audio data (e.g., stored in flash memory) + /// @param length Size of the data in bytes + /// @return ESP_OK + esp_err_t add_source(const uint8_t *data_pointer, size_t length); + + /// @brief Adds a callback as the sink for decoded audio. + /// @param callback Pointer to the AudioSinkCallback implementation + /// @return ESP_OK if successful, ESP_ERR_NO_MEM if the transfer buffer wasn't allocated + esp_err_t add_sink(AudioSinkCallback *callback); + + /// @brief Sets up decoding the file + /// @param audio_file_type AudioFileType of the file + /// @return ESP_OK if successful, ESP_ERR_NO_MEM if the transfer buffers fail to allocate, or ESP_ERR_NOT_SUPPORTED if + /// the format isn't supported. + esp_err_t start(AudioFileType audio_file_type); + + /// @brief Decodes audio from the ring buffer source and writes to the sink. + /// @param stop_gracefully If true, it indicates the file source is finished. The decoder will decode all the + /// reamining data and then finish. + /// @return AudioDecoderState + AudioDecoderState decode(bool stop_gracefully); + + /// @brief Gets the audio stream information, if it has been decoded from the files header + /// @return optional with the audio information. If not available yet, returns no value. + const optional &get_audio_stream_info() const { return this->audio_stream_info_; } + + /// @brief Returns the duration of audio (in milliseconds) decoded and sent to the sink + /// @return Duration of decoded audio in milliseconds + uint32_t get_playback_ms() const { return this->playback_ms_; } + + /// @brief Pauses sending resampled audio to the sink. If paused, it will continue to process internal buffers. + /// @param pause_state If true, audio data is not sent to the sink. + void set_pause_output_state(bool pause_state) { this->pause_output_ = pause_state; } + + protected: +#ifdef USE_AUDIO_FLAC_SUPPORT + FileDecoderState decode_flac_(); + std::unique_ptr flac_decoder_; +#endif +#ifdef USE_AUDIO_MP3_SUPPORT + FileDecoderState decode_mp3_(); + std::unique_ptr mp3_decoder_; +#endif +#ifdef USE_AUDIO_OPUS_SUPPORT + FileDecoderState decode_opus_(); + std::unique_ptr opus_decoder_; +#endif +#ifdef USE_AUDIO_WAV_SUPPORT + FileDecoderState decode_wav_(); + std::unique_ptr wav_decoder_; +#endif + + std::unique_ptr input_buffer_; + std::unique_ptr output_transfer_buffer_; + + AudioFileType audio_file_type_{AudioFileType::NONE}; + optional audio_stream_info_{}; + + size_t input_buffer_size_{0}; + size_t free_buffer_required_{0}; + + uint32_t potentially_failed_count_{0}; + uint32_t accumulated_frames_written_{0}; + uint32_t playback_ms_{0}; + + bool end_of_file_{false}; + + bool pause_output_{false}; +}; +} // namespace esphome::audio + +#endif diff --git a/firmware/esphome/components/audio/audio_reader.cpp b/firmware/esphome/components/audio/audio_reader.cpp new file mode 100644 index 0000000..769a5a6 --- /dev/null +++ b/firmware/esphome/components/audio/audio_reader.cpp @@ -0,0 +1,316 @@ +#include "audio_reader.h" +#ifdef OPENATHAN_QURAN +#include "esphome/components/openathan_audio/quran_http.h" +#endif + +#ifdef USE_ESP32 + +#include "esphome/core/defines.h" +#include "esphome/core/hal.h" +#include "esphome/core/helpers.h" +#include "esphome/core/log.h" + +#if CONFIG_MBEDTLS_CERTIFICATE_BUNDLE +#include "esp_crt_bundle.h" +#endif + +namespace esphome::audio { + +static const uint32_t READ_WRITE_TIMEOUT_MS = 20; + +static const uint32_t CONNECTION_TIMEOUT_MS = 5000; +static const uint8_t MAX_FETCHING_HEADER_ATTEMPTS = 6; + +static const size_t HTTP_STREAM_BUFFER_SIZE = 2048; + +static const uint8_t MAX_REDIRECTIONS = 5; + +static const char *const TAG = "audio_reader"; + +// Some common HTTP status codes - borrowed from http_request component accessed 20241224 +enum HttpStatus { + HTTP_STATUS_OK = 200, + HTTP_STATUS_NO_CONTENT = 204, + HTTP_STATUS_PARTIAL_CONTENT = 206, + + /* 3xx - Redirection */ + HTTP_STATUS_MULTIPLE_CHOICES = 300, + HTTP_STATUS_MOVED_PERMANENTLY = 301, + HTTP_STATUS_FOUND = 302, + HTTP_STATUS_SEE_OTHER = 303, + HTTP_STATUS_NOT_MODIFIED = 304, + HTTP_STATUS_TEMPORARY_REDIRECT = 307, + HTTP_STATUS_PERMANENT_REDIRECT = 308, + + /* 4XX - CLIENT ERROR */ + HTTP_STATUS_BAD_REQUEST = 400, + HTTP_STATUS_UNAUTHORIZED = 401, + HTTP_STATUS_FORBIDDEN = 403, + HTTP_STATUS_NOT_FOUND = 404, + HTTP_STATUS_METHOD_NOT_ALLOWED = 405, + HTTP_STATUS_NOT_ACCEPTABLE = 406, + HTTP_STATUS_LENGTH_REQUIRED = 411, + + /* 5xx - Server Error */ + HTTP_STATUS_INTERNAL_ERROR = 500 +}; + +AudioReader::~AudioReader() { this->cleanup_connection_(); } + +esp_err_t AudioReader::add_sink(const std::weak_ptr &output_ring_buffer) { + if (current_audio_file_ != nullptr) { + // A transfer buffer isn't ncessary for a local file + this->file_ring_buffer_ = output_ring_buffer.lock(); + if (this->file_ring_buffer_ == nullptr) { + return ESP_ERR_INVALID_STATE; + } + return ESP_OK; + } + + if (this->output_transfer_buffer_ != nullptr) { + this->output_transfer_buffer_->set_sink(output_ring_buffer); + return ESP_OK; + } + + return ESP_ERR_INVALID_STATE; +} + +esp_err_t AudioReader::start(AudioFile *audio_file, AudioFileType &file_type) { + file_type = AudioFileType::NONE; + + this->current_audio_file_ = audio_file; + + this->file_current_ = audio_file->data; + file_type = audio_file->file_type; + + return ESP_OK; +} + +esp_err_t AudioReader::start(const std::string &uri, AudioFileType &file_type) { +#ifdef OPENATHAN_QURAN + file_type = AudioFileType::NONE; + cleanup_connection_(); + openathan_device::QuranHttp response; + if (!response.open(uri, true, HTTP_STREAM_BUFFER_SIZE, CONNECTION_TIMEOUT_MS)) return ESP_FAIL; + file_type = detect_audio_file_type(response.content_type.c_str(), nullptr); + if (file_type != AudioFileType::MP3) return ESP_ERR_NOT_SUPPORTED; + client_ = response.release(); + last_data_read_ms_ = millis(); + output_transfer_buffer_ = AudioSinkTransferBuffer::create(buffer_size_); + if (!output_transfer_buffer_) { cleanup_connection_(); return ESP_ERR_NO_MEM; } + return ESP_OK; +#else + + file_type = AudioFileType::NONE; + + this->cleanup_connection_(); + + if (uri.empty()) { + return ESP_ERR_INVALID_ARG; + } + + esp_http_client_config_t client_config = {}; + + client_config.url = uri.c_str(); + client_config.cert_pem = nullptr; + client_config.disable_auto_redirect = false; + client_config.max_redirection_count = MAX_REDIRECTIONS; + client_config.event_handler = http_event_handler; + client_config.user_data = this; + client_config.buffer_size = HTTP_STREAM_BUFFER_SIZE; + client_config.keep_alive_enable = true; + client_config.timeout_ms = CONNECTION_TIMEOUT_MS; // Shouldn't trigger watchdog resets if caller runs in a task + +#if CONFIG_MBEDTLS_CERTIFICATE_BUNDLE + if (uri.find("https:") != std::string::npos) { + client_config.crt_bundle_attach = esp_crt_bundle_attach; + } +#endif + + this->client_ = esp_http_client_init(&client_config); + + if (this->client_ == nullptr) { + return ESP_FAIL; + } + + esp_err_t err = esp_http_client_open(this->client_, 0); + + if (err != ESP_OK) { + ESP_LOGE(TAG, "Failed to open URL"); + this->cleanup_connection_(); + return err; + } + + int64_t header_length = esp_http_client_fetch_headers(this->client_); + uint8_t reattempt_count = 0; + while ((header_length < 0) && (reattempt_count < MAX_FETCHING_HEADER_ATTEMPTS)) { + this->cleanup_connection_(); + if (header_length != -ESP_ERR_HTTP_EAGAIN) { + // Serious error, no recovery + return ESP_FAIL; + } else { + // Reconnect from a fresh state to avoid a bug where it never reads the headers even if made available + this->client_ = esp_http_client_init(&client_config); + esp_http_client_open(this->client_, 0); + header_length = esp_http_client_fetch_headers(this->client_); + ++reattempt_count; + } + } + + if (header_length < 0) { + ESP_LOGE(TAG, "Failed to fetch headers"); + this->cleanup_connection_(); + return ESP_FAIL; + } + + int status_code = esp_http_client_get_status_code(this->client_); + + if ((status_code < HTTP_STATUS_OK) || (status_code > HTTP_STATUS_PERMANENT_REDIRECT)) { + this->cleanup_connection_(); + return ESP_FAIL; + } + + ssize_t redirect_count = 0; + + while ((esp_http_client_set_redirection(this->client_) == ESP_OK) && (redirect_count < MAX_REDIRECTIONS)) { + err = esp_http_client_open(this->client_, 0); + if (err != ESP_OK) { + this->cleanup_connection_(); + return ESP_FAIL; + } + + header_length = esp_http_client_fetch_headers(this->client_); + if (header_length < 0) { + this->cleanup_connection_(); + return ESP_FAIL; + } + + status_code = esp_http_client_get_status_code(this->client_); + + if ((status_code < HTTP_STATUS_OK) || (status_code > HTTP_STATUS_PERMANENT_REDIRECT)) { + this->cleanup_connection_(); + return ESP_FAIL; + } + + ++redirect_count; + } + + if (this->audio_file_type_ == AudioFileType::NONE) { + // Failed to determine the file type from the header, fallback to using the url + char url[500]; + err = esp_http_client_get_url(this->client_, url, 500); + if (err != ESP_OK) { + this->cleanup_connection_(); + return err; + } + + file_type = detect_audio_file_type(nullptr, url); + if (file_type == AudioFileType::NONE) { + this->cleanup_connection_(); + return ESP_ERR_NOT_SUPPORTED; + } + } else { + file_type = this->audio_file_type_; + } + + this->last_data_read_ms_ = millis(); + + this->output_transfer_buffer_ = AudioSinkTransferBuffer::create(this->buffer_size_); + if (this->output_transfer_buffer_ == nullptr) { + return ESP_ERR_NO_MEM; + } + + return ESP_OK; +#endif +} + +AudioReaderState AudioReader::read() { + if (this->client_ != nullptr) { + return this->http_read_(); + } else if (this->current_audio_file_ != nullptr) { + return this->file_read_(); + } + + return AudioReaderState::FAILED; +} + +esp_err_t AudioReader::http_event_handler(esp_http_client_event_t *evt) { + // Based on https://github.com/maroc81/WeatherLily/tree/main/main/net accessed 20241224 + AudioReader *this_reader = (AudioReader *) evt->user_data; + if (!this_reader) return ESP_OK; + + switch (evt->event_id) { + case HTTP_EVENT_ON_HEADER: + + if (strcasecmp(evt->header_key, "Content-Type") == 0) { + this_reader->audio_file_type_ = detect_audio_file_type(evt->header_value, nullptr); + } + break; + default: + break; + } + return ESP_OK; +} + +AudioReaderState AudioReader::file_read_() { + size_t remaining_bytes = this->current_audio_file_->length - (this->file_current_ - this->current_audio_file_->data); + if (remaining_bytes > 0) { + size_t bytes_written = this->file_ring_buffer_->write_without_replacement(this->file_current_, remaining_bytes, + pdMS_TO_TICKS(READ_WRITE_TIMEOUT_MS)); + this->file_current_ += bytes_written; + + return AudioReaderState::READING; + } + + return AudioReaderState::FINISHED; +} + +AudioReaderState AudioReader::http_read_() { + this->output_transfer_buffer_->transfer_data_to_sink(pdMS_TO_TICKS(READ_WRITE_TIMEOUT_MS), false); + + if (esp_http_client_is_complete_data_received(this->client_)) { + if (this->output_transfer_buffer_->available() == 0) { + this->cleanup_connection_(); + return AudioReaderState::FINISHED; + } + } else if (this->output_transfer_buffer_->free() > 0) { + int received_len = esp_http_client_read(this->client_, (char *) this->output_transfer_buffer_->get_buffer_end(), + this->output_transfer_buffer_->free()); + + if (received_len > 0) { + this->output_transfer_buffer_->increase_buffer_length(received_len); + this->last_data_read_ms_ = millis(); + return AudioReaderState::READING; + } else if (received_len <= 0) { + // HTTP read error + if (received_len == -1) { + // A true connection error occured, no chance at recovery + this->cleanup_connection_(); + return AudioReaderState::FAILED; + } + + // Read timed out, manually verify if it has been too long since the last successful read + if ((millis() - this->last_data_read_ms_) > MAX_FETCHING_HEADER_ATTEMPTS * CONNECTION_TIMEOUT_MS) { + ESP_LOGE(TAG, "Timed out"); + this->cleanup_connection_(); + return AudioReaderState::FAILED; + } + + delay(READ_WRITE_TIMEOUT_MS); + } + } + + return AudioReaderState::READING; +} + +void AudioReader::cleanup_connection_() { + if (this->client_ != nullptr) { + esp_http_client_close(this->client_); + esp_http_client_cleanup(this->client_); + this->client_ = nullptr; + } +} + +} // namespace esphome::audio + +#endif diff --git a/firmware/esphome/components/audio/audio_reader.h b/firmware/esphome/components/audio/audio_reader.h new file mode 100644 index 0000000..b1f7617 --- /dev/null +++ b/firmware/esphome/components/audio/audio_reader.h @@ -0,0 +1,78 @@ +#pragma once + +#ifdef USE_ESP32 + +#include "audio.h" +#include "audio_transfer_buffer.h" + +#include "esphome/components/ring_buffer/ring_buffer.h" + +#include "esp_err.h" + +#include + +namespace esphome::audio { + +enum class AudioReaderState : uint8_t { + READING = 0, // More data is available to read + FINISHED, // All data has been read and transferred + FAILED, // Encountered an error +}; + +class AudioReader { + /* + * @brief Class that facilitates reading a raw audio file. + * Files can be read from flash (stored in a AudioFile struct) or from an http source. + * The file data is sent to a ring buffer sink. + */ + public: + /// @brief Constructs an AudioReader object. + /// The transfer buffer isn't allocated here, but only if necessary (an http source) in the start function. + /// @param buffer_size Transfer buffer size in bytes. + AudioReader(size_t buffer_size) : buffer_size_(buffer_size) {} + ~AudioReader(); + + /// @brief Adds a sink ring buffer for audio data. Takes ownership of the ring buffer in a shared_ptr + /// @param output_ring_buffer weak_ptr of a shared_ptr of the sink ring buffer to transfer ownership + /// @return ESP_OK if successful, ESP_ERR_INVALID_STATE otherwise + esp_err_t add_sink(const std::weak_ptr &output_ring_buffer); + + /// @brief Starts reading an audio file from an http source. The transfer buffer is allocated here. + /// @param uri Web url to the http file. + /// @param file_type AudioFileType variable passed-by-reference indicating the type of file being read. + /// @return ESP_OK if successful, an ESP_ERR* code otherwise. + esp_err_t start(const std::string &uri, AudioFileType &file_type); + + /// @brief Starts reading an audio file from flash. No transfer buffer is allocated. + /// @param audio_file AudioFile struct containing the file. + /// @param file_type AudioFileType variable passed-by-reference indicating the type of file being read. + /// @return ESP_OK + esp_err_t start(AudioFile *audio_file, AudioFileType &file_type); + + /// @brief Reads new file data from the source and sends to the ring buffer sink. + /// @return AudioReaderState + AudioReaderState read(); + + protected: + /// @brief Monitors the http client events to attempt determining the file type from the Content-Type header + static esp_err_t http_event_handler(esp_http_client_event_t *evt); + + AudioReaderState file_read_(); + AudioReaderState http_read_(); + + std::shared_ptr file_ring_buffer_; + std::unique_ptr output_transfer_buffer_; + void cleanup_connection_(); + + size_t buffer_size_; + uint32_t last_data_read_ms_; + + esp_http_client_handle_t client_{nullptr}; + + AudioFile *current_audio_file_{nullptr}; + AudioFileType audio_file_type_{AudioFileType::NONE}; + const uint8_t *file_current_{nullptr}; +}; +} // namespace esphome::audio + +#endif diff --git a/firmware/esphome/components/audio/audio_resampler.cpp b/firmware/esphome/components/audio/audio_resampler.cpp new file mode 100644 index 0000000..bef62ce --- /dev/null +++ b/firmware/esphome/components/audio/audio_resampler.cpp @@ -0,0 +1,190 @@ +#include "audio_resampler.h" + +#ifdef USE_ESP32 + +#include "esphome/core/hal.h" + +#include + +namespace esphome::audio { + +static const uint32_t READ_WRITE_TIMEOUT_MS = 20; + +AudioResampler::AudioResampler(size_t input_buffer_size, size_t output_buffer_size) + : input_buffer_size_(input_buffer_size), output_buffer_size_(output_buffer_size) { + this->output_transfer_buffer_ = AudioSinkTransferBuffer::create(output_buffer_size); +} + +esp_err_t AudioResampler::add_source(std::weak_ptr &input_ring_buffer) { + // The zero-copy RingBufferAudioSource is created lazily on the first resample() call, once both the ring + // buffer (stored here) and the input stream info (set by start()) are available, in either order. + this->source_ring_buffer_ = input_ring_buffer.lock(); + if (this->source_ring_buffer_ == nullptr) { + return ESP_ERR_INVALID_STATE; + } + return ESP_OK; +} + +esp_err_t AudioResampler::add_sink(std::weak_ptr &output_ring_buffer) { + if (this->output_transfer_buffer_ != nullptr) { + this->output_transfer_buffer_->set_sink(output_ring_buffer); + return ESP_OK; + } + return ESP_ERR_NO_MEM; +} + +#ifdef USE_SPEAKER +esp_err_t AudioResampler::add_sink(speaker::Speaker *speaker) { + if (this->output_transfer_buffer_ != nullptr) { + this->output_transfer_buffer_->set_sink(speaker); + return ESP_OK; + } + return ESP_ERR_NO_MEM; +} +#endif + +esp_err_t AudioResampler::start(AudioStreamInfo &input_stream_info, AudioStreamInfo &output_stream_info, + uint16_t number_of_taps, uint16_t number_of_filters) { + this->input_stream_info_ = input_stream_info; + this->output_stream_info_ = output_stream_info; + + if (this->output_transfer_buffer_ == nullptr) { + return ESP_ERR_NO_MEM; + } + + if ((input_stream_info.get_bits_per_sample() > 32) || (output_stream_info.get_bits_per_sample() > 32) || + (input_stream_info_.get_channels() != output_stream_info.get_channels())) { + return ESP_ERR_NOT_SUPPORTED; + } + + // Reject frame sizes that can't be used as the zero-copy source's alignment up front, where the caller checks + // the return code. The lazy create() in resample() keeps its own guard since it runs before the uint8_t cast. + const size_t bytes_per_frame = this->input_stream_info_.frames_to_bytes(1); + if ((bytes_per_frame == 0) || (bytes_per_frame > RingBufferAudioSource::MAX_ALIGNMENT_BYTES)) { + return ESP_ERR_NOT_SUPPORTED; + } + + if ((input_stream_info.get_sample_rate() != output_stream_info.get_sample_rate()) || + (input_stream_info.get_bits_per_sample() != output_stream_info.get_bits_per_sample())) { + this->resampler_ = make_unique( + input_stream_info.bytes_to_samples(this->input_buffer_size_), + output_stream_info.bytes_to_samples(this->output_buffer_size_)); + + // Use cascaded biquad filters when downsampling to avoid aliasing + bool use_pre_filter = output_stream_info.get_sample_rate() < input_stream_info.get_sample_rate(); + + esp_audio_libs::resampler::ResamplerConfiguration resample_config = { + .source_sample_rate = static_cast(input_stream_info.get_sample_rate()), + .target_sample_rate = static_cast(output_stream_info.get_sample_rate()), + .source_bits_per_sample = input_stream_info.get_bits_per_sample(), + .target_bits_per_sample = output_stream_info.get_bits_per_sample(), + .channels = input_stream_info_.get_channels(), + .use_pre_or_post_filter = use_pre_filter, + .subsample_interpolate = false, // Doubles the CPU load. Using more filters is a better alternative + .number_of_taps = number_of_taps, + .number_of_filters = number_of_filters, + }; + + if (!this->resampler_->initialize(resample_config)) { + // Failed to allocate the resampler's internal buffers + return ESP_ERR_NO_MEM; + } + } + + return ESP_OK; +} + +AudioResamplerState AudioResampler::resample(bool stop_gracefully, int32_t *ms_differential) { + if (this->audio_source_ == nullptr) { + // Lazily create the zero-copy source on first use. Frame-aligned reads ensure multi-channel frames are + // never split across the ring buffer's wrap boundary. + const size_t bytes_per_frame = this->input_stream_info_.frames_to_bytes(1); + if ((bytes_per_frame == 0) || (bytes_per_frame > RingBufferAudioSource::MAX_ALIGNMENT_BYTES)) { + // Stream info is unset or the frame is too large to use as an alignment; the uint8_t cast below would + // truncate it and could yield a source that tears frames. + return AudioResamplerState::FAILED; + } + // Pass the shared_ptr by copy so a failed create() leaves source_ring_buffer_ intact; release our + // reference only after the source has taken ownership. + this->audio_source_ = RingBufferAudioSource::create(this->source_ring_buffer_, this->input_buffer_size_, + static_cast(bytes_per_frame)); + if (this->audio_source_ == nullptr) { + return AudioResamplerState::FAILED; + } + this->source_ring_buffer_.reset(); + } + + if (stop_gracefully) { + if (!this->audio_source_->has_buffered_data() && (this->output_transfer_buffer_->available() == 0)) { + return AudioResamplerState::FINISHED; + } + } + + if (!this->pause_output_) { + // Move audio data to the sink without shifting the data in the output transfer buffer to avoid unnecessary, slow + // data moves + this->output_transfer_buffer_->transfer_data_to_sink(pdMS_TO_TICKS(READ_WRITE_TIMEOUT_MS), false); + } else { + // If paused, block to avoid wasting CPU resources + delay(READ_WRITE_TIMEOUT_MS); + } + + // Expose a chunk of the ring buffer's internal storage. pre_shift is ignored by RingBufferAudioSource + // (there is no intermediate transfer buffer to compact). + this->audio_source_->fill(pdMS_TO_TICKS(READ_WRITE_TIMEOUT_MS), false); + + if (this->audio_source_->available() == 0) { + // No samples available to process + return AudioResamplerState::RESAMPLING; + } + + const size_t bytes_free = this->output_transfer_buffer_->free(); + const uint32_t frames_free = this->output_stream_info_.bytes_to_frames(bytes_free); + + const size_t bytes_available = this->audio_source_->available(); + const uint32_t frames_available = this->input_stream_info_.bytes_to_frames(bytes_available); + + if ((this->input_stream_info_.get_sample_rate() != this->output_stream_info_.get_sample_rate()) || + (this->input_stream_info_.get_bits_per_sample() != this->output_stream_info_.get_bits_per_sample())) { + // Adjust gain by -3 dB to avoid clipping due to the resampling process + esp_audio_libs::resampler::ResamplerResults results = + this->resampler_->resample(this->audio_source_->data(), this->output_transfer_buffer_->get_buffer_end(), + frames_available, frames_free, -3); + + this->audio_source_->consume(this->input_stream_info_.frames_to_bytes(results.frames_used)); + this->output_transfer_buffer_->increase_buffer_length( + this->output_stream_info_.frames_to_bytes(results.frames_generated)); + + // Resampling causes slight differences in the durations used versus generated. Computes the difference in + // millisconds. The callback function passing the played audio duration uses the difference to convert from output + // duration to input duration. + this->accumulated_frames_used_ += results.frames_used; + this->accumulated_frames_generated_ += results.frames_generated; + + const int32_t used_ms = + this->input_stream_info_.frames_to_milliseconds_with_remainder(&this->accumulated_frames_used_); + const int32_t generated_ms = + this->output_stream_info_.frames_to_milliseconds_with_remainder(&this->accumulated_frames_generated_); + + *ms_differential = used_ms - generated_ms; + + } else { + // No resampling required, copy samples directly to the output transfer buffer + *ms_differential = 0; + + const size_t bytes_to_transfer = std::min(this->output_stream_info_.frames_to_bytes(frames_free), + this->input_stream_info_.frames_to_bytes(frames_available)); + + std::memcpy((void *) this->output_transfer_buffer_->get_buffer_end(), (const void *) this->audio_source_->data(), + bytes_to_transfer); + + this->audio_source_->consume(bytes_to_transfer); + this->output_transfer_buffer_->increase_buffer_length(bytes_to_transfer); + } + + return AudioResamplerState::RESAMPLING; +} + +} // namespace esphome::audio + +#endif diff --git a/firmware/esphome/components/audio/audio_resampler.h b/firmware/esphome/components/audio/audio_resampler.h new file mode 100644 index 0000000..c09070c --- /dev/null +++ b/firmware/esphome/components/audio/audio_resampler.h @@ -0,0 +1,103 @@ +#pragma once + +#ifdef USE_ESP32 + +#include "audio.h" +#include "audio_transfer_buffer.h" + +#include "esphome/components/ring_buffer/ring_buffer.h" +#include "esphome/core/defines.h" +#include "esphome/core/helpers.h" + +#ifdef USE_SPEAKER +#include "esphome/components/speaker/speaker.h" +#endif + +#include "esp_err.h" + +#include // esp-audio-libs + +namespace esphome::audio { + +enum class AudioResamplerState : uint8_t { + RESAMPLING, // More data is available to resample + FINISHED, // All file data has been resampled and transferred + FAILED, // Failed to allocate the audio source +}; + +class AudioResampler { + /* + * @brief Class that facilitates resampling audio. + * The audio data is read from a ring buffer source, resampled, and sent to an audio sink (ring buffer or speaker + * component). Also supports converting bits per sample. + */ + public: + /// @brief Allocates the output transfer buffer. The input source is created later in resample(). + /// @param input_buffer_size Max bytes exposed per fill() call on the zero-copy input source. + /// @param output_buffer_size Size of the output transfer buffer in bytes. + AudioResampler(size_t input_buffer_size, size_t output_buffer_size); + + /// @brief Sets the ring buffer the audio is read from and takes shared ownership of it. The zero-copy + /// RingBufferAudioSource that reads directly from its internal storage is created lazily on the first + /// resample() call, so add_source() and start() may be called in any order. + /// @param input_ring_buffer weak_ptr of a shared_ptr of the source ring buffer to transfer ownership + /// @return ESP_OK if successful, ESP_ERR_INVALID_STATE if the ring buffer is no longer alive + esp_err_t add_source(std::weak_ptr &input_ring_buffer); + + /// @brief Adds a sink ring buffer for resampled audio. Takes ownership of the ring buffer in a shared_ptr. + /// @param output_ring_buffer weak_ptr of a shared_ptr of the sink ring buffer to transfer ownership + /// @return ESP_OK if successsful, ESP_ERR_NO_MEM if the transfer buffer wasn't allocated + esp_err_t add_sink(std::weak_ptr &output_ring_buffer); + +#ifdef USE_SPEAKER + /// @brief Adds a sink speaker for decoded audio. + /// @param speaker pointer to speaker component + /// @return ESP_OK if successsful, ESP_ERR_NO_MEM if the transfer buffer wasn't allocated + esp_err_t add_sink(speaker::Speaker *speaker); +#endif + + /// @brief Sets up the class to resample. + /// @param input_stream_info The incoming sample rate, bits per sample, and number of channels + /// @param output_stream_info The desired outgoing sample rate, bits per sample, and number of channels + /// @param number_of_taps Number of taps per FIR filter + /// @param number_of_filters Number of FIR filters + /// @return ESP_OK if it is able to convert the incoming stream, + /// ESP_ERR_NO_MEM if the transfer buffers failed to allocate, + /// ESP_ERR_NOT_SUPPORTED if the stream can't be converted. + esp_err_t start(AudioStreamInfo &input_stream_info, AudioStreamInfo &output_stream_info, uint16_t number_of_taps, + uint16_t number_of_filters); + + /// @brief Resamples audio from the ring buffer source and writes to the sink. + /// @param stop_gracefully If true, it indicates the file decoder is finished. The resampler will resample all the + /// remaining audio and then finish. + /// @param ms_differential Pointer to a (int32_t) variable that will store the difference, in milliseconds, between + /// the duration of input audio used and the duration of output audio generated. + /// @return AudioResamplerState + AudioResamplerState resample(bool stop_gracefully, int32_t *ms_differential); + + /// @brief Pauses sending resampled audio to the sink. If paused, it will continue to process internal buffers. + /// @param pause_state If true, audio data is not sent to the sink. + void set_pause_output_state(bool pause_state) { this->pause_output_ = pause_state; } + + protected: + std::shared_ptr source_ring_buffer_; + std::unique_ptr audio_source_; + std::unique_ptr output_transfer_buffer_; + + size_t input_buffer_size_; + size_t output_buffer_size_; + + uint32_t accumulated_frames_used_{0}; + uint32_t accumulated_frames_generated_{0}; + + bool pause_output_{false}; + + AudioStreamInfo input_stream_info_; + AudioStreamInfo output_stream_info_; + + std::unique_ptr resampler_; +}; + +} // namespace esphome::audio + +#endif diff --git a/firmware/esphome/components/audio/audio_transfer_buffer.cpp b/firmware/esphome/components/audio/audio_transfer_buffer.cpp new file mode 100644 index 0000000..01fd4bb --- /dev/null +++ b/firmware/esphome/components/audio/audio_transfer_buffer.cpp @@ -0,0 +1,359 @@ +#include "audio_transfer_buffer.h" + +#ifdef USE_ESP32 + +#include + +#include "esphome/core/helpers.h" + +namespace esphome::audio { + +AudioTransferBuffer::~AudioTransferBuffer() { this->deallocate_buffer_(); }; + +std::unique_ptr AudioSinkTransferBuffer::create(size_t buffer_size) { + std::unique_ptr sink_buffer = make_unique(); + + if (!sink_buffer->allocate_buffer_(buffer_size)) { + return nullptr; + } + + return sink_buffer; +} + +std::unique_ptr AudioSourceTransferBuffer::create(size_t buffer_size) { + std::unique_ptr source_buffer = make_unique(); + + if (!source_buffer->allocate_buffer_(buffer_size)) { + return nullptr; + } + + return source_buffer; +} + +size_t AudioTransferBuffer::free() const { + if (this->buffer_size_ == 0) { + return 0; + } + return this->buffer_size_ - (this->buffer_length_ + (this->data_start_ - this->buffer_)); +} + +void AudioTransferBuffer::decrease_buffer_length(size_t bytes) { + this->buffer_length_ -= bytes; + if (this->buffer_length_ > 0) { + this->data_start_ += bytes; + } else { + // All the data in the buffer has been consumed, reset the start pointer + this->data_start_ = this->buffer_; + } +} + +void AudioTransferBuffer::increase_buffer_length(size_t bytes) { this->buffer_length_ += bytes; } + +void AudioTransferBuffer::clear_buffered_data() { + this->buffer_length_ = 0; + if (this->ring_buffer_ != nullptr) { + this->ring_buffer_->reset(); + } +} + +void AudioSinkTransferBuffer::clear_buffered_data() { + this->buffer_length_ = 0; + if (this->ring_buffer_ != nullptr) { + this->ring_buffer_->reset(); + } +#ifdef USE_SPEAKER + if (this->speaker_ != nullptr) { + this->speaker_->stop(); + } +#endif +} + +bool AudioTransferBuffer::has_buffered_data() const { + if (this->ring_buffer_ != nullptr) { + return ((this->ring_buffer_->available() > 0) || (this->available() > 0)); + } + return (this->available() > 0); +} + +bool AudioTransferBuffer::reallocate(size_t new_buffer_size) { + if (this->buffer_ == nullptr) { + return this->allocate_buffer_(new_buffer_size); + } + + if (new_buffer_size < this->buffer_length_) { + // New size is too small to hold existing data + return false; + } + + // Shift existing data to the start of the buffer so realloc preserves it + if ((this->buffer_length_ > 0) && (this->data_start_ != this->buffer_)) { + std::memmove(this->buffer_, this->data_start_, this->buffer_length_); + this->data_start_ = this->buffer_; + } + + RAMAllocator allocator; + uint8_t *new_buffer = allocator.reallocate(this->buffer_, new_buffer_size); + if (new_buffer == nullptr) { + // Reallocation failed, but the original buffer is still valid + return false; + } + + this->buffer_ = new_buffer; + this->data_start_ = this->buffer_; + this->buffer_size_ = new_buffer_size; + return true; +} + +bool AudioTransferBuffer::allocate_buffer_(size_t buffer_size) { + this->buffer_size_ = buffer_size; + + RAMAllocator allocator; + + this->buffer_ = allocator.allocate(this->buffer_size_); + if (this->buffer_ == nullptr) { + return false; + } + + this->data_start_ = this->buffer_; + this->buffer_length_ = 0; + + return true; +} + +void AudioTransferBuffer::deallocate_buffer_() { + if (this->buffer_ != nullptr) { + RAMAllocator allocator; + allocator.deallocate(this->buffer_, this->buffer_size_); + this->buffer_ = nullptr; + this->data_start_ = nullptr; + } + + this->buffer_size_ = 0; + this->buffer_length_ = 0; +} + +size_t AudioSourceTransferBuffer::transfer_data_from_source(TickType_t ticks_to_wait, bool pre_shift) { + if (pre_shift) { + // Shift data in buffer to start + if (this->buffer_length_ > 0) { + std::memmove(this->buffer_, this->data_start_, this->buffer_length_); + } + this->data_start_ = this->buffer_; + } + + size_t bytes_to_read = AudioTransferBuffer::free(); + size_t bytes_read = 0; + if (bytes_to_read > 0) { + if (this->ring_buffer_ != nullptr) { + bytes_read = this->ring_buffer_->read((void *) this->get_buffer_end(), bytes_to_read, ticks_to_wait); + } + + this->increase_buffer_length(bytes_read); + } + return bytes_read; +} + +size_t AudioSinkTransferBuffer::transfer_data_to_sink(TickType_t ticks_to_wait, bool post_shift) { + size_t bytes_written = 0; + if (this->available()) { +#ifdef USE_SPEAKER + if (this->speaker_ != nullptr) { + bytes_written = this->speaker_->play(this->data_start_, this->available(), ticks_to_wait); + } else +#endif + if (this->ring_buffer_ != nullptr) { + bytes_written = + this->ring_buffer_->write_without_replacement((void *) this->data_start_, this->available(), ticks_to_wait); + } else if (this->sink_callback_ != nullptr) { + bytes_written = this->sink_callback_->audio_sink_write(this->data_start_, this->available(), ticks_to_wait); + } + + this->decrease_buffer_length(bytes_written); + } + + if (post_shift) { + // Shift unwritten data to the start of the buffer + std::memmove(this->buffer_, this->data_start_, this->buffer_length_); + this->data_start_ = this->buffer_; + } + + return bytes_written; +} + +bool AudioSinkTransferBuffer::has_buffered_data() const { +#ifdef USE_SPEAKER + if (this->speaker_ != nullptr) { + return (this->speaker_->has_buffered_data() || (this->available() > 0)); + } +#endif + if (this->ring_buffer_ != nullptr) { + return ((this->ring_buffer_->available() > 0) || (this->available() > 0)); + } + return (this->available() > 0); +} + +size_t AudioSourceTransferBuffer::free() const { return AudioTransferBuffer::free(); } + +bool AudioSourceTransferBuffer::has_buffered_data() const { return AudioTransferBuffer::has_buffered_data(); } + +void ConstAudioSourceBuffer::set_data(const uint8_t *data, size_t length) { + this->data_start_ = data; + this->length_ = length; +} + +void ConstAudioSourceBuffer::consume(size_t bytes) { + bytes = std::min(bytes, this->length_); + this->length_ -= bytes; + this->data_start_ += bytes; +} + +std::unique_ptr RingBufferAudioSource::create( + std::shared_ptr ring_buffer, size_t max_fill_bytes, uint8_t alignment_bytes) { + if (ring_buffer == nullptr || max_fill_bytes == 0 || alignment_bytes == 0 || alignment_bytes > MAX_ALIGNMENT_BYTES) { + return nullptr; + } + return std::unique_ptr( + new RingBufferAudioSource(std::move(ring_buffer), max_fill_bytes, alignment_bytes)); +} + +RingBufferAudioSource::~RingBufferAudioSource() { + if (this->acquired_item_ != nullptr) { + this->ring_buffer_->receive_release(this->acquired_item_); + this->acquired_item_ = nullptr; + } +} + +void RingBufferAudioSource::release_item_() { + if (this->acquired_item_ == nullptr) { + return; + } + if (this->item_trailing_length_ > 0) { + // Copy the trailing sub-frame bytes into the splice buffer before returning the item; the next + // fill() will complete the frame from the head of the next chunk. + std::memcpy(this->splice_buffer_, this->item_trailing_ptr_, this->item_trailing_length_); + this->splice_length_ = this->item_trailing_length_; + this->item_trailing_ptr_ = nullptr; + this->item_trailing_length_ = 0; + } + this->ring_buffer_->receive_release(this->acquired_item_); + this->acquired_item_ = nullptr; +} + +void RingBufferAudioSource::consume(size_t bytes) { + bytes = std::min(bytes, this->current_available_); + this->current_data_ += bytes; + this->current_available_ -= bytes; + // Promotion of queued data is deferred to fill() so callers see new data as a fresh return value + // rather than appearing silently after consume(). When the held item has nothing left depending + // on it (no exposed bytes and no queued region), release it now so the ring buffer can be + // reclaimed by writers even if fill() is never called again. + if (this->current_available_ == 0 && this->queued_length_ == 0) { + this->release_item_(); + } +} + +void RingBufferAudioSource::clear_buffered_data() { + // Release the held item before reset() so the source no longer references memory the reset will reclaim. + if (this->acquired_item_ != nullptr) { + this->ring_buffer_->receive_release(this->acquired_item_); + this->acquired_item_ = nullptr; + } + this->current_data_ = nullptr; + this->current_available_ = 0; + this->queued_data_ = nullptr; + this->queued_length_ = 0; + this->item_trailing_ptr_ = nullptr; + this->item_trailing_length_ = 0; + this->splice_length_ = 0; + this->ring_buffer_->reset(); +} + +bool RingBufferAudioSource::has_buffered_data() const { + // splice_length_ is deliberately not considered here. It holds an incomplete frame whose completion + // bytes must still arrive through the ring buffer, which ring_buffer_->available() already reports. + // Counting it separately would strand a drain loop when a stream ends mid-frame and those completion + // bytes never come. + return (this->current_available_ > 0) || (this->queued_length_ > 0) || (this->ring_buffer_->available() > 0); +} + +size_t RingBufferAudioSource::fill(TickType_t ticks_to_wait, bool /*pre_shift*/) { + if (this->current_available_ > 0) { + // Caller has not finished consuming the current exposure + return 0; + } + + // If a queued region (the aligned remainder of the new chunk after a splice frame) is waiting, + // promote it to the exposed region and report its size as fresh data. + if (this->queued_length_ > 0) { + this->current_data_ = this->queued_data_; + this->current_available_ = this->queued_length_; + this->queued_data_ = nullptr; + this->queued_length_ = 0; + return this->current_available_; + } + + // Nothing exposed and nothing queued: release the previously held item (saving any sub-frame tail + // to splice_buffer_) and acquire a new chunk. + this->release_item_(); + + size_t chunk_length = 0; + void *item = this->ring_buffer_->receive_acquire(chunk_length, this->max_fill_bytes_, ticks_to_wait); + if (item == nullptr) { + return 0; + } + + uint8_t *chunk_data = static_cast(item); + bool exposing_splice_frame = false; + + // Complete any pending splice frame from the head of the new chunk. + if (this->splice_length_ > 0) { + const size_t needed = static_cast(this->alignment_bytes_) - this->splice_length_; + if (chunk_length < needed) { + // Not enough data to complete the spliced frame yet; absorb everything and wait for more. + std::memcpy(this->splice_buffer_ + this->splice_length_, chunk_data, chunk_length); + this->splice_length_ += chunk_length; + this->ring_buffer_->receive_release(item); + return 0; + } + std::memcpy(this->splice_buffer_ + this->splice_length_, chunk_data, needed); + chunk_data += needed; + chunk_length -= needed; + this->splice_length_ = 0; + exposing_splice_frame = true; + } + + this->acquired_item_ = item; + + // Split the remaining chunk into its aligned region and a (possibly zero) sub-frame trailing tail. + const size_t trailing = (this->alignment_bytes_ > 1) ? (chunk_length % this->alignment_bytes_) : 0; + const size_t aligned_bytes = chunk_length - trailing; + if (trailing > 0) { + this->item_trailing_ptr_ = chunk_data + aligned_bytes; + this->item_trailing_length_ = trailing; + } + + if (exposing_splice_frame) { + // Expose the spliced frame from splice_buffer_, queuing the chunk's aligned region for the next + // fill() call. + this->current_data_ = this->splice_buffer_; + this->current_available_ = this->alignment_bytes_; + this->queued_data_ = chunk_data; + this->queued_length_ = aligned_bytes; + return this->alignment_bytes_; + } + + if (aligned_bytes == 0) { + // The entire chunk is a sub-frame tail (only possible when alignment exceeds chunk size). Save it + // to the splice buffer and release the item so the next fill() can complete the frame. + this->release_item_(); + return 0; + } + + this->current_data_ = chunk_data; + this->current_available_ = aligned_bytes; + return aligned_bytes; +} + +} // namespace esphome::audio + +#endif diff --git a/firmware/esphome/components/audio/audio_transfer_buffer.h b/firmware/esphome/components/audio/audio_transfer_buffer.h new file mode 100644 index 0000000..074684f --- /dev/null +++ b/firmware/esphome/components/audio/audio_transfer_buffer.h @@ -0,0 +1,303 @@ +#pragma once + +#ifdef USE_ESP32 +#include "esphome/components/ring_buffer/ring_buffer.h" +#include "esphome/core/defines.h" + +#ifdef USE_SPEAKER +#include "esphome/components/speaker/speaker.h" +#endif + +#include "esp_err.h" + +#include + +namespace esphome::audio { + +/// @brief Abstract interface for writing decoded audio data to a sink. +class AudioSinkCallback { + public: + virtual size_t audio_sink_write(uint8_t *data, size_t length, TickType_t ticks_to_wait) = 0; +}; + +class AudioTransferBuffer { + /* + * @brief Class that facilitates tranferring data between a buffer and an audio source or sink. + * The transfer buffer is a typical C array that temporarily holds data for processing in other audio components. + * Both sink and source transfer buffers can use a ring buffer as the sink/source. + * - The ring buffer is stored in a shared_ptr, so destroying the transfer buffer object will release ownership. + */ + public: + /// @brief Destructor that deallocates the transfer buffer + ~AudioTransferBuffer(); + + /// @brief Returns a pointer to the start of the transfer buffer where available() bytes of existing data can be read + uint8_t *get_buffer_start() const { return this->data_start_; } + + /// @brief Returns a pointer to the end of the transfer buffer where free() bytes of new data can be written + uint8_t *get_buffer_end() const { return this->data_start_ + this->buffer_length_; } + + /// @brief Updates the internal state of the transfer buffer. This should be called after reading data + /// @param bytes The number of bytes consumed/read + void decrease_buffer_length(size_t bytes); + + /// @brief Updates the internal state of the transfer buffer. This should be called after writing data + /// @param bytes The number of bytes written + void increase_buffer_length(size_t bytes); + + /// @brief Returns the transfer buffer's currently available bytes to read + size_t available() const { return this->buffer_length_; } + + /// @brief Returns the transfer buffers allocated bytes + size_t capacity() const { return this->buffer_size_; } + + /// @brief Returns the transfer buffer's currrently free bytes available to write + size_t free() const; + + /// @brief Clears data in the transfer buffer and, if possible, the source/sink. + virtual void clear_buffered_data(); + + /// @brief Tests if there is any data in the tranfer buffer or the source/sink. + /// @return True if there is data, false otherwise. + virtual bool has_buffered_data() const; + + /// @brief Reallocates the transfer buffer, preserving any existing data. + /// @param new_buffer_size The new size in bytes. Must be at least as large as available(). + /// @return True if successful, false otherwise. On failure, the original buffer remains valid. + bool reallocate(size_t new_buffer_size); + + protected: + /// @brief Allocates the transfer buffer in external memory, if available. + /// @param buffer_size The number of bytes to allocate + /// @return True is successful, false otherwise. + bool allocate_buffer_(size_t buffer_size); + + /// @brief Deallocates the buffer and resets the class variables. + void deallocate_buffer_(); + + // A possible source or sink for the transfer buffer + std::shared_ptr ring_buffer_; + + uint8_t *buffer_{nullptr}; + uint8_t *data_start_{nullptr}; + + size_t buffer_size_{0}; + size_t buffer_length_{0}; +}; + +class AudioSinkTransferBuffer : public AudioTransferBuffer { + /* + * @brief A class that implements a transfer buffer for audio sinks. + * Supports writing processed data in the transfer buffer to a ring buffer or a speaker component. + */ + public: + /// @brief Creates a new sink transfer buffer. + /// @param buffer_size Size of the transfer buffer in bytes. + /// @return unique_ptr if successfully allocated, nullptr otherwise + static std::unique_ptr create(size_t buffer_size); + + /// @brief Writes any available data in the transfer buffer to the sink. + /// @param ticks_to_wait FreeRTOS ticks to block while waiting for the sink to have enough space + /// @param post_shift If true, all remaining data is moved to the start of the buffer after transferring to the sink. + /// Defaults to true. + /// @return Number of bytes written + size_t transfer_data_to_sink(TickType_t ticks_to_wait, bool post_shift = true); + + /// @brief Adds a ring buffer as the transfer buffer's sink. + /// @param ring_buffer weak_ptr to the allocated ring buffer + void set_sink(const std::weak_ptr &ring_buffer) { this->ring_buffer_ = ring_buffer.lock(); } + +#ifdef USE_SPEAKER + /// @brief Adds a speaker as the transfer buffer's sink. + /// @param speaker Pointer to the speaker component + void set_sink(speaker::Speaker *speaker) { this->speaker_ = speaker; } +#endif + + /// @brief Adds a callback as the transfer buffer's sink. + /// @param callback Pointer to the AudioSinkCallback implementation + void set_sink(AudioSinkCallback *callback) { this->sink_callback_ = callback; } + + void clear_buffered_data() override; + + bool has_buffered_data() const override; + + protected: +#ifdef USE_SPEAKER + speaker::Speaker *speaker_{nullptr}; +#endif + AudioSinkCallback *sink_callback_{nullptr}; +}; + +/// @brief Abstract interface for reading audio data from a buffer. +/// Provides a common read interface for both mutable transfer buffers and read-only const buffers. +class AudioReadableBuffer { + public: + virtual ~AudioReadableBuffer() = default; + + /// @brief Returns a pointer to the start of readable data + virtual const uint8_t *data() const = 0; + + /// @brief Returns the number of bytes available to read + virtual size_t available() const = 0; + + /// @brief Returns the number of free bytes available to write. Defaults to 0 for read-only buffers. + virtual size_t free() const { return 0; } + + /// @brief Advances past consumed data + /// @param bytes Number of bytes consumed + virtual void consume(size_t bytes) = 0; + + /// @brief Tests if there is any buffered data + virtual bool has_buffered_data() const = 0; + + /// @brief Refills the buffer from its source. No-op by default for read-only buffers. + /// @param ticks_to_wait FreeRTOS ticks to block while waiting for data + /// @param pre_shift If true, shifts existing data to the start of the buffer before reading + /// @return Number of bytes read + virtual size_t fill(TickType_t ticks_to_wait, bool pre_shift) { return 0; } + size_t fill(TickType_t ticks_to_wait) { return this->fill(ticks_to_wait, true); } +}; + +class AudioSourceTransferBuffer : public AudioTransferBuffer, public AudioReadableBuffer { + /* + * @brief A class that implements a transfer buffer for audio sources. + * Supports reading audio data from a ring buffer into the transfer buffer for processing. + * Implements AudioReadableBuffer for use by consumers that only need read access. + */ + public: + /// @brief Creates a new source transfer buffer. + /// @param buffer_size Size of the transfer buffer in bytes. + /// @return unique_ptr if successfully allocated, nullptr otherwise + static std::unique_ptr create(size_t buffer_size); + + /// @brief Reads any available data from the source into the transfer buffer. + /// @param ticks_to_wait FreeRTOS ticks to block while waiting for the source to have enough data + /// @param pre_shift If true, any unwritten data is moved to the start of the buffer before transferring from the + /// source. Defaults to true. + /// @return Number of bytes read + size_t transfer_data_from_source(TickType_t ticks_to_wait, bool pre_shift = true); + + /// @brief Adds a ring buffer as the transfer buffer's source. + /// @param ring_buffer weak_ptr to the allocated ring buffer + void set_source(const std::weak_ptr &ring_buffer) { + this->ring_buffer_ = ring_buffer.lock(); + }; + + // AudioReadableBuffer interface + const uint8_t *data() const override { return this->data_start_; } + size_t available() const override { return this->buffer_length_; } + size_t free() const override; + void consume(size_t bytes) override { this->decrease_buffer_length(bytes); } + bool has_buffered_data() const override; + size_t fill(TickType_t ticks_to_wait, bool pre_shift) override { + return this->transfer_data_from_source(ticks_to_wait, pre_shift); + } +}; + +/// @brief A lightweight read-only audio buffer for const data sources (e.g., flash memory). +/// Does not allocate memory or transfer data from external sources. +class ConstAudioSourceBuffer : public AudioReadableBuffer { + public: + /// @brief Sets the data pointer and length for the buffer + /// @param data Pointer to the const audio data + /// @param length Size of the data in bytes + void set_data(const uint8_t *data, size_t length); + + // AudioReadableBuffer interface + const uint8_t *data() const override { return this->data_start_; } + size_t available() const override { return this->length_; } + void consume(size_t bytes) override; + bool has_buffered_data() const override { return this->length_ > 0; } + + protected: + const uint8_t *data_start_{nullptr}; + size_t length_{0}; +}; + +/// @brief Zero-copy audio source that reads directly from a ring buffer's internal storage. +/// +/// Optionally enforces a minimum read alignment (e.g. one audio frame). When alignment_bytes > 1, the +/// source transparently stitches frames that straddle the ring buffer's wrap boundary by buffering the +/// trailing partial frame from one chunk and joining it with the head of the next chunk in a small +/// internal splice buffer, so callers always see frame-aligned data. +/// +/// Not thread-safe. The underlying ring_buffer::RingBuffer supports one producer and one consumer +/// running concurrently, but a given RingBufferAudioSource (its acquired item, splice buffer, and +/// queued region) must be used by only one thread, and that thread is the ring buffer's consumer. +class RingBufferAudioSource : public AudioReadableBuffer { + public: + /// Maximum supported alignment. Sized to cover 32-bit samples across up to 2 channels (8 bytes). + static constexpr size_t MAX_ALIGNMENT_BYTES = 8; + + /// @brief Creates a new ring-buffer-backed audio source after validating its parameters. + /// @param ring_buffer The ring buffer to read from. Must be non-null. + /// @param max_fill_bytes Soft cap on bytes acquired per fill() call. Must be > 0. + /// @param alignment_bytes Minimum exposed-region alignment in bytes (defaults to 1, i.e. byte-aligned). + /// Pass bytes_per_frame to make every exposed region a whole number of frames. Must be in + /// [1, MAX_ALIGNMENT_BYTES]. + /// @return unique_ptr if parameters are valid, nullptr otherwise + static std::unique_ptr create(std::shared_ptr ring_buffer, + size_t max_fill_bytes, uint8_t alignment_bytes = 1); + + ~RingBufferAudioSource() override; + + // AudioReadableBuffer interface + const uint8_t *data() const override { return this->current_data_; } + size_t available() const override { return this->current_available_; } + void consume(size_t bytes) override; + bool has_buffered_data() const override; + /// pre_shift is ignored: there is no intermediate transfer buffer to compact, so an unconsumed + /// exposure stays in place and fill() returns 0 until it is fully consumed. + size_t fill(TickType_t ticks_to_wait, bool pre_shift) override; + + /// @brief Discards all buffered audio: releases any held ring buffer item, clears the source's in-flight + /// state, and resets the underlying ring buffer. Must be invoked from the ring buffer's consumer thread. + void clear_buffered_data(); + + /// @brief Returns a mutable pointer to the currently exposed audio data. + /// The pointer may reference the ring buffer's internal storage or, when exposing a stitched frame + /// across a wrap boundary, an internal splice buffer. In either case mutations are safe but data + /// should be discarded after use, since the underlying storage will be reused on the next fill(). + /// Use only when the caller is the sole consumer of this source. + uint8_t *mutable_data() { return this->current_data_; } + + protected: + /// @brief Constructs a new ring-buffer-backed audio source. Use create() instead, which validates + /// arguments before construction. + explicit RingBufferAudioSource(std::shared_ptr ring_buffer, size_t max_fill_bytes, + uint8_t alignment_bytes) + : ring_buffer_(std::move(ring_buffer)), max_fill_bytes_(max_fill_bytes), alignment_bytes_(alignment_bytes) {} + + /// @brief Releases the currently held ring buffer item, first copying any trailing sub-frame bytes + /// into the splice buffer so they can be stitched with the next chunk. + void release_item_(); + + std::shared_ptr ring_buffer_; + size_t max_fill_bytes_; + + void *acquired_item_{nullptr}; + uint8_t *current_data_{nullptr}; + + // Sub-frame trailing bytes inside the held item that will be copied to splice_buffer_ on release. + uint8_t *item_trailing_ptr_{nullptr}; + + // After the currently-exposed splice frame is consumed, fill() will promote this region (the aligned + // remainder of the new chunk) to the exposed region. queued_length_ == 0 when nothing is queued. + uint8_t *queued_data_{nullptr}; + + // Splice buffer holds the start of a partial frame whose remainder lives at the head of the next + // chunk. While splice_length_ > 0, the buffer is incomplete and waiting for completion bytes. + uint8_t splice_buffer_[MAX_ALIGNMENT_BYTES]; + + size_t current_available_{0}; + size_t queued_length_{0}; + + // item_trailing_length_ and splice_length_ are bounded by MAX_ALIGNMENT_BYTES. + uint8_t alignment_bytes_; + uint8_t item_trailing_length_{0}; + uint8_t splice_length_{0}; +}; + +} // namespace esphome::audio + +#endif diff --git a/firmware/esphome/components/audio/upstream.json b/firmware/esphome/components/audio/upstream.json new file mode 100644 index 0000000..5f53f4a --- /dev/null +++ b/firmware/esphome/components/audio/upstream.json @@ -0,0 +1,26 @@ +{ + "version": "2026.9.0", + "source": "https://github.com/esphome/esphome/tree/2026.9.0/esphome/components", + "sha256": { + "audio/audio_reader.h": "68fccd34a2f6d44005dff8a1d7c543c6027d5d21d4c1fb217a379a94f6edd833", + "audio/audio_reader.cpp": "ff8d8ccc509f7d8445fe7aeab414d62be79fd5dfeee0ac7d7bf9b0d311f8cf77", + "audio/audio_resampler.h": "0ab374e42151ff89fdede0aec34c6263d44d9a0c24272f9a067eaf9fdcd3171c", + "audio/audio_decoder.cpp": "13e0b532f97188709a231436b5775f432c27150a33b2cfaad5f67035a3e603aa", + "audio/audio_resampler.cpp": "f53b8c4fddc582e693963be6ead994ace19899177076ba3bd7aca7bdb137789d", + "audio/audio.cpp": "f1733737ce5edd49342c7ad9cda5038fa6d39229aacdb61bed1087413fc1e5fb", + "audio/audio_transfer_buffer.cpp": "cadbfc7b569e8bb30076b7475fee54f0bc637532990c002b55ed72e03dd5b255", + "audio/__init__.py": "2827cfd296c207484e4818500203da25fd9163eb0ab65dae3a30c82e5bcd645a", + "audio/audio.h": "266932072fd7438e299323a2d72a07299e856d5f22d42d794118c35233c53fe9", + "audio/audio_transfer_buffer.h": "20522231f7b535c6c2368ea12d20b5267ed5ff7615ca0ef0aea9e2cece838a2a", + "audio/audio_decoder.h": "a2af44a56593f4b130732ac4232a77bf0cd7266bb7ca6d610992337a06cecffd", + "speaker/__init__.py": "a77ec9a37ddf8274b2b934fcbc1a9d358559a10998013043cd47c27f99d94cb0", + "speaker/automation.h": "5e736ee324ed88951dcfa4bf9af4b8a7298f7d64deda11c80e6f663a521e50b3", + "speaker/speaker.h": "fe852c5ebad730ac93799e0cf055213c9ce137484efed598a87e143abe81c826", + "speaker/media_player/speaker_media_player.cpp": "af0af73e2f3e774b77443b0ed6640de88ea3c029a958972ee85eb8faac332eba", + "speaker/media_player/audio_pipeline.h": "13abb2c86474c3c67d8308e5f8920621ace88a853d5bb0613f22ae7486cc5e5d", + "speaker/media_player/__init__.py": "267539a827e29db915365e0f0da840dc8b2854df0dc289eb0c841eeddcf708a6", + "speaker/media_player/automation.h": "e58215b9e360c30a7f07cf56a0e94a3e612e69290538b7b0a9193a7d03dd31e7", + "speaker/media_player/audio_pipeline.cpp": "be6401bd69577b98cc5fbca4b11669653775ed7b04dfb285aa0c80f58ad4985a", + "speaker/media_player/speaker_media_player.h": "3d4364971cc67e1ff8652adba35b8e49f089590127e0c1eb80fdbc552cd1ff60" + } +} diff --git a/firmware/esphome/components/openathan/openathan.h b/firmware/esphome/components/openathan/openathan.h index 55e761d..3ca4a68 100644 --- a/firmware/esphome/components/openathan/openathan.h +++ b/firmware/esphome/components/openathan/openathan.h @@ -31,6 +31,7 @@ class OpenAthan : public PollingComponent, public ::openathan::Clock { const char *light_application_status() const; ::openathan::LightMode current_light_mode() const { return light_mode_; } void set_playback(::openathan::Playback *playback) { playback_ = playback; } + ::openathan::Playback *playback() const { return playback_; } // Internal injection points, wired before setup by the developer test harness. void set_calculator(::openathan::DayCalculator *calculator) { calculator_source_ = calculator; } void set_state_store(::openathan::StateStore *store) { state_store_ = store; } diff --git a/firmware/esphome/components/openathan_audio/partition_audio.cpp b/firmware/esphome/components/openathan_audio/partition_audio.cpp index 20e5071..0db84f5 100644 --- a/firmware/esphome/components/openathan_audio/partition_audio.cpp +++ b/firmware/esphome/components/openathan_audio/partition_audio.cpp @@ -4,6 +4,10 @@ #include #include #include +#include "quran_policy.h" +#ifdef OPENATHAN_QURAN +#include "esphome/core/hal.h" +#endif namespace esphome::openathan_audio { static const char *const TAG = "openathan_audio"; @@ -77,7 +81,17 @@ bool PartitionAudio::play(::openathan::Track track) { ESP_LOGW(TAG, "Playback unavailable or invalid track"); return false; } +#ifdef OPENATHAN_QURAN + stop(); + awaiting_stop_ = true; + pending_file_ = &files_[id - 1]; +#else player_->play_file(&files_[id - 1], true, false); + ++epoch_; +#endif + source_ = ::openathan::PlaybackSource::ATHAN; + stream_state_ = ::openathan::StreamState::IDLE; + observed_playing_ = false; ESP_LOGI(TAG, "Playback queued: track=%u", id); return true; } @@ -87,8 +101,17 @@ bool PartitionAudio::playing() const { player_->state == media_player::MEDIA_PLAYER_STATE_ANNOUNCING); } void PartitionAudio::stop() { + ++epoch_; + source_ = ::openathan::PlaybackSource::NONE; + stream_state_ = ::openathan::StreamState::IDLE; + observed_playing_ = false; +#ifdef OPENATHAN_QURAN + awaiting_stop_ = false; + pending_file_ = nullptr; + pending_url_.clear(); +#endif if (!player_ || !player_->is_ready()) return; - // Stop both pipelines. Commands and the subsequent file request share a FIFO. + // Stop both pipelines before submitting any replacement recording. for (bool announcement : {false, true}) { auto call = player_->make_call(); call.set_command(media_player::MEDIA_PLAYER_COMMAND_STOP); @@ -96,9 +119,68 @@ void PartitionAudio::stop() { call.perform(); } } +bool PartitionAudio::start_stream(const std::string &url) { +#ifdef OPENATHAN_QURAN + if (!ready() || !openathan_device::quran_audio_url(url)) return false; + stop(); + awaiting_stop_ = true; + pending_url_ = url; + source_ = ::openathan::PlaybackSource::QURAN; + stream_state_ = ::openathan::StreamState::LOADING; + stream_started_ = millis(); + return true; +#else + return false; +#endif +} +void PartitionAudio::loop() { +#ifdef OPENATHAN_QURAN + if (source_ == ::openathan::PlaybackSource::NONE) return; + const bool streaming = source_ == ::openathan::PlaybackSource::QURAN; + if (!ready() || (streaming && !observed_playing_ && millis() - stream_started_ >= 30000)) { + stop(); + if (streaming) stream_state_ = ::openathan::StreamState::ERROR; + return; + } + if (awaiting_stop_) { + // Outgoing PLAYING/error observations belong to the previous session. + if (!player_->playback_stopped()) return; + if (pending_file_) player_->play_file(pending_file_, true, false); + else { + auto call = player_->make_call(); + call.set_media_url(pending_url_); + call.set_announcement(true); + call.perform(); + } + awaiting_stop_ = false; + pending_file_ = nullptr; + pending_url_.clear(); + return; // Observe the new pipeline only after submitting its start. + } + const auto state = player_->announcement_state(); + if (state == speaker::AudioPipelineState::ERROR_READING || state == speaker::AudioPipelineState::ERROR_DECODING) { + stop(); + if (streaming) stream_state_ = ::openathan::StreamState::ERROR; + } else if (state == speaker::AudioPipelineState::PLAYING) { + observed_playing_ = true; + if (streaming) + stream_state_ = ::openathan::StreamState::PLAYING; + } else if (observed_playing_ && state == speaker::AudioPipelineState::STOPPED) { + source_ = ::openathan::PlaybackSource::NONE; + stream_state_ = ::openathan::StreamState::IDLE; + } +#else + if (source_ == ::openathan::PlaybackSource::ATHAN) { + if (playing()) observed_playing_ = true; + else if (observed_playing_) source_ = ::openathan::PlaybackSource::NONE; + } +#endif +} bool PartitionAudio::start(::openathan::Track track) { if (!ready() || (track != ::openathan::Track::NORMAL && track != ::openathan::Track::FAJR)) return false; +#ifndef OPENATHAN_QURAN stop(); +#endif return play(track); } } // namespace esphome::openathan_audio diff --git a/firmware/esphome/components/openathan_audio/partition_audio.h b/firmware/esphome/components/openathan_audio/partition_audio.h index f58ee61..f2503e4 100644 --- a/firmware/esphome/components/openathan_audio/partition_audio.h +++ b/firmware/esphome/components/openathan_audio/partition_audio.h @@ -11,11 +11,23 @@ class PartitionAudio : public Component, public ::openathan::Playback { void set_player(speaker::SpeakerMediaPlayer *player) { player_ = player; } void set_partition(const std::string &partition) { partition_ = partition; } void setup() override; + void loop() override; bool play(::openathan::Track track); bool ready() const override { return ready_ && player_ && player_->is_ready(); } bool playing() const override; bool start(::openathan::Track track) override; void stop() override; + bool stream_supported() const override { +#ifdef OPENATHAN_QURAN + return true; +#else + return false; +#endif + } + uint32_t playback_epoch() const override { return epoch_; } + ::openathan::PlaybackSource source() const override { return source_; } + ::openathan::StreamState stream_state() const override { return stream_state_; } + bool start_stream(const std::string &url) override; std::optional volume_percent() const override; bool request_volume_percent(unsigned percent) override; protected: @@ -25,5 +37,17 @@ class PartitionAudio : public Component, public ::openathan::Playback { esp_partition_mmap_handle_t mapping_{}; std::array files_{}; bool ready_{false}; + uint32_t epoch_{}; + ::openathan::PlaybackSource source_{::openathan::PlaybackSource::NONE}; + ::openathan::StreamState stream_state_{::openathan::StreamState::IDLE}; + bool observed_playing_{}; +#ifdef OPENATHAN_QURAN + // Reserve ownership while the outgoing commands and pipelines drain. Only + // one replacement survives; Stop clears its recording and owned URL. + bool awaiting_stop_{}; + audio::AudioFile *pending_file_{}; + std::string pending_url_; + uint32_t stream_started_{}; +#endif }; } // namespace esphome::openathan_audio diff --git a/firmware/esphome/components/openathan_audio/quran_http.h b/firmware/esphome/components/openathan_audio/quran_http.h new file mode 100644 index 0000000..54092bb --- /dev/null +++ b/firmware/esphome/components/openathan_audio/quran_http.h @@ -0,0 +1,54 @@ +#pragma once +#include "quran_policy.h" +#include +#include +#include + +namespace esphome::openathan_device { +// Shared by metadata and the pinned audio reader. Redirect validation happens +// before initializing the next HTTP client, not after following the redirect. +class QuranHttp { + public: + esp_http_client_handle_t client{}; + std::string content_type; + ~QuranHttp() { if (client) esp_http_client_cleanup(client); } + QuranHttp() = default; + QuranHttp(const QuranHttp &) = delete; + QuranHttp &operator=(const QuranHttp &) = delete; + bool open(std::string url, bool audio = false, int buffer_size = 2048, int timeout_ms = 4000) { + for (unsigned redirects = 0; redirects <= 4; ++redirects) { + const bool metadata = url.size() <= 256 && url.rfind("https://www.mp3quran.net/api/v3/", 0) == 0; + if (audio ? !quran_audio_url(url) : !metadata) return false; + esp_http_client_config_t config{}; + config.url = url.c_str(); config.crt_bundle_attach = esp_crt_bundle_attach; + config.disable_auto_redirect = true; config.timeout_ms = timeout_ms; + config.buffer_size = buffer_size; config.buffer_size_tx = 1024; + config.event_handler = event_; config.user_data = this; + location_.clear(); content_type.clear(); client = esp_http_client_init(&config); + if (!client || esp_http_client_open(client, 0) != 0 || esp_http_client_fetch_headers(client) < 0) return false; + const int status = esp_http_client_get_status_code(client); + if (status == 200) return true; + if (status != 301 && status != 302 && status != 303 && status != 307 && status != 308) return false; + url = location_; esp_http_client_cleanup(client); client = nullptr; + } + return false; + } + esp_http_client_handle_t release() { + // IDF may deliver events while reading; revoke the soon-to-expire callback context. + esp_http_client_set_user_data(client, nullptr); + auto result = client; client = nullptr; return result; + } + private: + static esp_err_t event_(esp_http_client_event_t *event) { + if (!event->user_data || event->event_id != HTTP_EVENT_ON_HEADER) return 0; + auto *self = static_cast(event->user_data); + if (strcasecmp(event->header_key, "Location") == 0) { + self->location_ = strnlen(event->header_value, 513) <= 512 ? event->header_value : ""; + } else if (strcasecmp(event->header_key, "Content-Type") == 0) { + self->content_type = strnlen(event->header_value, 129) <= 128 ? event->header_value : ""; + } + return 0; + } + std::string location_; +}; +} diff --git a/firmware/esphome/components/openathan_audio/quran_policy.h b/firmware/esphome/components/openathan_audio/quran_policy.h new file mode 100644 index 0000000..9153bc3 --- /dev/null +++ b/firmware/esphome/components/openathan_audio/quran_policy.h @@ -0,0 +1,71 @@ +#pragma once +#include +#include + +namespace esphome::openathan_device { +// Explicit provider hosts, no credentials, ports, queries or fragments. +inline bool quran_audio_url(const std::string &url) { + if (url.size() > 512) return false; + const auto start = url.find('/', 8); + if (url.rfind("https://", 0) != 0 || start == std::string::npos) return false; + const auto host = url.substr(8, start - 8); + bool allowed = host == "cdn.mp3quran.net"; + if (host.rfind("server", 0) == 0 && host.size() > 6 + 13 && + host.compare(host.size() - 13, 13, ".mp3quran.net") == 0) { + const auto number = host.substr(6, host.size() - 6 - 13); + allowed = !number.empty() && number.size() <= 3; + for (char c : number) allowed = allowed && c >= '0' && c <= '9'; + } + if (!allowed) return false; + for (size_t i = start; i < url.size(); ++i) { + const unsigned char c = url[i]; + if (!(std::isalnum(c) || c == '/' || c == '-' || c == '_' || c == '.')) return false; + } + return url.find("..", start) == std::string::npos; +} +// Extract top-level array objects incrementally; bounds apply before JSON parsing. +class QuranObjectStream { + public: + explicit QuranObjectStream(std::string key) : prefix_("{\"" + key + "\":[") {} + bool feed(char c, std::string &object) { + object.clear(); + if (++bytes_ > 1048576 || failed_) { failed_ = true; return false; } + if (prefix_pos_ < prefix_.size()) { + if (std::isspace(static_cast(c)) && + (prefix_pos_ <= 1 || prefix_pos_ >= prefix_.size() - 2)) return true; + if (c != prefix_[prefix_pos_++]) return fail_(); + return true; + } + if (depth_) { + if (item_.size() >= 16384) return fail_(); + item_ += c; + if (quoted_) { + if (escaped_) escaped_ = false; + else if (c == '\\') escaped_ = true; + else if (c == '"') quoted_ = false; + } else if (c == '"') quoted_ = true; + else if (c == '{') ++depth_; + else if (c == '}' && --depth_ == 0) { object.swap(item_); separator_ = true; } + return true; + } + if (std::isspace(static_cast(c))) return true; + if (finished_) return fail_(); + if (closed_) { if (c != '}') return fail_(); finished_ = true; return true; } + if (separator_) { + if (c == ',') { separator_ = false; comma_ = true; return true; } + if (c == ']') { closed_ = true; return true; } + return fail_(); + } + if (c == ']' && !comma_) { closed_ = true; return true; } + if (c != '{') return fail_(); + comma_ = false; depth_ = 1; item_ = "{"; return true; + } + bool complete() const { return finished_ && !failed_; } + private: + bool fail_() { failed_ = true; return false; } + std::string prefix_, item_; + size_t prefix_pos_{}, bytes_{}; + unsigned depth_{}; + bool quoted_{}, escaped_{}, separator_{}, comma_{}, closed_{}, finished_{}, failed_{}; +}; +} diff --git a/firmware/esphome/components/openathan_device/__init__.py b/firmware/esphome/components/openathan_device/__init__.py index cd59056..c4c517d 100644 --- a/firmware/esphome/components/openathan_device/__init__.py +++ b/firmware/esphome/components/openathan_device/__init__.py @@ -61,6 +61,7 @@ def rule(value): async def to_code(config): + cg.add_define("OPENATHAN_QURAN") root = Path(__file__).resolve().parents[4] release = json.loads((root / "release/firmware.json").read_text()) version = release["version"] diff --git a/firmware/esphome/components/openathan_device/device.cpp b/firmware/esphome/components/openathan_device/device.cpp index 5866932..e671205 100644 --- a/firmware/esphome/components/openathan_device/device.cpp +++ b/firmware/esphome/components/openathan_device/device.cpp @@ -68,6 +68,8 @@ const char* http_status(int code) { void Device::setup() { api_ = std::make_unique(athan_, zones_, zone_count_); api_->set_upgrade(&upgrade_); + quran_.begin(athan_, random_nonce()); + api_->set_quran(&quran_); hostname_ = std::string(App.get_name().c_str()) + ".local"; realm_ = "OpenAthan-" + get_mac_address(); wifi_record_ = store_.load_wifi(saved_wifi_); @@ -284,6 +286,7 @@ void Device::on_wifi_scan_results(const wifi::wifi_scan_vector_tis_connected()); + quran_.loop(wifi::global_wifi_component->is_connected(), maintenance_ || upgrade_.streaming_blocked()); const auto now = now_ms(); if (wifi_attempt_.active()) { auto* wifi = wifi::global_wifi_component; diff --git a/firmware/esphome/components/openathan_device/device.h b/firmware/esphome/components/openathan_device/device.h index 1e05dfa..9ed9587 100644 --- a/firmware/esphome/components/openathan_device/device.h +++ b/firmware/esphome/components/openathan_device/device.h @@ -16,6 +16,7 @@ #include "protocol.h" #include "wifi_attempt.h" #include "upgrade.h" +#include "quran.h" namespace esphome::openathan_device { struct Asset { @@ -87,6 +88,7 @@ class Device : public Component, public wifi::WiFiScanResultsListener, public wi Asset assets_[3]; std::unique_ptr api_; Upgrade upgrade_; + Quran quran_; httpd_handle_t server_{}; std::mutex queue_mutex_; std::deque> requests_; diff --git a/firmware/esphome/components/openathan_device/local_api.cpp b/firmware/esphome/components/openathan_device/local_api.cpp index 5634177..35db21f 100644 --- a/firmware/esphome/components/openathan_device/local_api.cpp +++ b/firmware/esphome/components/openathan_device/local_api.cpp @@ -99,6 +99,10 @@ void LocalApi::snapshot_(JsonObject root) { root["hostname"] = hostname_; const auto status = athan_->status(); root["playing"] = status.playing; + const auto source = athan_->playback() ? athan_->playback()->source() : ::openathan::PlaybackSource::NONE; + root["playback_source"] = source == ::openathan::PlaybackSource::QURAN ? "quran" : + source == ::openathan::PlaybackSource::ATHAN ? "athan" : "none"; + if (quran_) quran_->snapshot(root["quran"].to()); if (status.next) { auto next = root["next"].to(); event_json(next, *status.next); @@ -131,6 +135,8 @@ void LocalApi::handle(ApiExchange& request) { snapshot_(root); else if (request.method == "GET" && request.uri == "/api/firmware" && upgrade_) upgrade_->snapshot(root); + else if (request.method == "GET" && request.uri == "/api/quran/catalog" && quran_) + quran_->catalog_snapshot(root); else if (request.method == "GET" && request.uri == "/api/time-format") time_format_(root); else if (request.method == "GET" && request.uri == "/api/display") @@ -150,7 +156,13 @@ void LocalApi::handle(ApiExchange& request) { auto payload = input.as(); const bool settings_action = request.uri == "/api/settings" || request.uri == "/api/preview" || request.uri == "/api/activate"; - if (request.uri.rfind("/api/firmware/", 0) == 0 && upgrade_) { + if ((request.uri == "/api/quran/catalog" || request.uri == "/api/quran/play") && quran_) { + std::string message; + const int code = quran_->action(request.uri == "/api/quran/play" ? "play" : "catalog", payload, message); + if (code != 200) { error(request, code, message.c_str()); return; } + if (request.uri == "/api/quran/catalog") quran_->catalog_snapshot(root); + else snapshot_(root); + } else if (request.uri.rfind("/api/firmware/", 0) == 0 && upgrade_) { std::string message; const int code = upgrade_->action(request.uri.substr(14), payload, message); if (code != 200) { error(request, code, message.c_str()); return; } diff --git a/firmware/esphome/components/openathan_device/local_api.h b/firmware/esphome/components/openathan_device/local_api.h index b36cdc2..5e51a22 100644 --- a/firmware/esphome/components/openathan_device/local_api.h +++ b/firmware/esphome/components/openathan_device/local_api.h @@ -1,6 +1,7 @@ #pragma once #include "../openathan/openathan.h" #include "upgrade_api.h" +#include "quran_api.h" namespace esphome::openathan_device { struct ZoneEntry { @@ -24,6 +25,7 @@ class LocalApi { } void handle(ApiExchange& request); void set_upgrade(UpgradeApi *upgrade) { upgrade_ = upgrade; } + void set_quran(QuranApi *quran) { quran_ = quran; } private: void time_format_(JsonObject root); @@ -38,5 +40,6 @@ class LocalApi { std::string hostname_; bool wifi_connected_{}; UpgradeApi *upgrade_{}; + QuranApi *quran_{}; }; } // namespace esphome::openathan_device diff --git a/firmware/esphome/components/openathan_device/quran.cpp b/firmware/esphome/components/openathan_device/quran.cpp new file mode 100644 index 0000000..83899fa --- /dev/null +++ b/firmware/esphome/components/openathan_device/quran.cpp @@ -0,0 +1,240 @@ +#include "quran.h" +#include "../openathan_audio/quran_policy.h" +#include "../openathan_audio/quran_http.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace esphome::openathan_device { +namespace { +uint64_t now_ms() { return esp_timer_get_time() / 1000; } +bool objects(const std::string &url, const std::string &key, std::atomic &cancel, + const std::function &consume) { + const auto deadline = now_ms() + 30000; + QuranHttp reader; + if (cancel || !reader.open(url)) return false; + QuranObjectStream parser(key); + std::array buffer{}; + std::string object; + while (!cancel && now_ms() < deadline) { + const auto size = esp_http_client_read(reader.client, buffer.data(), buffer.size()); + if (size < 0) return false; + if (size == 0) return esp_http_client_is_complete_data_received(reader.client) && parser.complete(); + for (int i = 0; i < size; ++i) { + if (!parser.feed(buffer[i], object)) return false; + if (!object.empty()) { + JsonDocument entry; + if (deserializeJson(entry, object) || !entry.is() || !consume(entry.as())) return false; + } + } + } + return false; +} +bool name(JsonVariantConst value) { + return value.is() && !value.as().empty() && value.as().size() <= 256; +} +bool surahs(const std::string &text, JsonArray target) { + if (text.empty() || text.size() > 512) return false; + std::array seen{}; + unsigned n = 0; bool digits = false; + for (size_t i = 0; i <= text.size(); ++i) { + const char c = i == text.size() ? ',' : text[i]; + if (c >= '0' && c <= '9') { n = n * 10 + c - '0'; digits = true; if (n > 114) return false; } + else if (c == ',') { + if (!digits || !n || seen[n]) return false; + seen[n] = true; target.add(n); n = 0; digits = false; + } else return false; + } + return true; +} +} +void Quran::fetch_(Job &job) { + JsonDocument output; + auto root = output.to(); + bool ok = true; + if (job.kind == "reciters") { + auto list = root["reciters"].to(); + unsigned count = 0, scanned = 0; + ok = objects("https://www.mp3quran.net/api/v3/reciters?language=eng", "reciters", job.cancel, + [&](JsonObjectConst entry) { + if (++scanned > 1024 || !entry["id"].is() || !entry["id"].as() || !name(entry["name"])) return false; + auto value = entry["name"].as(); + for (auto &c : value) c = std::tolower(static_cast(c)); + if (value.find(job.query) == std::string::npos) return true; + ++count; + if (count > job.offset && count <= job.offset + 20) { + auto row = list.add(); row["id"] = entry["id"]; row["name"] = entry["name"]; + } + return true; + }); + root["total"] = count; root["offset"] = job.offset; + if (job.offset + 20 < count) root["next_offset"] = job.offset + 20; + else root["next_offset"] = nullptr; + auto chapters = root["surahs"].to(); + std::array seen{}; + if (ok) ok = objects("https://www.mp3quran.net/api/v3/suwar?language=eng", "suwar", job.cancel, + [&](JsonObjectConst entry) { + const auto id = entry["id"].as(); + if (!entry["id"].is() || !id || id > 114 || seen[id] || !name(entry["name"])) return false; + seen[id] = true; + auto row = chapters.add(); row["id"] = id; row["name"] = entry["name"]; return true; + }); + ok = ok && chapters.size() == 114; + } else { + bool found = false; + auto editions = root["editions"].to(); + ok = objects("https://www.mp3quran.net/api/v3/reciters?language=eng&reciter=" + std::to_string(job.reciter), + "reciters", job.cancel, [&](JsonObjectConst entry) { + if (found || !entry["id"].is() || entry["id"].as() != job.reciter || + !entry["moshaf"].is() || entry["moshaf"].size() > 32) return false; + found = true; + for (auto edition : entry["moshaf"].as()) { + if (!edition["id"].is() || !edition["id"].as() || !name(edition["name"]) || + !edition["surah_list"].is() || !edition["server"].is()) return false; + JsonDocument numbers; + auto available = numbers.to(); + if (!surahs(edition["surah_list"].as(), available)) return false; + if (job.kind == "play") { + if (edition["id"].as() != job.edition) continue; + bool listed = false; + for (auto id : available) if (id.as() == job.surah) listed = true; + if (!listed || !job.url.empty()) return false; + const auto server = edition["server"].as(); + if (!quran_audio_url(server) || server.back() != '/') return false; + char file[8]; std::snprintf(file, sizeof(file), "%03u.mp3", job.surah); + job.url = server + file; + } else { + auto row = editions.add(); row["id"] = edition["id"]; row["name"] = edition["name"]; + row["surahs"].set(available); + } + } + return true; + }); + ok = ok && found && (job.kind != "play" || !job.url.empty()); + } + if (!ok) { job.error = "MP3Quran is unavailable or returned an unsupported recording. Try again."; return; } + if (measureJson(output) > 32768) { job.error = "MP3Quran catalog exceeds the device limit"; return; } + serializeJson(output, job.response); +} +void Quran::worker_(void *context) { + std::unique_ptr>> work( + static_cast> *>(context)); + work->first->fetch_(*work->second); + work->second->done = true; + work.reset(); + vTaskDelete(nullptr); +} +std::string Quran::playback_token_() const { + const auto *playback = athan_ ? athan_->playback() : nullptr; + return boot_id_ + ":" + std::to_string(playback ? playback->playback_epoch() : 0); +} +int Quran::action(const std::string &action, JsonObjectConst input, std::string &error) { + auto *playback = athan_ ? athan_->playback() : nullptr; + if (!playback || !playback->stream_supported()) { error = "Quran streaming is unavailable"; return 503; } + if (!connected_ || !athan_->read().valid) { error = "Connect Wi-Fi and wait for the clock before streaming"; return 503; } + if (!athan_->activated() || !athan_->upgrade_health() || !athan_->settings_service() || !athan_->settings_service()->healthy()) { + error = "Finish setup and resolve device faults before streaming"; return 409; + } + if (blocked_ || job_ || playback->source() == ::openathan::PlaybackSource::ATHAN) { + error = "Wait for Athan, the current request or firmware maintenance to finish"; return 409; + } + std::shared_ptr job; + if (action == "play") { + if ((input.size() != 3 && input.size() != 4) || !input["reciter"].is() || !input["edition"].is() || + !input["surah"].is() || !input["reciter"].as() || !input["edition"].as() || + !input["surah"].as() || input["surah"].as() > 114) { error = "Select a reciter, edition and surah"; return 400; } + if (input["expected_playback_token"].isUnbound()) { + if (input.size() != 3) { error = "Invalid Play request"; return 400; } + error = "Reload the page and check playback before pressing Play"; return 409; + } + if (input.size() != 4 || !input["expected_playback_token"].is() || + input["expected_playback_token"].as().size() > 59) { + error = "Invalid playback token"; return 400; + } + // Main-loop admission and Stop share the adapter epoch. This also rejects + // requests that arrive after physical Stop, Athan or a device restart. + if (input["expected_playback_token"].as() != playback_token_()) { + error = "Playback changed; reload playback status before pressing Play"; return 409; + } + job = std::make_shared(); + job->kind = "play"; job->reciter = input["reciter"]; job->edition = input["edition"]; job->surah = input["surah"]; + } else { + const auto kind = input["kind"].as(); + if ((kind == "reciters" ? input.size() != 3 : input.size() != 2) || (kind != "reciters" && kind != "editions") || + (kind == "reciters" && (!input["query"].is() || input["query"].as().size() > 128)) || + (kind == "reciters" && (!input["offset"].is() || input["offset"].as() > 1024)) || + (kind == "editions" && (!input["reciter"].is() || !input["reciter"].as()))) { + error = "Invalid catalog request"; return 400; + } + job = std::make_shared(); + job->kind = kind; job->offset = input["offset"] | 0U; job->reciter = input["reciter"] | 0U; + job->query = input["query"] | std::string{}; + for (auto &c : job->query) c = std::tolower(static_cast(c)); + } + auto *context = new (std::nothrow) std::pair>(this, job); + if (!context) { error = "Not enough memory to stream"; return 503; } + // Only a play request replaces Quran; catalog browsing does not alter audio. + if (action == "play") athan_->stop(); + job->epoch = playback->playback_epoch(); job->request = ++request_; + job_ = job; + if (xTaskCreate(worker_, "oa_quran", 8192, context, 3, nullptr) != pdPASS) { + delete context; job_.reset(); error = "Not enough memory to stream"; return 503; + } + if (action == "play") { + resolving_ = true; playback_error_.clear(); reciter_ = job->reciter; edition_ = job->edition; surah_ = job->surah; + } else { + catalog_state_ = "loading"; catalog_request_ = job->request; catalog_response_.clear(); catalog_error_.clear(); + } + return 200; +} +void Quran::loop(bool connected, bool blocked) { + connected_ = connected; blocked_ = blocked; + auto *playback = athan_ ? athan_->playback() : nullptr; + if (!playback) return; + if (job_ && (!connected || blocked || job_->epoch != playback->playback_epoch())) job_->cancel = true; + if ((!connected || !playback->ready()) && playback->source() == ::openathan::PlaybackSource::QURAN) { + athan_->stop(); playback_error_ = !connected ? "Wi-Fi disconnected. Press Play after reconnecting." : "The speaker is unavailable. Check the device before retrying."; + } + if (blocked && (resolving_ || playback->source() == ::openathan::PlaybackSource::QURAN)) athan_->stop(); + if (job_ && job_->cancel && job_->kind == "play") resolving_ = false; + if (!job_ || !job_->done) return; + auto job = std::move(job_); + if (job->kind == "play") { + resolving_ = false; + if (!job->cancel) { + if (!job->error.empty()) playback_error_ = job->error; + else if (!playback->start_stream(job->url)) playback_error_ = "The speaker could not start Quran playback"; + } + } else { + catalog_state_ = !job->cancel && job->error.empty() ? "ready" : "error"; + catalog_error_ = job->cancel ? "Catalog request interrupted. Try again." : job->error; + catalog_response_ = job->cancel ? "" : std::move(job->response); + } +} +void Quran::shutdown() { if (job_) job_->cancel = true; resolving_ = false; } +void Quran::snapshot(JsonObject root) { + auto *playback = athan_ ? athan_->playback() : nullptr; + root["supported"] = playback && playback->stream_supported(); + root["playback_token"] = playback_token_(); + const auto state = playback ? playback->stream_state() : ::openathan::StreamState::IDLE; + const bool pending = resolving_ && job_ && playback && job_->epoch == playback->playback_epoch(); + root["state"] = pending ? "loading" : !playback_error_.empty() || state == ::openathan::StreamState::ERROR ? "error" : + state == ::openathan::StreamState::PLAYING ? "playing" : state == ::openathan::StreamState::LOADING ? "loading" : "idle"; + root["error"] = state == ::openathan::StreamState::ERROR ? "The recording could not be played. Try another recording or retry." : playback_error_; + root["reciter"] = reciter_; root["edition"] = edition_; root["surah"] = surah_; +} +void Quran::catalog_snapshot(JsonObject root) { + root["state"] = catalog_state_; root["request"] = catalog_request_; root["error"] = catalog_error_; + if (!catalog_response_.empty()) { + JsonDocument document; + if (!deserializeJson(document, catalog_response_)) root["data"].set(document.as()); + } +} +} diff --git a/firmware/esphome/components/openathan_device/quran.h b/firmware/esphome/components/openathan_device/quran.h new file mode 100644 index 0000000..968575a --- /dev/null +++ b/firmware/esphome/components/openathan_device/quran.h @@ -0,0 +1,34 @@ +#pragma once +#include "quran_api.h" +#include "esphome/components/openathan/openathan.h" +#include +#include + +namespace esphome::openathan_device { +class Quran : public QuranApi { + public: + void begin(openathan_component::OpenAthan *athan, const std::string &boot_id) { athan_ = athan; boot_id_ = boot_id; } + void loop(bool connected, bool blocked); + void shutdown(); + void snapshot(JsonObject root) override; + void catalog_snapshot(JsonObject root) override; + int action(const std::string &action, JsonObjectConst input, std::string &error) override; + private: + struct Job { + std::string kind, response, url, error, query; + unsigned offset{}, reciter{}, edition{}, surah{}; + uint32_t epoch{}, request{}; + std::atomic done{}, cancel{}; + }; + static void worker_(void *context); + void fetch_(Job &job); + std::string playback_token_() const; + openathan_component::OpenAthan *athan_{}; + std::string boot_id_; + std::shared_ptr job_; + std::string catalog_state_{"idle"}, catalog_response_, catalog_error_, playback_error_; + uint32_t request_{}, catalog_request_{}; + unsigned reciter_{}, edition_{}, surah_{}; + bool connected_{}, blocked_{}, resolving_{}; +}; +} diff --git a/firmware/esphome/components/openathan_device/quran_api.h b/firmware/esphome/components/openathan_device/quran_api.h new file mode 100644 index 0000000..675ea15 --- /dev/null +++ b/firmware/esphome/components/openathan_device/quran_api.h @@ -0,0 +1,12 @@ +#pragma once +#include "esphome/components/json/json_util.h" +#include +namespace esphome::openathan_device { +class QuranApi { + public: + virtual ~QuranApi() = default; + virtual void snapshot(JsonObject root) = 0; + virtual void catalog_snapshot(JsonObject root) = 0; + virtual int action(const std::string &action, JsonObjectConst input, std::string &error) = 0; +}; +} diff --git a/firmware/esphome/components/openathan_device/shutdown.cpp b/firmware/esphome/components/openathan_device/shutdown.cpp index e89a0f7..e81e1d1 100644 --- a/firmware/esphome/components/openathan_device/shutdown.cpp +++ b/firmware/esphome/components/openathan_device/shutdown.cpp @@ -3,6 +3,7 @@ namespace esphome::openathan_device { void Device::on_shutdown() { upgrade_.shutdown(); + quran_.shutdown(); // Shutdown runs on the main loop. httpd_stop can wait for a handler that // is itself waiting for this loop, exhausting the watchdog deadline. // Restart/power-down terminates the server tasks and sockets; keep this diff --git a/firmware/esphome/components/openathan_device/upgrade.h b/firmware/esphome/components/openathan_device/upgrade.h index 83efe0d..e6060ac 100644 --- a/firmware/esphome/components/openathan_device/upgrade.h +++ b/firmware/esphome/components/openathan_device/upgrade.h @@ -42,6 +42,7 @@ class Upgrade : public UpgradeApi { uint8_t usb_command(uint8_t command, const std::vector &fields, std::vector &reply); bool usb_chunk(const ::openathan::usb_upgrade::Chunk &chunk); bool usb_busy() const { return usb_token_ || usb_request_; } + bool streaming_blocked() const { return usb_busy() || active_ || state_ == "queued" || state_ == "installing"; } private: static void worker_(void *argument); void run_(); diff --git a/firmware/esphome/components/openathan_display/presenter.h b/firmware/esphome/components/openathan_display/presenter.h index 058604f..22b77ea 100644 --- a/firmware/esphome/components/openathan_display/presenter.h +++ b/firmware/esphome/components/openathan_display/presenter.h @@ -16,6 +16,7 @@ struct Inputs { bool round{}; std::optional visual_next; int64_t utc{}; + bool quran{}; }; struct Frame { std::array clock{}; @@ -67,7 +68,7 @@ inline Frame present(const Inputs &in) { } else if (!in.setup_complete) { text(f.heading, "Setup"); text(f.main, "Needed"); text(f.detail, "Use phone setup"); } else if (in.status.playing) { - text(f.heading, "Athan"); text(f.main, "Playing"); text(f.detail, in.stop_button ? "Button to stop" : "Use phone"); + text(f.heading, in.quran ? "Quran" : "Athan"); text(f.main, "Playing"); text(f.detail, in.stop_button ? "Button to stop" : "Use phone"); } else if (!in.clock_valid) { text(f.heading, "Time"); text(f.main, "Waiting"); text(f.detail, in.wifi_connected ? "Syncing clock" : "Connect Wi-Fi"); } else if (!in.prayers_enabled && !in.round) { diff --git a/firmware/esphome/components/openathan_display/status_display.cpp b/firmware/esphome/components/openathan_display/status_display.cpp index b240ec2..a567a0d 100644 --- a/firmware/esphome/components/openathan_display/status_display.cpp +++ b/firmware/esphome/components/openathan_display/status_display.cpp @@ -41,7 +41,8 @@ void StatusDisplay::update() { std::strcmp(athan_->setup_state(), "storage_fault") == 0 || (service && !service->healthy()), clock.valid, wifi::global_wifi_component && wifi::global_wifi_component->is_connected(), enabled, hhmm(local), hhmm(next), athan_->time_format_preferences().hours(), stop_button_, - round_, next_event, clock.utc}; + round_, next_event, clock.utc, + athan_->playback() && athan_->playback()->source() == ::openathan::PlaybackSource::QURAN}; if (cache_.accept(::openathan::screen::present(input))) display_->update(); } } diff --git a/firmware/esphome/components/speaker/LICENSE b/firmware/esphome/components/speaker/LICENSE new file mode 100644 index 0000000..215460a --- /dev/null +++ b/firmware/esphome/components/speaker/LICENSE @@ -0,0 +1,709 @@ +# ESPHome License + +Copyright (c) 2019 ESPHome + +The ESPHome License is made up of two base licenses: MIT and the GNU GENERAL PUBLIC LICENSE. +The C++/runtime codebase of the ESPHome project (file extensions .c, .cpp, .h, .hpp, .tcc, .ino) are +published under the GPLv3 license. The python codebase and all other parts of this codebase are +published under the MIT license. + +Both MIT and GPLv3 licenses are attached to this document. + +## MIT License + +Copyright (c) 2019 ESPHome + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +## GPLv3 License + + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. diff --git a/firmware/esphome/components/speaker/__init__.py b/firmware/esphome/components/speaker/__init__.py new file mode 100644 index 0000000..98b5abe --- /dev/null +++ b/firmware/esphome/components/speaker/__init__.py @@ -0,0 +1,152 @@ +from esphome import automation +import esphome.codegen as cg +from esphome.components import audio, audio_dac +import esphome.config_validation as cv +from esphome.const import CONF_AUDIO_DAC, CONF_DATA, CONF_ID, CONF_VOLUME +from esphome.core import CORE, ID +from esphome.coroutine import CoroPriority, coroutine_with_priority + +AUTO_LOAD = ["audio"] +CODEOWNERS = ["@jesserockz", "@kahrendt"] + +IS_PLATFORM_COMPONENT = True + +speaker_ns = cg.esphome_ns.namespace("speaker") + +Speaker = speaker_ns.class_("Speaker") + +PlayAction = speaker_ns.class_( + "PlayAction", automation.Action, cg.Parented.template(Speaker) +) +StopAction = speaker_ns.class_( + "StopAction", automation.Action, cg.Parented.template(Speaker) +) +FinishAction = speaker_ns.class_( + "FinishAction", automation.Action, cg.Parented.template(Speaker) +) +VolumeSetAction = speaker_ns.class_( + "VolumeSetAction", automation.Action, cg.Parented.template(Speaker) +) +MuteOnAction = speaker_ns.class_( + "MuteOnAction", automation.Action, cg.Parented.template(Speaker) +) +MuteOffAction = speaker_ns.class_( + "MuteOffAction", automation.Action, cg.Parented.template(Speaker) +) + + +IsPlayingCondition = speaker_ns.class_("IsPlayingCondition", automation.Condition) +IsStoppedCondition = speaker_ns.class_("IsStoppedCondition", automation.Condition) + + +async def setup_speaker_core_(var, config): + if audio_dac_config := config.get(CONF_AUDIO_DAC): + aud_dac = await cg.get_variable(audio_dac_config) + cg.add(var.set_audio_dac(aud_dac)) + + +async def register_speaker(var, config): + if not CORE.has_id(config[CONF_ID]): + var = cg.Pvariable(config[CONF_ID], var) + await setup_speaker_core_(var, config) + + +SPEAKER_SCHEMA = cv.Schema.extend(audio.AUDIO_COMPONENT_SCHEMA).extend( + { + cv.Optional(CONF_AUDIO_DAC): cv.use_id(audio_dac.AudioDac), + } +) + +SPEAKER_AUTOMATION_SCHEMA = automation.maybe_simple_id( + {cv.GenerateID(): cv.use_id(Speaker)} +) + + +async def speaker_action(config, action_id, template_arg, args): + var = cg.new_Pvariable(action_id, template_arg) + await cg.register_parented(var, config[CONF_ID]) + return var + + +@automation.register_action( + "speaker.play", + PlayAction, + cv.maybe_simple_value( + { + cv.GenerateID(): cv.use_id(Speaker), + cv.Required(CONF_DATA): cv.templatable(cv.ensure_list(cv.hex_uint8_t)), + }, + key=CONF_DATA, + ), + synchronous=True, +) +async def speaker_play_action(config, action_id, template_arg, args): + var = cg.new_Pvariable(action_id, template_arg) + await cg.register_parented(var, config[CONF_ID]) + data = config[CONF_DATA] + + if cg.is_template(data): + templ = await cg.templatable(data, args, cg.std_vector.template(cg.uint8)) + cg.add(var.set_data_template(templ)) + else: + # Generate static array in flash to avoid RAM copy + arr_id = ID(f"{action_id}_data", is_declaration=True, type=cg.uint8) + arr = cg.static_const_array(arr_id, cg.ArrayInitializer(*data)) + cg.add(var.set_data_static(arr, len(data))) + return var + + +automation.register_action( + "speaker.stop", StopAction, SPEAKER_AUTOMATION_SCHEMA, synchronous=True +)(speaker_action) +automation.register_action( + "speaker.finish", FinishAction, SPEAKER_AUTOMATION_SCHEMA, synchronous=True +)(speaker_action) + +automation.register_condition( + "speaker.is_playing", IsPlayingCondition, SPEAKER_AUTOMATION_SCHEMA +)(speaker_action) + +automation.register_condition( + "speaker.is_stopped", IsStoppedCondition, SPEAKER_AUTOMATION_SCHEMA +)(speaker_action) + + +@automation.register_action( + "speaker.volume_set", + VolumeSetAction, + cv.maybe_simple_value( + { + cv.GenerateID(): cv.use_id(Speaker), + cv.Required(CONF_VOLUME): cv.templatable(cv.percentage), + }, + key=CONF_VOLUME, + ), + synchronous=True, +) +async def speaker_volume_set_action(config, action_id, template_arg, args): + var = cg.new_Pvariable(action_id, template_arg) + await cg.register_parented(var, config[CONF_ID]) + volume = await cg.templatable(config[CONF_VOLUME], args, cg.float_) + cg.add(var.set_volume(volume)) + return var + + +@automation.register_action( + "speaker.mute_off", + MuteOffAction, + SPEAKER_AUTOMATION_SCHEMA, + synchronous=True, +) +@automation.register_action( + "speaker.mute_on", MuteOnAction, SPEAKER_AUTOMATION_SCHEMA, synchronous=True +) +async def speaker_mute_action_to_code(config, action_id, template_arg, args): + paren = await cg.get_variable(config[CONF_ID]) + return cg.new_Pvariable(action_id, template_arg, paren) + + +@coroutine_with_priority(CoroPriority.CORE) +async def to_code(config): + cg.add_global(speaker_ns.using) + cg.add_define("USE_SPEAKER") diff --git a/firmware/esphome/components/speaker/automation.h b/firmware/esphome/components/speaker/automation.h new file mode 100644 index 0000000..443588a --- /dev/null +++ b/firmware/esphome/components/speaker/automation.h @@ -0,0 +1,86 @@ +#pragma once + +#include "esphome/core/automation.h" +#include "speaker.h" + +#include + +namespace esphome::speaker { + +template class PlayAction final : public Action, public Parented { + public: + void set_data_template(std::vector (*func)(Ts...)) { + this->data_.func = func; + this->len_ = -1; // Sentinel value indicates template mode + } + + void set_data_static(const uint8_t *data, size_t len) { + this->data_.data = data; + this->len_ = len; // Length >= 0 indicates static mode + } + + void play(const Ts &...x) override { + if (this->len_ >= 0) { + // Static mode: pass pointer directly to play(const uint8_t *, size_t) + this->parent_->play(this->data_.data, static_cast(this->len_)); + } else { + // Template mode: call function and pass vector to play(const std::vector &) + auto val = this->data_.func(x...); + this->parent_->play(val); + } + } + + protected: + ssize_t len_{-1}; // -1 = template mode, >=0 = static mode with length + union Data { + std::vector (*func)(Ts...); // Function pointer (stateless lambdas) + const uint8_t *data; // Pointer to static data in flash + } data_; +}; + +template class VolumeSetAction final : public Action, public Parented { + TEMPLATABLE_VALUE(float, volume) + void play(const Ts &...x) override { this->parent_->set_volume(this->volume_.value(x...)); } +}; + +template class MuteOnAction final : public Action { + public: + explicit MuteOnAction(Speaker *speaker) : speaker_(speaker) {} + + void play(const Ts &...x) override { this->speaker_->set_mute_state(true); } + + protected: + Speaker *speaker_; +}; + +template class MuteOffAction final : public Action { + public: + explicit MuteOffAction(Speaker *speaker) : speaker_(speaker) {} + + void play(const Ts &...x) override { this->speaker_->set_mute_state(false); } + + protected: + Speaker *speaker_; +}; + +template class StopAction final : public Action, public Parented { + public: + void play(const Ts &...x) override { this->parent_->stop(); } +}; + +template class FinishAction final : public Action, public Parented { + public: + void play(const Ts &...x) override { this->parent_->finish(); } +}; + +template class IsPlayingCondition final : public Condition, public Parented { + public: + bool check(const Ts &...x) override { return this->parent_->is_running(); } +}; + +template class IsStoppedCondition final : public Condition, public Parented { + public: + bool check(const Ts &...x) override { return this->parent_->is_stopped(); } +}; + +} // namespace esphome::speaker diff --git a/firmware/esphome/components/speaker/media_player/__init__.py b/firmware/esphome/components/speaker/media_player/__init__.py new file mode 100644 index 0000000..90eb19d --- /dev/null +++ b/firmware/esphome/components/speaker/media_player/__init__.py @@ -0,0 +1,277 @@ +"""Speaker Media Player Setup.""" + +import logging + +from esphome import automation +import esphome.codegen as cg +from esphome.components import ( + audio, + audio_file, + media_player, + network, + ota, + psram, + speaker, +) +from esphome.components.const import ( + CONF_VOLUME_INCREMENT, + CONF_VOLUME_INITIAL, + CONF_VOLUME_MAX, + CONF_VOLUME_MIN, +) +import esphome.config_validation as cv +from esphome.const import ( + CONF_BUFFER_SIZE, + CONF_FILES, + CONF_FORMAT, + CONF_ID, + CONF_NUM_CHANNELS, + CONF_ON_TURN_OFF, + CONF_ON_TURN_ON, + CONF_SAMPLE_RATE, + CONF_SPEAKER, + CONF_TASK_STACK_IN_PSRAM, +) + +_LOGGER = logging.getLogger(__name__) + + +AUTO_LOAD = ["audio"] +DEPENDENCIES = ["network"] + +CODEOWNERS = ["@kahrendt", "@synesthesiam"] +DOMAIN = "media_player" + +CONF_ANNOUNCEMENT = "announcement" +CONF_ANNOUNCEMENT_PIPELINE = "announcement_pipeline" +CONF_CODEC_SUPPORT_ENABLED = "codec_support_enabled" # Remove before 2026.10.0 +CONF_ENQUEUE = "enqueue" +CONF_MEDIA_FILE = "media_file" +CONF_MEDIA_PIPELINE = "media_pipeline" +CONF_ON_MUTE = "on_mute" +CONF_ON_UNMUTE = "on_unmute" +CONF_ON_VOLUME = "on_volume" +CONF_STREAM = "stream" + + +speaker_ns = cg.esphome_ns.namespace("speaker") +SpeakerMediaPlayer = speaker_ns.class_( + "SpeakerMediaPlayer", + media_player.MediaPlayer, + cg.Component, +) + +AudioPipeline = speaker_ns.class_("AudioPipeline") +AudioPipelineType = speaker_ns.enum("AudioPipelineType", is_class=True) +AUDIO_PIPELINE_TYPE_ENUM = { + "MEDIA": AudioPipelineType.MEDIA, + "ANNOUNCEMENT": AudioPipelineType.ANNOUNCEMENT, +} + +PlayOnDeviceMediaAction = speaker_ns.class_( + "PlayOnDeviceMediaAction", + automation.Action, + cg.Parented.template(SpeakerMediaPlayer), +) +StopStreamAction = speaker_ns.class_( + "StopStreamAction", automation.Action, cg.Parented.template(SpeakerMediaPlayer) +) + + +_PURPOSE_MAP = { + "MEDIA": media_player.MEDIA_PLAYER_FORMAT_PURPOSE_ENUM["default"], + "ANNOUNCEMENT": media_player.MEDIA_PLAYER_FORMAT_PURPOSE_ENUM["announcement"], +} + + +_validate_pipeline = media_player.validate_preferred_format( + "speaker media_player", CONF_SPEAKER +) + + +def _validate_repeated_speaker(config): + if ( + (announcement_config := config.get(CONF_ANNOUNCEMENT_PIPELINE)) + and (media_config := config.get(CONF_MEDIA_PIPELINE)) + and announcement_config[CONF_SPEAKER] == media_config[CONF_SPEAKER] + ): + raise cv.Invalid( + "The announcement and media pipelines cannot use the same speaker. Use the `mixer` speaker component to create two source speakers." + ) + + return config + + +def _final_validate(config): + # Remove before 2026.10.0 + if CONF_CODEC_SUPPORT_ENABLED in config: + _LOGGER.warning( + "'%s' is deprecated and will be removed in 2026.10.0. " + "Codec support is now automatically determined from the pipeline " + "'format' setting. Set format to 'NONE' to enable all codecs.", + CONF_CODEC_SUPPORT_ENABLED, + ) + + # Request codecs based on pipeline formats. Codecs needed by local files are + # already requested during CONFIG_SCHEMA validation (via audio_files_schema). + media_player.request_codecs_for_format_configs( + config, [CONF_ANNOUNCEMENT_PIPELINE, CONF_MEDIA_PIPELINE] + ) + + return config + + +PIPELINE_SCHEMA = cv.Schema( + { + cv.GenerateID(): cv.declare_id(AudioPipeline), + cv.Required(CONF_SPEAKER): cv.use_id(speaker.Speaker), + cv.Optional(CONF_FORMAT, default="FLAC"): cv.enum(audio.AUDIO_FILE_TYPE_ENUM), + cv.Optional(CONF_SAMPLE_RATE): cv.int_range(min=1), + cv.Optional(CONF_NUM_CHANNELS): cv.int_range(1, 2), + } +) + + +def _request_high_performance_networking(config): + """Request high performance networking for streaming media. + + Speaker media player streams audio data, so it always benefits from + optimized WiFi and lwip settings regardless of codec support. + Called during config validation to ensure flags are set before to_code(). + """ + network.require_high_performance_networking() + return config + + +CONFIG_SCHEMA = cv.All( + media_player.media_player_schema(SpeakerMediaPlayer).extend( + { + cv.Required(CONF_ANNOUNCEMENT_PIPELINE): PIPELINE_SCHEMA, + cv.Optional(CONF_MEDIA_PIPELINE): PIPELINE_SCHEMA, + cv.Optional(CONF_BUFFER_SIZE, default=1000000): cv.int_range( + min=4000, max=4000000 + ), + # Remove before 2026.10.0 + cv.Optional(CONF_CODEC_SUPPORT_ENABLED): cv.Any(cv.boolean, cv.string), + cv.Optional(CONF_FILES): audio_file.audio_files_schema(), + cv.Optional(CONF_TASK_STACK_IN_PSRAM): psram.validate_task_stack_in_psram, + cv.Optional(CONF_VOLUME_INCREMENT, default=0.05): cv.percentage, + cv.Optional(CONF_VOLUME_INITIAL, default=0.5): cv.percentage, + cv.Optional(CONF_VOLUME_MAX, default=1.0): cv.percentage, + cv.Optional(CONF_VOLUME_MIN, default=0.0): cv.percentage, + cv.Optional(CONF_ON_MUTE): automation.validate_automation(single=True), + cv.Optional(CONF_ON_UNMUTE): automation.validate_automation(single=True), + cv.Optional(CONF_ON_VOLUME): automation.validate_automation(single=True), + } + ), + cv.only_on_esp32, + _validate_repeated_speaker, + _request_high_performance_networking, +) + + +FINAL_VALIDATE_SCHEMA = cv.All( + cv.Schema( + { + cv.Optional(CONF_ANNOUNCEMENT_PIPELINE): _validate_pipeline, + cv.Optional(CONF_MEDIA_PIPELINE): _validate_pipeline, + }, + extra=cv.ALLOW_EXTRA, + ), + _final_validate, +) + + +async def to_code(config): + if CONF_ON_TURN_OFF in config or CONF_ON_TURN_ON in config: + cg.add_define("USE_SPEAKER_MEDIA_PLAYER_ON_OFF", True) + + var = await media_player.new_media_player(config) + await cg.register_component(var, config) + + ota.request_ota_state_listeners() + + cg.add(var.set_buffer_size(config[CONF_BUFFER_SIZE])) + + if config.get(CONF_TASK_STACK_IN_PSRAM): + cg.add(var.set_task_stack_in_psram(True)) + psram.request_external_task_stack() + + cg.add(var.set_volume_increment(config[CONF_VOLUME_INCREMENT])) + cg.add(var.set_volume_initial(config[CONF_VOLUME_INITIAL])) + cg.add(var.set_volume_max(config[CONF_VOLUME_MAX])) + cg.add(var.set_volume_min(config[CONF_VOLUME_MIN])) + + announcement_pipeline_config = config[CONF_ANNOUNCEMENT_PIPELINE] + spkr = await cg.get_variable(announcement_pipeline_config[CONF_SPEAKER]) + cg.add(var.set_announcement_speaker(spkr)) + if announcement_pipeline_config[CONF_FORMAT] != "NONE": + cg.add( + var.set_announcement_format( + media_player.build_supported_format_struct( + announcement_pipeline_config, _PURPOSE_MAP["ANNOUNCEMENT"] + ) + ) + ) + + if media_pipeline_config := config.get(CONF_MEDIA_PIPELINE): + spkr = await cg.get_variable(media_pipeline_config[CONF_SPEAKER]) + cg.add(var.set_media_speaker(spkr)) + if media_pipeline_config[CONF_FORMAT] != "NONE": + cg.add( + var.set_media_format( + media_player.build_supported_format_struct( + media_pipeline_config, _PURPOSE_MAP["MEDIA"] + ) + ) + ) + + if on_mute := config.get(CONF_ON_MUTE): + await automation.build_automation( + var.get_mute_trigger(), + [], + on_mute, + ) + if on_unmute := config.get(CONF_ON_UNMUTE): + await automation.build_automation( + var.get_unmute_trigger(), + [], + on_unmute, + ) + if on_volume := config.get(CONF_ON_VOLUME): + await automation.build_automation( + var.get_volume_trigger(), + [(cg.float_, "x")], + on_volume, + ) + + for file_config in config.get(CONF_FILES, []): + audio_file.generate_audio_file_code(file_config) + + +@automation.register_action( + "media_player.speaker.play_on_device_media_file", + PlayOnDeviceMediaAction, + cv.maybe_simple_value( + { + cv.GenerateID(): cv.use_id(SpeakerMediaPlayer), + cv.Required(CONF_MEDIA_FILE): cv.use_id(audio.AudioFile), + cv.Optional(CONF_ANNOUNCEMENT, default=False): cv.templatable(cv.boolean), + cv.Optional(CONF_ENQUEUE, default=False): cv.templatable(cv.boolean), + }, + key=CONF_MEDIA_FILE, + ), + synchronous=True, +) +async def play_on_device_media_media_action(config, action_id, template_arg, args): + var = cg.new_Pvariable(action_id, template_arg) + await cg.register_parented(var, config[CONF_ID]) + media_file = await cg.get_variable(config[CONF_MEDIA_FILE]) + announcement = await cg.templatable(config[CONF_ANNOUNCEMENT], args, cg.bool_) + enqueue = await cg.templatable(config[CONF_ENQUEUE], args, cg.bool_) + + template_ = await cg.templatable(media_file, args, audio.AudioFile.operator("ptr")) + cg.add(var.set_audio_file(template_)) + cg.add(var.set_announcement(announcement)) + cg.add(var.set_enqueue(enqueue)) + return var diff --git a/firmware/esphome/components/speaker/media_player/audio_pipeline.cpp b/firmware/esphome/components/speaker/media_player/audio_pipeline.cpp new file mode 100644 index 0000000..509984c --- /dev/null +++ b/firmware/esphome/components/speaker/media_player/audio_pipeline.cpp @@ -0,0 +1,526 @@ +#include "audio_pipeline.h" + +#ifdef USE_ESP32 + +#include "esphome/core/defines.h" +#include "esphome/core/hal.h" +#include "esphome/core/helpers.h" +#include "esphome/core/log.h" + +namespace esphome::speaker { + +static const uint32_t INITIAL_BUFFER_MS = 1000; // Start playback after buffering this duration of the file + +static const uint32_t READ_TASK_STACK_SIZE = 5 * 1024; +// Opus decoding uses more stack than other codecs +#ifdef USE_AUDIO_OPUS_SUPPORT +static const uint32_t DECODE_TASK_STACK_SIZE = 5 * 1024; +#else +static const uint32_t DECODE_TASK_STACK_SIZE = 3 * 1024; +#endif + +static const uint32_t INFO_ERROR_QUEUE_COUNT = 5; + +static const char *const TAG = "speaker_media_player.pipeline"; + +enum EventGroupBits : uint32_t { + // MESSAGE_* bits are only set by their respective tasks + + // Stops all activity in the pipeline elements; cleared by process_state() and set by stop() or by each task + PIPELINE_COMMAND_STOP = (1 << 0), + + // Read audio from an HTTP source; cleared by reader task and set by start_url + READER_COMMAND_INIT_HTTP = (1 << 4), + // Read audio from an audio file from the flash; cleared by reader task and set by start_file + READER_COMMAND_INIT_FILE = (1 << 5), + + // Audio file type is read after checking it is supported; cleared by decoder task + READER_MESSAGE_LOADED_MEDIA_TYPE = (1 << 6), + // Reader is done (either through a failure or just end of the stream); cleared by reader task + READER_MESSAGE_FINISHED = (1 << 7), + // Error reading the file; cleared by process_state() + READER_MESSAGE_ERROR = (1 << 8), + + // Decoder is done (either through a faiilure or the end of the stream); cleared by decoder task + DECODER_MESSAGE_FINISHED = (1 << 12), + // Error decoding the file; cleared by process_state() by decoder task + DECODER_MESSAGE_ERROR = (1 << 13), +}; + +AudioPipeline::AudioPipeline(speaker::Speaker *speaker, size_t buffer_size, bool task_stack_in_psram, + std::string base_name, UBaseType_t priority) + : base_name_(std::move(base_name)), + priority_(priority), + task_stack_in_psram_(task_stack_in_psram), + speaker_(speaker), + buffer_size_(buffer_size) { + this->allocate_communications_(); + this->transfer_buffer_size_ = std::min(buffer_size_ / 4, DEFAULT_TRANSFER_BUFFER_SIZE); +} + +void AudioPipeline::start_url(const std::string &uri) { + if (this->is_playing_) { + xEventGroupSetBits(this->event_group_, PIPELINE_COMMAND_STOP); + } + this->current_uri_ = uri; + this->pending_url_ = true; +} + +void AudioPipeline::start_file(audio::AudioFile *audio_file) { + if (this->is_playing_) { + xEventGroupSetBits(this->event_group_, PIPELINE_COMMAND_STOP); + } + this->current_audio_file_ = audio_file; + this->pending_file_ = true; +} + +esp_err_t AudioPipeline::stop() { + xEventGroupSetBits(this->event_group_, EventGroupBits::PIPELINE_COMMAND_STOP); + + return ESP_OK; +} +void AudioPipeline::set_pause_state(bool pause_state) { + this->speaker_->set_pause_state(pause_state); + + this->pause_state_ = pause_state; +} + +void AudioPipeline::suspend_tasks() { + if (this->read_task_.is_created()) { + vTaskSuspend(this->read_task_.get_handle()); + } + if (this->decode_task_.is_created()) { + vTaskSuspend(this->decode_task_.get_handle()); + } +} + +void AudioPipeline::resume_tasks() { + if (this->read_task_.is_created()) { + vTaskResume(this->read_task_.get_handle()); + } + if (this->decode_task_.is_created()) { + vTaskResume(this->decode_task_.get_handle()); + } +} + +AudioPipelineState AudioPipeline::process_state() { + /* + * Log items from info error queue + */ + InfoErrorEvent event; + if (this->info_error_queue_ != nullptr) { + while (xQueueReceive(this->info_error_queue_, &event, 0)) { + switch (event.source) { + case InfoErrorSource::READER: + if (event.err.has_value()) { + ESP_LOGE(TAG, "Media reader encountered an error: %s", esp_err_to_name(event.err.value())); + } else if (event.file_type.has_value()) { + ESP_LOGD(TAG, "Reading %s file type", audio_file_type_to_string(event.file_type.value())); + } + + break; + case InfoErrorSource::DECODER: + if (event.err.has_value()) { + ESP_LOGE(TAG, "Decoder encountered an error: %s", esp_err_to_name(event.err.value())); + } + + if (event.audio_stream_info.has_value()) { + ESP_LOGD(TAG, "Decoded audio has %d channels, %" PRId32 " Hz sample rate, and %d bits per sample", + event.audio_stream_info.value().get_channels(), event.audio_stream_info.value().get_sample_rate(), + event.audio_stream_info.value().get_bits_per_sample()); + } + + if (event.decoding_err.has_value()) { + switch (event.decoding_err.value()) { + case DecodingError::FAILED_HEADER: + ESP_LOGE(TAG, "Failed to parse the file's header."); + break; + case DecodingError::INCOMPATIBLE_BITS_PER_SAMPLE: + ESP_LOGE(TAG, "Incompatible bits per sample. Only 16 bits per sample is supported"); + break; + case DecodingError::INCOMPATIBLE_CHANNELS: + ESP_LOGE(TAG, "Incompatible number of channels. Only 1 or 2 channel audio is supported."); + break; + } + } + break; + } + } + } + + /* + * Determine the current state based on the event group bits and tasks' status + */ + + EventBits_t event_bits = xEventGroupGetBits(this->event_group_); + + if (this->pending_url_ || this->pending_file_) { + // Init command pending + if (!(event_bits & EventGroupBits::PIPELINE_COMMAND_STOP)) { + // Only start if there is no pending stop command + if (!this->read_task_.is_created() || !this->decode_task_.is_created()) { + // At least one task isn't running + this->start_tasks_(); + } + + if (this->pending_url_) { + xEventGroupSetBits(this->event_group_, EventGroupBits::READER_COMMAND_INIT_HTTP); + this->playback_ms_ = 0; + this->pending_url_ = false; + } else if (this->pending_file_) { + xEventGroupSetBits(this->event_group_, EventGroupBits::READER_COMMAND_INIT_FILE); + this->playback_ms_ = 0; + this->pending_file_ = false; + } + + this->is_playing_ = true; + return AudioPipelineState::PLAYING; + } + } + + if ((event_bits & EventGroupBits::READER_MESSAGE_ERROR)) { + xEventGroupClearBits(this->event_group_, EventGroupBits::READER_MESSAGE_ERROR); + return AudioPipelineState::ERROR_READING; + } + + if ((event_bits & EventGroupBits::DECODER_MESSAGE_ERROR)) { + xEventGroupClearBits(this->event_group_, EventGroupBits::DECODER_MESSAGE_ERROR); + return AudioPipelineState::ERROR_DECODING; + } + + if ((event_bits & EventGroupBits::READER_MESSAGE_FINISHED) && + (!(event_bits & EventGroupBits::READER_MESSAGE_LOADED_MEDIA_TYPE) && + (event_bits & EventGroupBits::DECODER_MESSAGE_FINISHED))) { + // Tasks are finished and there's no media in between the reader and decoder + + if (event_bits & EventGroupBits::PIPELINE_COMMAND_STOP) { + // Stop command is fully processed, so clear the command bit + xEventGroupClearBits(this->event_group_, EventGroupBits::PIPELINE_COMMAND_STOP); + this->hard_stop_ = true; + } + + if (!this->is_playing_) { + // The tasks have been stopped for two ``process_state`` calls in a row, so delete the tasks + if (this->read_task_.is_created() || this->decode_task_.is_created()) { + // Both are attempted every time; a task that is still running on the other core is freed by a + // subsequent call, and freeing an already freed task succeeds without doing anything + bool read_task_freed = this->read_task_.deallocate(); + bool decode_task_freed = this->decode_task_.deallocate(); + if (!read_task_freed || !decode_task_freed) { + // A task is still running on the other core, so keep the pipeline in its current state and try + // again on the next call + return AudioPipelineState::PLAYING; + } + if (this->hard_stop_) { + // Stop command was sent, so immediately end the playback + this->speaker_->stop(); + this->hard_stop_ = false; + } else { + // Decoded all the audio, so let the speaker finish playing before stopping + this->speaker_->finish(); + } + } + } + this->is_playing_ = false; + if (!this->speaker_->is_running()) { + return AudioPipelineState::STOPPED; + } else { + this->is_finishing_ = true; + } + } + + if (this->pause_state_) { + return AudioPipelineState::PAUSED; + } + + if (this->is_finishing_) { + if (!this->speaker_->is_running()) { + this->is_finishing_ = false; + } else { + return AudioPipelineState::PLAYING; + } + } + + if (!this->read_task_.is_created() && !this->decode_task_.is_created()) { + // No tasks are running, so the pipeline is stopped. + xEventGroupClearBits(this->event_group_, EventGroupBits::PIPELINE_COMMAND_STOP); + return AudioPipelineState::STOPPED; + } + + this->is_playing_ = true; + return AudioPipelineState::PLAYING; +} + +esp_err_t AudioPipeline::allocate_communications_() { + if (this->event_group_ == nullptr) + this->event_group_ = xEventGroupCreate(); + + if (this->event_group_ == nullptr) { + return ESP_ERR_NO_MEM; + } + + if (this->info_error_queue_ == nullptr) + this->info_error_queue_ = xQueueCreate(INFO_ERROR_QUEUE_COUNT, sizeof(InfoErrorEvent)); + + if (this->info_error_queue_ == nullptr) + return ESP_ERR_NO_MEM; + + return ESP_OK; +} + +esp_err_t AudioPipeline::start_tasks_() { + if (!this->read_task_.is_created()) { + // Reader task uses the AudioReader class which uses esp_http_client. This crashes on IDF 5.4 if the task stack is + // in PSRAM. As a workaround, always allocate the read task in internal memory. + if (!this->read_task_.create(read_task, (this->base_name_ + "_read").c_str(), READ_TASK_STACK_SIZE, (void *) this, + this->priority_, false)) { + return ESP_ERR_NO_MEM; + } + } + + if (!this->decode_task_.is_created()) { + if (!this->decode_task_.create(decode_task, (this->base_name_ + "_decode").c_str(), DECODE_TASK_STACK_SIZE, + (void *) this, this->priority_, this->task_stack_in_psram_)) { + return ESP_ERR_NO_MEM; + } + } + + return ESP_OK; +} + +void AudioPipeline::read_task(void *params) { + AudioPipeline *this_pipeline = (AudioPipeline *) params; + + while (true) { + xEventGroupSetBits(this_pipeline->event_group_, EventGroupBits::READER_MESSAGE_FINISHED); + + // Wait until the pipeline notifies us the source of the media file + EventBits_t event_bits = xEventGroupWaitBits( + this_pipeline->event_group_, + EventGroupBits::READER_COMMAND_INIT_FILE | EventGroupBits::READER_COMMAND_INIT_HTTP, // Bit message to read + pdFALSE, // Clear the bit on exit + pdFALSE, // Wait for all the bits, + portMAX_DELAY); // Block indefinitely until bit is set + + if (!(event_bits & EventGroupBits::PIPELINE_COMMAND_STOP)) { + xEventGroupClearBits(this_pipeline->event_group_, EventGroupBits::READER_MESSAGE_FINISHED | + EventGroupBits::READER_COMMAND_INIT_FILE | + EventGroupBits::READER_COMMAND_INIT_HTTP); + InfoErrorEvent event; + event.source = InfoErrorSource::READER; + esp_err_t err = ESP_OK; + + std::unique_ptr reader = + make_unique(this_pipeline->transfer_buffer_size_); + + if (event_bits & EventGroupBits::READER_COMMAND_INIT_FILE) { + err = reader->start(this_pipeline->current_audio_file_, this_pipeline->current_audio_file_type_); + } else { + err = reader->start(this_pipeline->current_uri_, this_pipeline->current_audio_file_type_); + } + + if (err == ESP_OK) { + size_t file_ring_buffer_size = this_pipeline->buffer_size_; + + std::shared_ptr temp_ring_buffer = this_pipeline->raw_file_ring_buffer_.lock(); + + if (temp_ring_buffer == nullptr) { + temp_ring_buffer = ring_buffer::RingBuffer::create(file_ring_buffer_size); + this_pipeline->raw_file_ring_buffer_ = temp_ring_buffer; + } + + if (temp_ring_buffer == nullptr) { + err = ESP_ERR_NO_MEM; + } else { + err = reader->add_sink(temp_ring_buffer); + } + } + + if (err != ESP_OK) { + // Send specific error message + event.err = err; + xQueueSend(this_pipeline->info_error_queue_, &event, portMAX_DELAY); + + // Setting up the reader failed, stop the pipeline + xEventGroupSetBits(this_pipeline->event_group_, + EventGroupBits::READER_MESSAGE_ERROR | EventGroupBits::PIPELINE_COMMAND_STOP); + } else { + // Send the file type to the pipeline + event.file_type = this_pipeline->current_audio_file_type_; + xQueueSend(this_pipeline->info_error_queue_, &event, portMAX_DELAY); + xEventGroupSetBits(this_pipeline->event_group_, EventGroupBits::READER_MESSAGE_LOADED_MEDIA_TYPE); + } + + while (true) { + event_bits = xEventGroupGetBits(this_pipeline->event_group_); + + if (event_bits & EventGroupBits::PIPELINE_COMMAND_STOP) { + break; + } + + audio::AudioReaderState reader_state = reader->read(); + + if (reader_state == audio::AudioReaderState::FINISHED) { + break; + } else if (reader_state == audio::AudioReaderState::FAILED) { + xEventGroupSetBits(this_pipeline->event_group_, + EventGroupBits::READER_MESSAGE_ERROR | EventGroupBits::PIPELINE_COMMAND_STOP); + break; + } + } + event_bits = xEventGroupGetBits(this_pipeline->event_group_); + if ((event_bits & EventGroupBits::READER_MESSAGE_LOADED_MEDIA_TYPE) || + (this_pipeline->raw_file_ring_buffer_.use_count() == 1)) { + // Decoder task hasn't started yet, so delay a bit before releasing ownership of the ring buffer + delay(10); + } + } + } +} + +void AudioPipeline::decode_task(void *params) { + AudioPipeline *this_pipeline = (AudioPipeline *) params; + + while (true) { + xEventGroupSetBits(this_pipeline->event_group_, EventGroupBits::DECODER_MESSAGE_FINISHED); + + // Wait until the reader notifies us that the media type is available + EventBits_t event_bits = + xEventGroupWaitBits(this_pipeline->event_group_, + EventGroupBits::READER_MESSAGE_LOADED_MEDIA_TYPE, // Bit message to read + pdFALSE, // Clear the bit on exit + pdFALSE, // Wait for all the bits, + portMAX_DELAY); // Block indefinitely until bit is set + + xEventGroupClearBits(this_pipeline->event_group_, + EventGroupBits::DECODER_MESSAGE_FINISHED | EventGroupBits::READER_MESSAGE_LOADED_MEDIA_TYPE); + + if (!(event_bits & EventGroupBits::PIPELINE_COMMAND_STOP)) { + InfoErrorEvent event; + event.source = InfoErrorSource::DECODER; + + std::unique_ptr decoder = + make_unique(this_pipeline->transfer_buffer_size_, this_pipeline->transfer_buffer_size_); + + esp_err_t err = decoder->start(this_pipeline->current_audio_file_type_); + if (err == ESP_OK) { + err = decoder->add_source(this_pipeline->raw_file_ring_buffer_); + } + + if (err != ESP_OK) { + // Send specific error message + event.err = err; + xQueueSend(this_pipeline->info_error_queue_, &event, portMAX_DELAY); + + // Setting up the decoder failed, stop the pipeline + xEventGroupSetBits(this_pipeline->event_group_, + EventGroupBits::DECODER_MESSAGE_ERROR | EventGroupBits::PIPELINE_COMMAND_STOP); + } + + bool has_stream_info = false; + bool started_playback = false; + + size_t initial_bytes_to_buffer = 0; + + while (true) { + event_bits = xEventGroupGetBits(this_pipeline->event_group_); + + if (event_bits & EventGroupBits::PIPELINE_COMMAND_STOP) { + break; + } + + // Update pause state + if (!started_playback) { + if (!(event_bits & EventGroupBits::READER_MESSAGE_FINISHED)) { + decoder->set_pause_output_state(true); + } else { + started_playback = true; + } + } else { + decoder->set_pause_output_state(this_pipeline->pause_state_); + } + + // Stop gracefully if the reader has finished + audio::AudioDecoderState decoder_state = decoder->decode(event_bits & EventGroupBits::READER_MESSAGE_FINISHED); + + if ((decoder_state == audio::AudioDecoderState::DECODING) || + (decoder_state == audio::AudioDecoderState::FINISHED)) { + this_pipeline->playback_ms_ = decoder->get_playback_ms(); + } + + if (decoder_state == audio::AudioDecoderState::FINISHED) { + break; + } else if (decoder_state == audio::AudioDecoderState::FAILED) { + if (!has_stream_info) { + event.decoding_err = DecodingError::FAILED_HEADER; + xQueueSend(this_pipeline->info_error_queue_, &event, portMAX_DELAY); + } + xEventGroupSetBits(this_pipeline->event_group_, + EventGroupBits::DECODER_MESSAGE_ERROR | EventGroupBits::PIPELINE_COMMAND_STOP); + break; + } + + if (!has_stream_info && decoder->get_audio_stream_info().has_value()) { + has_stream_info = true; + + this_pipeline->current_audio_stream_info_ = decoder->get_audio_stream_info().value(); + + // Send the stream information to the pipeline + event.audio_stream_info = this_pipeline->current_audio_stream_info_; + + if (this_pipeline->current_audio_stream_info_.get_bits_per_sample() != 16) { + // Error state, incompatible bits per sample + event.decoding_err = DecodingError::INCOMPATIBLE_BITS_PER_SAMPLE; + xEventGroupSetBits(this_pipeline->event_group_, + EventGroupBits::DECODER_MESSAGE_ERROR | EventGroupBits::PIPELINE_COMMAND_STOP); + } else if ((this_pipeline->current_audio_stream_info_.get_channels() > 2)) { + // Error state, incompatible number of channels + event.decoding_err = DecodingError::INCOMPATIBLE_CHANNELS; + xEventGroupSetBits(this_pipeline->event_group_, + EventGroupBits::DECODER_MESSAGE_ERROR | EventGroupBits::PIPELINE_COMMAND_STOP); + } else { + // Send audio directly to the speaker + this_pipeline->speaker_->set_audio_stream_info(this_pipeline->current_audio_stream_info_); + decoder->add_sink(this_pipeline->speaker_); + } + + initial_bytes_to_buffer = std::min(this_pipeline->current_audio_stream_info_.ms_to_bytes(INITIAL_BUFFER_MS), + this_pipeline->buffer_size_ * 3 / 4); + + switch (this_pipeline->current_audio_file_type_) { +#ifdef USE_AUDIO_MP3_SUPPORT + case audio::AudioFileType::MP3: + initial_bytes_to_buffer /= 8; // Estimate the MP3 compression factor is 8 + break; +#endif +#ifdef USE_AUDIO_FLAC_SUPPORT + case audio::AudioFileType::FLAC: + initial_bytes_to_buffer /= 2; // Estimate the FLAC compression factor is 2 + break; +#endif +#ifdef USE_AUDIO_OPUS_SUPPORT + case audio::AudioFileType::OPUS: + initial_bytes_to_buffer /= 8; // Estimate the Opus compression factor is 8 + break; +#endif + default: + break; + } + xQueueSend(this_pipeline->info_error_queue_, &event, portMAX_DELAY); + } + + if (!started_playback && has_stream_info) { + // Verify enough data is available before starting playback + std::shared_ptr temp_ring_buffer = this_pipeline->raw_file_ring_buffer_.lock(); + if (temp_ring_buffer != nullptr && temp_ring_buffer->available() >= initial_bytes_to_buffer) { + started_playback = true; + } + } + } + } + } +} + +} // namespace esphome::speaker + +#endif diff --git a/firmware/esphome/components/speaker/media_player/audio_pipeline.h b/firmware/esphome/components/speaker/media_player/audio_pipeline.h new file mode 100644 index 0000000..02dad15 --- /dev/null +++ b/firmware/esphome/components/speaker/media_player/audio_pipeline.h @@ -0,0 +1,151 @@ +#pragma once + +#ifdef USE_ESP32 + +#include "esphome/components/audio/audio.h" +#include "esphome/components/audio/audio_reader.h" +#include "esphome/components/audio/audio_decoder.h" +#include "esphome/components/ring_buffer/ring_buffer.h" +#include "esphome/components/speaker/speaker.h" + +#include "esphome/core/static_task.h" + +#include "esp_err.h" + +#include +#include + +namespace esphome::speaker { + +// Internal sink/source buffers for reader and decoder +static const size_t DEFAULT_TRANSFER_BUFFER_SIZE = 24 * 1024; + +enum class AudioPipelineType : uint8_t { + MEDIA, + ANNOUNCEMENT, +}; + +enum class AudioPipelineState : uint8_t { + STARTING_FILE, + STARTING_URL, + PLAYING, + STOPPING, + STOPPED, + PAUSED, + ERROR_READING, + ERROR_DECODING, +}; + +enum class InfoErrorSource : uint8_t { + READER = 0, + DECODER, +}; + +enum class DecodingError : uint8_t { + FAILED_HEADER = 0, + INCOMPATIBLE_BITS_PER_SAMPLE, + INCOMPATIBLE_CHANNELS, +}; + +// Used to pass information from each task. +struct InfoErrorEvent { + InfoErrorSource source; + optional err; + optional file_type; + optional audio_stream_info; + optional decoding_err; +}; + +class AudioPipeline final { + public: + /// @param speaker ESPHome speaker component for pipeline's audio output + /// @param buffer_size Size of the buffer in bytes between the reader and decoder + /// @param task_stack_in_psram True if the task stack should be allocated in PSRAM, false otherwise + /// @param task_name FreeRTOS task base name + /// @param priority FreeRTOS task priority + AudioPipeline(speaker::Speaker *speaker, size_t buffer_size, bool task_stack_in_psram, std::string base_name, + UBaseType_t priority); + + /// @brief Starts an audio pipeline given a media url + /// @param uri media file url + /// @return ESP_OK if successful or an appropriate error if not + void start_url(const std::string &uri); + + /// @brief Starts an audio pipeline given a AudioFile pointer + /// @param audio_file pointer to an AudioFile object + /// @return ESP_OK if successful or an appropriate error if not + void start_file(audio::AudioFile *audio_file); + + /// @brief Stops the pipeline. Sends a stop signal to each task (if running) and clears the ring buffers. + /// @return ESP_OK if successful or ESP_ERR_TIMEOUT if the tasks did not indicate they stopped + esp_err_t stop(); + + /// @brief Processes the state of the audio pipeline based on the info_error_queue_ and event_group_. Handles creating + /// and stopping the pipeline tasks. Needs to be regularly called to update the internal pipeline state. + /// @return AudioPipelineState + AudioPipelineState process_state(); + + /// @brief Suspends any running tasks + void suspend_tasks(); + /// @brief Resumes any running tasks + void resume_tasks(); + + uint32_t get_playback_ms() { return this->playback_ms_; } + + void set_pause_state(bool pause_state); + + protected: + /// @brief Allocates the event group and info error queue. + /// @return ESP_OK if successful or ESP_ERR_NO_MEM if it is unable to allocate all parts + esp_err_t allocate_communications_(); + + /// @brief Common start code for the pipeline, regardless if the source is a file or url. + /// @return ESP_OK if successful or an appropriate error if not + esp_err_t start_tasks_(); + + std::string base_name_; + UBaseType_t priority_; + + uint32_t playback_ms_{0}; + + bool hard_stop_{false}; + bool is_playing_{false}; + bool is_finishing_{false}; + bool pause_state_{false}; + bool task_stack_in_psram_; + + // Pending file start state used to ensure the pipeline fully stops before attempting to start the next file + bool pending_url_{false}; + bool pending_file_{false}; + + speaker::Speaker *speaker_{nullptr}; + + std::string current_uri_{}; + audio::AudioFile *current_audio_file_{nullptr}; + + audio::AudioFileType current_audio_file_type_; + audio::AudioStreamInfo current_audio_stream_info_; + + size_t buffer_size_; // Ring buffer between reader and decoder + size_t transfer_buffer_size_; // Internal source/sink buffers for the audio reader and decoder + + std::weak_ptr raw_file_ring_buffer_; + + // Handles basic control/state of the three tasks + EventGroupHandle_t event_group_{nullptr}; + + // Receives detailed info (file type, stream info, resampling info) or specific errors from the three tasks + QueueHandle_t info_error_queue_{nullptr}; + + // Handles reading the media file from flash or a url + static void read_task(void *params); + StaticTask read_task_; + + // Decodes the media file into PCM audio + static void decode_task(void *params); + StaticTask decode_task_; +}; + +} // namespace esphome::speaker + +#endif diff --git a/firmware/esphome/components/speaker/media_player/automation.h b/firmware/esphome/components/speaker/media_player/automation.h new file mode 100644 index 0000000..f9e2127 --- /dev/null +++ b/firmware/esphome/components/speaker/media_player/automation.h @@ -0,0 +1,25 @@ +#pragma once + +#include "speaker_media_player.h" + +#ifdef USE_ESP32 + +#include "esphome/components/audio/audio.h" +#include "esphome/core/automation.h" + +namespace esphome::speaker { + +template +class PlayOnDeviceMediaAction final : public Action, public Parented { + TEMPLATABLE_VALUE(audio::AudioFile *, audio_file) + TEMPLATABLE_VALUE(bool, announcement) + TEMPLATABLE_VALUE(bool, enqueue) + void play(const Ts &...x) override { + this->parent_->play_file(this->audio_file_.value(x...), this->announcement_.value(x...), + this->enqueue_.value(x...)); + } +}; + +} // namespace esphome::speaker + +#endif diff --git a/firmware/esphome/components/speaker/media_player/speaker_media_player.cpp b/firmware/esphome/components/speaker/media_player/speaker_media_player.cpp new file mode 100644 index 0000000..fe994f4 --- /dev/null +++ b/firmware/esphome/components/speaker/media_player/speaker_media_player.cpp @@ -0,0 +1,626 @@ +#include "speaker_media_player.h" + +#ifdef USE_ESP32 + +#include "esphome/core/log.h" + +#include "esphome/components/audio/audio.h" +#ifdef USE_OTA +#include "esphome/components/ota/ota_backend.h" +#endif + +namespace esphome::speaker { + +// Framework: +// - Media player that can handle two streams: one for media and one for announcements +// - Each stream has an individual speaker component for output +// - Each stream is handled by an ``AudioPipeline`` object with two parts/tasks +// - ``AudioReader`` handles reading from an HTTP source or from a PROGMEM flash set at compile time +// - ``AudioDecoder`` handles decoding the audio file. All formats are limited to two channels and 16 bits per +// sample. +// Each format is enabled independently at compile time: +// - FLAC +// - MP3 (based on the libhelix decoder) +// - Ogg Opus +// - WAV +// - Each task runs until it is done processing the file or it receives a stop command +// - Inter-task communication uses a FreeRTOS Event Group +// - The ``AudioPipeline`` sets up a ring buffer between the reader and decoder tasks. The decoder task outputs audio +// directly to a speaker component. +// - The pipelines internal state needs to be processed by regularly calling ``process_state``. +// - Generic media player commands are received by the ``control`` function. The commands are added to the +// ``media_control_command_queue_`` to be processed in the component's loop +// - Local file play back is initiatied with ``play_file`` and adds it to the ``media_control_command_queue_`` +// - Starting a stream intializes the appropriate pipeline or stops it if it is already running +// - Volume and mute commands are achieved by the ``mute``, ``unmute``, ``set_volume`` functions. +// - Volume commands are ignored if the media control queue is full to avoid crashing with rapid volume +// increases/decreases. +// - These functions all send the appropriate information to the speakers to implement. +// - Pausing is implemented in the decoder task and is also sent directly to the media speaker component to decrease +// latency. +// - The components main loop performs housekeeping: +// - It reads the media control queue and processes it directly +// - It determines the overall state of the media player by considering the state of each pipeline +// - announcement playback takes highest priority +// - Handles playlists and repeating by starting the appropriate file when a previous file is finished +// - Logging only happens in the main loop task to reduce task stack memory usage. + +static const uint32_t MEDIA_CONTROLS_QUEUE_LENGTH = 20; + +static const UBaseType_t MEDIA_PIPELINE_TASK_PRIORITY = 1; +static const UBaseType_t ANNOUNCEMENT_PIPELINE_TASK_PRIORITY = 1; + +static const char *const TAG = "speaker_media_player"; + +void SpeakerMediaPlayer::setup() { +#ifdef USE_SPEAKER_MEDIA_PLAYER_ON_OFF + state = media_player::MEDIA_PLAYER_STATE_OFF; +#else + state = media_player::MEDIA_PLAYER_STATE_IDLE; +#endif + + this->media_control_command_queue_ = xQueueCreate(MEDIA_CONTROLS_QUEUE_LENGTH, sizeof(MediaCallCommand)); + + this->pref_ = this->make_entity_preference(); + + VolumeRestoreState volume_restore_state; + if (this->pref_.load(&volume_restore_state)) { + this->set_volume_(volume_restore_state.volume); + this->set_mute_state_(volume_restore_state.is_muted); + } else { + this->set_volume_(this->volume_initial_); + this->set_mute_state_(false); + } + +#ifdef USE_OTA_STATE_LISTENER + ota::get_global_ota_callback()->add_global_state_listener(this); +#endif + + this->announcement_pipeline_ = + make_unique(this->announcement_speaker_, this->buffer_size_, this->task_stack_in_psram_, "ann", + ANNOUNCEMENT_PIPELINE_TASK_PRIORITY); + + if (this->announcement_pipeline_ == nullptr) { + ESP_LOGE(TAG, "Failed to create announcement pipeline"); + this->mark_failed(); + } + + if (!this->single_pipeline_()) { + this->media_pipeline_ = make_unique(this->media_speaker_, this->buffer_size_, + this->task_stack_in_psram_, "med", MEDIA_PIPELINE_TASK_PRIORITY); + + if (this->media_pipeline_ == nullptr) { + ESP_LOGE(TAG, "Failed to create media pipeline"); + this->mark_failed(); + } + } + + ESP_LOGI(TAG, "Set up speaker media player"); +} + +void SpeakerMediaPlayer::set_playlist_delay_ms(AudioPipelineType pipeline_type, uint32_t delay_ms) { + switch (pipeline_type) { + case AudioPipelineType::ANNOUNCEMENT: + this->announcement_playlist_delay_ms_ = delay_ms; + break; + case AudioPipelineType::MEDIA: + this->media_playlist_delay_ms_ = delay_ms; + break; + } +} + +void SpeakerMediaPlayer::stop_and_unpause_media_() { + this->media_pipeline_->stop(); + this->unpause_media_remaining_ = 3; + this->set_interval("unpause_med", 50, [this]() { + if (this->media_pipeline_state_ == AudioPipelineState::STOPPED) { + this->cancel_interval("unpause_med"); + this->media_pipeline_->set_pause_state(false); + this->is_paused_ = false; + } else if (--this->unpause_media_remaining_ == 0) { + this->cancel_interval("unpause_med"); + } + }); +} + +void SpeakerMediaPlayer::watch_media_commands_() { + if (!this->is_ready()) { + return; + } + + MediaCallCommand media_command; + + if (xQueueReceive(this->media_control_command_queue_, &media_command, 0) == pdTRUE) { + bool enqueue = media_command.enqueue.has_value() && media_command.enqueue.value(); + + if (media_command.url.has_value() || media_command.file.has_value()) { +#ifdef USE_SPEAKER_MEDIA_PLAYER_ON_OFF + if (this->state == media_player::MEDIA_PLAYER_STATE_OFF) { + this->state = media_player::MEDIA_PLAYER_STATE_ON; + publish_state(); + } +#endif + PlaylistItem playlist_item; + if (media_command.url.has_value()) { + playlist_item.url = *media_command.url.value(); + delete media_command.url.value(); + } + if (media_command.file.has_value()) { + playlist_item.file = media_command.file; + } + + if (this->single_pipeline_() || (media_command.announce.has_value() && media_command.announce.value())) { + if (!enqueue) { + // Ensure the loaded next item doesn't start playing, clear the queue, start the file, and unpause + this->cancel_timeout("next_ann"); + this->announcement_playlist_.clear(); + if (media_command.file.has_value()) { + this->announcement_pipeline_->start_file(playlist_item.file.value()); + } else if (media_command.url.has_value()) { + this->announcement_pipeline_->start_url(playlist_item.url.value()); + } + this->announcement_pipeline_->set_pause_state(false); + } + this->announcement_playlist_.push_back(playlist_item); + } else { + if (!enqueue) { + // Ensure the loaded next item doesn't start playing, clear the queue, start the file, and unpause + this->cancel_timeout("next_media"); + this->media_playlist_.clear(); + if (this->is_paused_) { + // If paused, stop the media pipeline and unpause it after confirming its stopped. This avoids playing a + // short segment of the paused file before starting the new one. + this->stop_and_unpause_media_(); + } else { + // Not paused, just directly start the file + if (media_command.file.has_value()) { + this->media_pipeline_->start_file(playlist_item.file.value()); + } else if (media_command.url.has_value()) { + this->media_pipeline_->start_url(playlist_item.url.value()); + } + this->media_pipeline_->set_pause_state(false); + this->is_paused_ = false; + } + } + this->media_playlist_.push_back(playlist_item); + } + + return; // Don't process the new file play command further + } + + if (media_command.volume.has_value()) { + this->set_volume_(media_command.volume.value()); + this->publish_state(); + } + + if (media_command.command.has_value()) { + switch (media_command.command.value()) { + case media_player::MEDIA_PLAYER_COMMAND_PLAY: +#ifdef USE_SPEAKER_MEDIA_PLAYER_ON_OFF + if (this->state == media_player::MEDIA_PLAYER_STATE_OFF) { + this->state = media_player::MEDIA_PLAYER_STATE_ON; + publish_state(); + } +#endif + if ((this->media_pipeline_ != nullptr) && (this->is_paused_)) { + this->media_pipeline_->set_pause_state(false); + } + this->is_paused_ = false; + break; + case media_player::MEDIA_PLAYER_COMMAND_PAUSE: + if ((this->media_pipeline_ != nullptr) && (!this->is_paused_)) { + this->media_pipeline_->set_pause_state(true); + } + this->is_paused_ = true; + break; +#ifdef USE_SPEAKER_MEDIA_PLAYER_ON_OFF + case media_player::MEDIA_PLAYER_COMMAND_TURN_ON: + if (this->state == media_player::MEDIA_PLAYER_STATE_OFF) { + this->state = media_player::MEDIA_PLAYER_STATE_ON; + this->publish_state(); + } + break; + case media_player::MEDIA_PLAYER_COMMAND_TURN_OFF: + this->is_turn_off_ = true; + // Intentional Fall-through +#endif + case media_player::MEDIA_PLAYER_COMMAND_STOP: + // Pipelines do not stop immediately after calling the stop command, so confirm its stopped before unpausing. + // This avoids an audible short segment playing after receiving the stop command in a paused state. +#ifdef USE_SPEAKER_MEDIA_PLAYER_ON_OFF + if (this->single_pipeline_() || (media_command.announce.has_value() && media_command.announce.value()) || + (this->is_turn_off_ && this->announcement_pipeline_state_ != AudioPipelineState::STOPPED)) { +#else + if (this->single_pipeline_() || (media_command.announce.has_value() && media_command.announce.value())) { +#endif + if (this->announcement_pipeline_ != nullptr) { + this->cancel_timeout("next_ann"); + this->announcement_playlist_.clear(); + this->announcement_pipeline_->stop(); + this->unpause_announcement_remaining_ = 3; + this->set_interval("unpause_ann", 50, [this]() { + if (this->announcement_pipeline_state_ == AudioPipelineState::STOPPED) { + this->cancel_interval("unpause_ann"); + this->announcement_pipeline_->set_pause_state(false); + } else if (--this->unpause_announcement_remaining_ == 0) { + this->cancel_interval("unpause_ann"); + } + }); + } + } else { + if (this->media_pipeline_ != nullptr) { + this->cancel_timeout("next_media"); + this->media_playlist_.clear(); + this->stop_and_unpause_media_(); + } + } + + break; + case media_player::MEDIA_PLAYER_COMMAND_TOGGLE: + if (this->media_pipeline_ != nullptr) { + if (this->is_paused_) { + this->media_pipeline_->set_pause_state(false); + this->is_paused_ = false; + } else { + this->media_pipeline_->set_pause_state(true); + this->is_paused_ = true; + } + } + break; + case media_player::MEDIA_PLAYER_COMMAND_MUTE: { + this->set_mute_state_(true); + + this->publish_state(); + break; + } + case media_player::MEDIA_PLAYER_COMMAND_UNMUTE: + this->set_mute_state_(false); + this->publish_state(); + break; + case media_player::MEDIA_PLAYER_COMMAND_VOLUME_UP: + this->set_volume_(std::min(1.0f, this->volume + this->volume_increment_)); + this->publish_state(); + break; + case media_player::MEDIA_PLAYER_COMMAND_VOLUME_DOWN: + this->set_volume_(std::max(0.0f, this->volume - this->volume_increment_)); + this->publish_state(); + break; + case media_player::MEDIA_PLAYER_COMMAND_REPEAT_ONE: + if (this->single_pipeline_() || (media_command.announce.has_value() && media_command.announce.value())) { + this->announcement_repeat_one_ = true; + } else { + this->media_repeat_one_ = true; + } + break; + case media_player::MEDIA_PLAYER_COMMAND_REPEAT_OFF: + if (this->single_pipeline_() || (media_command.announce.has_value() && media_command.announce.value())) { + this->announcement_repeat_one_ = false; + } else { + this->media_repeat_one_ = false; + } + break; + case media_player::MEDIA_PLAYER_COMMAND_CLEAR_PLAYLIST: + if (this->single_pipeline_() || (media_command.announce.has_value() && media_command.announce.value())) { + if (this->announcement_playlist_.empty()) { + this->announcement_playlist_.resize(1); + } + } else { + if (this->media_playlist_.empty()) { + this->media_playlist_.resize(1); + } + } + break; + default: + break; + } + } + } +} + +#ifdef USE_OTA_STATE_LISTENER +void SpeakerMediaPlayer::on_ota_global_state(ota::OTAState state, float progress, uint8_t error, + ota::OTAComponent *comp) { + if (state == ota::OTA_STARTED) { + if (this->media_pipeline_ != nullptr) { + this->media_pipeline_->suspend_tasks(); + } + if (this->announcement_pipeline_ != nullptr) { + this->announcement_pipeline_->suspend_tasks(); + } + } else if (state == ota::OTA_ERROR) { + if (this->media_pipeline_ != nullptr) { + this->media_pipeline_->resume_tasks(); + } + if (this->announcement_pipeline_ != nullptr) { + this->announcement_pipeline_->resume_tasks(); + } + } +} +#endif + +void SpeakerMediaPlayer::loop() { + this->watch_media_commands_(); + + // Determine state of the media player + media_player::MediaPlayerState old_state = this->state; + + AudioPipelineState old_media_pipeline_state = this->media_pipeline_state_; + if (this->media_pipeline_ != nullptr) { + this->media_pipeline_state_ = this->media_pipeline_->process_state(); + } + + if (this->media_pipeline_state_ == AudioPipelineState::ERROR_READING) { + ESP_LOGE(TAG, "The media pipeline's file reader encountered an error."); + } else if (this->media_pipeline_state_ == AudioPipelineState::ERROR_DECODING) { + ESP_LOGE(TAG, "The media pipeline's audio decoder encountered an error."); + } + + AudioPipelineState old_announcement_pipeline_state = this->announcement_pipeline_state_; + if (this->announcement_pipeline_ != nullptr) { + this->announcement_pipeline_state_ = this->announcement_pipeline_->process_state(); + } + + if (this->announcement_pipeline_state_ == AudioPipelineState::ERROR_READING) { + ESP_LOGE(TAG, "The announcement pipeline's file reader encountered an error."); + } else if (this->announcement_pipeline_state_ == AudioPipelineState::ERROR_DECODING) { + ESP_LOGE(TAG, "The announcement pipeline's audio decoder encountered an error."); + } + + if (this->announcement_pipeline_state_ != AudioPipelineState::STOPPED) { + this->state = media_player::MEDIA_PLAYER_STATE_ANNOUNCING; + } else { + if (!this->announcement_playlist_.empty()) { + uint32_t timeout_ms = 0; + if (old_announcement_pipeline_state == AudioPipelineState::PLAYING) { + // Finished the current announcement file + if (!this->announcement_repeat_one_) { + // Pop item off the playlist if repeat is disabled + this->announcement_playlist_.pop_front(); + } + // Only delay starting playback if moving on the next playlist item or repeating the current item + timeout_ms = this->announcement_playlist_delay_ms_; + } + + if (!this->announcement_playlist_.empty()) { + // Start the next announcement file + PlaylistItem playlist_item = this->announcement_playlist_.front(); + if (playlist_item.url.has_value()) { + this->announcement_pipeline_->start_url(playlist_item.url.value()); + } else if (playlist_item.file.has_value()) { + this->announcement_pipeline_->start_file(playlist_item.file.value()); + } + + if (timeout_ms > 0) { + // Pause pipeline internally to facilitate the delay between items + this->announcement_pipeline_->set_pause_state(true); + // Internally unpause the pipeline after the delay between playlist items. Announcements do not follow the + // media player's pause state. + this->set_timeout("next_ann", timeout_ms, [this]() { this->announcement_pipeline_->set_pause_state(false); }); + } + } + } else { + if (this->is_paused_) { +#ifdef USE_SPEAKER_MEDIA_PLAYER_ON_OFF + if (this->state != media_player::MEDIA_PLAYER_STATE_OFF) { + this->state = media_player::MEDIA_PLAYER_STATE_PAUSED; + } +#else + this->state = media_player::MEDIA_PLAYER_STATE_PAUSED; +#endif + } else if (this->media_pipeline_state_ == AudioPipelineState::PLAYING) { + this->state = media_player::MEDIA_PLAYER_STATE_PLAYING; + } else if (this->media_pipeline_state_ == AudioPipelineState::STOPPED) { + if (!media_playlist_.empty()) { + uint32_t timeout_ms = 0; + if (old_media_pipeline_state == AudioPipelineState::PLAYING) { + // Finished the current media file + if (!this->media_repeat_one_) { + // Pop item off the playlist if repeat is disabled + this->media_playlist_.pop_front(); + } + // Only delay starting playback if moving on the next playlist item or repeating the current item + timeout_ms = this->media_playlist_delay_ms_; + } + if (!this->media_playlist_.empty()) { + PlaylistItem playlist_item = this->media_playlist_.front(); + if (playlist_item.url.has_value()) { + this->media_pipeline_->start_url(playlist_item.url.value()); + } else if (playlist_item.file.has_value()) { + this->media_pipeline_->start_file(playlist_item.file.value()); + } + + if (timeout_ms > 0) { + // Pause pipeline internally to facilitate the delay between items + this->media_pipeline_->set_pause_state(true); + // Internally unpause the pipeline after the delay between playlist items, if the media player state is + // not paused. + this->set_timeout("next_media", timeout_ms, + [this]() { this->media_pipeline_->set_pause_state(this->is_paused_); }); + } + } + } else { +#ifdef USE_SPEAKER_MEDIA_PLAYER_ON_OFF + if (this->state != media_player::MEDIA_PLAYER_STATE_OFF) { + this->state = media_player::MEDIA_PLAYER_STATE_IDLE; + } +#else + this->state = media_player::MEDIA_PLAYER_STATE_IDLE; +#endif + } + } + } + } + + if (this->state != old_state) { + this->publish_state(); + ESP_LOGD(TAG, "State changed to %s", media_player::media_player_state_to_string(this->state)); + } +#ifdef USE_SPEAKER_MEDIA_PLAYER_ON_OFF + if (this->is_turn_off_ && (this->state == media_player::MEDIA_PLAYER_STATE_PAUSED || + this->state == media_player::MEDIA_PLAYER_STATE_IDLE)) { + this->is_turn_off_ = false; + if (this->state == media_player::MEDIA_PLAYER_STATE_PAUSED) { + this->state = media_player::MEDIA_PLAYER_STATE_IDLE; + this->publish_state(); + ESP_LOGD(TAG, "State changed to %s", media_player::media_player_state_to_string(this->state)); + } + this->state = media_player::MEDIA_PLAYER_STATE_OFF; + this->publish_state(); + ESP_LOGD(TAG, "State changed to %s", media_player::media_player_state_to_string(this->state)); + } +#endif +} + +void SpeakerMediaPlayer::play_file(audio::AudioFile *media_file, bool announcement, bool enqueue) { + if (!this->is_ready()) { + // Ignore any commands sent before the media player is setup + return; + } + + MediaCallCommand media_command; + + media_command.file = media_file; + if (this->single_pipeline_() || announcement) { + media_command.announce = true; + } else { + media_command.announce = false; + } + media_command.enqueue = enqueue; + xQueueSend(this->media_control_command_queue_, &media_command, portMAX_DELAY); +} + +void SpeakerMediaPlayer::control(const media_player::MediaPlayerCall &call) { + if (!this->is_ready()) { + // Ignore any commands sent before the media player is setup + return; + } + + MediaCallCommand media_command; + + auto ann = call.get_announcement(); + if (this->single_pipeline_() || (ann.has_value() && *ann)) { + media_command.announce = true; + } else { + media_command.announce = false; + } + + const auto &media_url = call.get_media_url(); + if (media_url.has_value()) { + media_command.url = + new std::string(*media_url); // Must be manually deleted after receiving media_command from a queue + + auto cmd = call.get_command(); + if (cmd.has_value()) { + if (*cmd == media_player::MEDIA_PLAYER_COMMAND_ENQUEUE) { + media_command.enqueue = true; + } + } + + xQueueSend(this->media_control_command_queue_, &media_command, portMAX_DELAY); + return; + } + + auto vol = call.get_volume(); + if (vol.has_value()) { + media_command.volume = vol; + // Wait 0 ticks for queue to be free, volume sets aren't that important! + xQueueSend(this->media_control_command_queue_, &media_command, 0); + return; + } + + auto cmd = call.get_command(); + if (cmd.has_value()) { + media_command.command = cmd; + TickType_t ticks_to_wait = portMAX_DELAY; + if ((*cmd == media_player::MEDIA_PLAYER_COMMAND_VOLUME_UP) || + (*cmd == media_player::MEDIA_PLAYER_COMMAND_VOLUME_DOWN)) { + ticks_to_wait = 0; // Wait 0 ticks for queue to be free, volume sets aren't that important! + } + xQueueSend(this->media_control_command_queue_, &media_command, ticks_to_wait); + return; + } +} + +media_player::MediaPlayerTraits SpeakerMediaPlayer::get_traits() { + auto traits = media_player::MediaPlayerTraits(); + if (!this->single_pipeline_()) { + traits.set_supports_pause(true); + } +#ifdef USE_SPEAKER_MEDIA_PLAYER_ON_OFF + traits.set_supports_turn_off_on(true); +#endif + + if (this->announcement_format_.has_value()) { + traits.get_supported_formats().push_back(this->announcement_format_.value()); + } + if (this->media_format_.has_value()) { + traits.get_supported_formats().push_back(this->media_format_.value()); + } else if (this->single_pipeline_() && this->announcement_format_.has_value()) { + // Only one pipeline is defined, so use the announcement format (if configured) for the default purpose + media_player::MediaPlayerSupportedFormat media_format = this->announcement_format_.value(); + media_format.purpose = media_player::MediaPlayerFormatPurpose::PURPOSE_DEFAULT; + traits.get_supported_formats().push_back(media_format); + } + + return traits; +}; + +void SpeakerMediaPlayer::save_volume_restore_state_() { + VolumeRestoreState volume_restore_state; + volume_restore_state.volume = this->volume; + volume_restore_state.is_muted = this->is_muted_; + this->pref_.save(&volume_restore_state); +} + +void SpeakerMediaPlayer::set_mute_state_(bool mute_state) { + if (this->media_speaker_ != nullptr) { + this->media_speaker_->set_mute_state(mute_state); + } + if (this->announcement_speaker_ != nullptr) { + this->announcement_speaker_->set_mute_state(mute_state); + } + + bool old_mute_state = this->is_muted_; + this->is_muted_ = mute_state; + + this->save_volume_restore_state_(); + + if (old_mute_state != mute_state) { + if (mute_state) { + this->defer([this]() { this->mute_trigger_.trigger(); }); + } else { + this->defer([this]() { this->unmute_trigger_.trigger(); }); + } + } +} + +void SpeakerMediaPlayer::set_volume_(float volume, bool publish) { + // Remap the volume to fit with in the configured limits + float bounded_volume = remap(volume, 0.0f, 1.0f, this->volume_min_, this->volume_max_); + + if (this->media_speaker_ != nullptr) { + this->media_speaker_->set_volume(bounded_volume); + } + + if (this->announcement_speaker_ != nullptr) { + this->announcement_speaker_->set_volume(bounded_volume); + } + + if (publish) { + this->volume = volume; + this->save_volume_restore_state_(); + } + + // Turn on the mute state if the volume is effectively zero, off otherwise + if (volume < 0.001f) { + this->set_mute_state_(true); + } else { + this->set_mute_state_(false); + } + + this->defer([this, volume]() { this->volume_trigger_.trigger(volume); }); +} + +} // namespace esphome::speaker + +#endif diff --git a/firmware/esphome/components/speaker/media_player/speaker_media_player.h b/firmware/esphome/components/speaker/media_player/speaker_media_player.h new file mode 100644 index 0000000..7d96de5 --- /dev/null +++ b/firmware/esphome/components/speaker/media_player/speaker_media_player.h @@ -0,0 +1,180 @@ +#pragma once + +#ifdef USE_ESP32 + +#include "audio_pipeline.h" + +#include "esphome/components/audio/audio.h" +#include "esphome/components/media_player/media_player.h" +#include "esphome/components/speaker/speaker.h" + +#include "esphome/core/automation.h" +#include "esphome/core/component.h" +#include "esphome/core/defines.h" +#include "esphome/core/preferences.h" + +#ifdef USE_OTA_STATE_LISTENER +#include "esphome/components/ota/ota_backend.h" +#endif + +#include +#include +#include + +namespace esphome::speaker { + +struct MediaCallCommand { + optional command; + optional volume; + optional announce; + optional url; // Must be manually deleted after receiving this struct from a queue + optional file; + optional enqueue; +}; + +struct PlaylistItem { + optional url; + optional file; +}; + +struct VolumeRestoreState { + float volume; + bool is_muted; +}; + +class SpeakerMediaPlayer final : public Component, + public media_player::MediaPlayer +#ifdef USE_OTA_STATE_LISTENER + , + public ota::OTAGlobalStateListener +#endif +{ + public: +#ifdef OPENATHAN_QURAN + AudioPipelineState announcement_state() const { return announcement_pipeline_state_; } + bool playback_stopped() const { + return this->is_ready() && this->media_control_command_queue_ != nullptr && + uxQueueMessagesWaiting(this->media_control_command_queue_) == 0 && + this->announcement_pipeline_state_ == AudioPipelineState::STOPPED && + this->media_pipeline_state_ == AudioPipelineState::STOPPED; + } +#endif + float get_setup_priority() const override { return esphome::setup_priority::PROCESSOR; } + void setup() override; + void loop() override; + +#ifdef USE_OTA_STATE_LISTENER + void on_ota_global_state(ota::OTAState state, float progress, uint8_t error, ota::OTAComponent *comp) override; +#endif + + // MediaPlayer implementations + media_player::MediaPlayerTraits get_traits() override; + bool is_muted() const override { return this->is_muted_; } + + void set_buffer_size(size_t buffer_size) { this->buffer_size_ = buffer_size; } + void set_task_stack_in_psram(bool task_stack_in_psram) { this->task_stack_in_psram_ = task_stack_in_psram; } + + // Percentage to increase or decrease the volume for volume up or volume down commands + void set_volume_increment(float volume_increment) { this->volume_increment_ = volume_increment; } + + // Volume used initially on first boot when no volume had been previously saved + void set_volume_initial(float volume_initial) { this->volume_initial_ = volume_initial; } + + void set_volume_max(float volume_max) { this->volume_max_ = volume_max; } + void set_volume_min(float volume_min) { this->volume_min_ = volume_min; } + + void set_announcement_speaker(Speaker *announcement_speaker) { this->announcement_speaker_ = announcement_speaker; } + void set_announcement_format(const media_player::MediaPlayerSupportedFormat &announcement_format) { + this->announcement_format_ = announcement_format; + } + void set_media_speaker(Speaker *media_speaker) { this->media_speaker_ = media_speaker; } + void set_media_format(const media_player::MediaPlayerSupportedFormat &media_format) { + this->media_format_ = media_format; + } + + Trigger<> *get_mute_trigger() { return &this->mute_trigger_; } + Trigger<> *get_unmute_trigger() { return &this->unmute_trigger_; } + Trigger *get_volume_trigger() { return &this->volume_trigger_; } + + void play_file(audio::AudioFile *media_file, bool announcement, bool enqueue); + + void set_playlist_delay_ms(AudioPipelineType pipeline_type, uint32_t delay_ms); + + protected: + // Receives commands from HA or from the voice assistant component + // Sends commands to the media_control_commanda_queue_ + void control(const media_player::MediaPlayerCall &call) override; + + /// @brief Updates this->volume and saves volume/mute state to flash for restortation if publish is true. + void set_volume_(float volume, bool publish = true); + + /// @brief Sets the mute state. Restores previous volume if unmuting. Always saves volume/mute state to flash for + /// restoration. + /// @param mute_state If true, audio will be muted. If false, audio will be unmuted + void set_mute_state_(bool mute_state); + + /// @brief Saves the current volume and mute state to the flash for restoration. + void save_volume_restore_state_(); + + /// Returns true if the media player has only the announcement pipeline defined, false if both the announcement and + /// media pipelines are defined. + inline bool single_pipeline_() { return (this->media_speaker_ == nullptr); } + + /// Stops the media pipeline and polls until stopped to unpause it, avoiding an audible glitch. + void stop_and_unpause_media_(); + + // Processes commands from media_control_command_queue_. + void watch_media_commands_(); + + std::unique_ptr announcement_pipeline_; + std::unique_ptr media_pipeline_; + Speaker *media_speaker_{nullptr}; + Speaker *announcement_speaker_{nullptr}; + + optional media_format_; + AudioPipelineState media_pipeline_state_{AudioPipelineState::STOPPED}; + bool media_repeat_one_{false}; + uint32_t media_playlist_delay_ms_{0}; + + optional announcement_format_; + AudioPipelineState announcement_pipeline_state_{AudioPipelineState::STOPPED}; + bool announcement_repeat_one_{false}; + uint32_t announcement_playlist_delay_ms_{0}; + + QueueHandle_t media_control_command_queue_; + + std::deque announcement_playlist_; + std::deque media_playlist_; + + size_t buffer_size_; + + bool task_stack_in_psram_; + + bool is_paused_{false}; + bool is_muted_{false}; +#ifdef USE_SPEAKER_MEDIA_PLAYER_ON_OFF + bool is_turn_off_{false}; +#endif + uint8_t unpause_media_remaining_{0}; + uint8_t unpause_announcement_remaining_{0}; + + // The amount to change the volume on volume up/down commands + float volume_increment_; + + // The initial volume used by Setup when no previous volume was saved + float volume_initial_; + + float volume_max_; + float volume_min_; + + // Used to save volume/mute state for restoration on reboot + ESPPreferenceObject pref_; + + Trigger<> mute_trigger_; + Trigger<> unmute_trigger_; + Trigger volume_trigger_; +}; + +} // namespace esphome::speaker + +#endif diff --git a/firmware/esphome/components/speaker/speaker.h b/firmware/esphome/components/speaker/speaker.h new file mode 100644 index 0000000..c89b6c5 --- /dev/null +++ b/firmware/esphome/components/speaker/speaker.h @@ -0,0 +1,125 @@ +#pragma once + +#include +#include +#include + +#ifdef USE_ESP32 +#include +#endif + +#include "esphome/core/defines.h" +#include "esphome/core/helpers.h" + +#include "esphome/components/audio/audio.h" +#ifdef USE_AUDIO_DAC +#include "esphome/components/audio_dac/audio_dac.h" +#endif + +namespace esphome::speaker { + +enum State : uint8_t { + STATE_STOPPED = 0, + STATE_STARTING, + STATE_RUNNING, + STATE_STOPPING, +}; + +class Speaker { + public: +#ifdef USE_ESP32 + /// @brief Plays the provided audio data. + /// If the speaker component doesn't implement this method, it falls back to the play method without this parameter. + /// @param data Audio data in the format specified by ``set_audio_stream_info`` method. + /// @param length The length of the audio data in bytes. + /// @param ticks_to_wait The FreeRTOS ticks to wait before writing as much data as possible to the ring buffer. + /// @return The number of bytes that were actually written to the speaker's internal buffer. + virtual size_t play(const uint8_t *data, size_t length, TickType_t ticks_to_wait) { + return this->play(data, length); + }; +#endif + + /// @brief Plays the provided audio data. + /// If the audio stream is not the default defined in "esphome/core/audio.h" and the speaker component implements it, + /// then this should be called after calling ``set_audio_stream_info``. + /// @param data Audio data in the format specified by ``set_audio_stream_info`` method. + /// @param length The length of the audio data in bytes. + /// @return The number of bytes that were actually written to the speaker's internal buffer. + virtual size_t play(const uint8_t *data, size_t length) = 0; + + size_t play(const std::vector &data) { return this->play(data.data(), data.size()); } + + virtual void start() = 0; + virtual void stop() = 0; + // In compare between *STOP()* and *FINISH()*; *FINISH()* will stop after emptying the play buffer, + // while *STOP()* will break directly. + // When finish() is not implemented on the platform component it should just do a normal stop. + virtual void finish() { this->stop(); } + + // Pauses processing incoming audio. Needs to be implemented specifically per speaker component + virtual void set_pause_state(bool pause_state) {} + virtual bool get_pause_state() const { return false; } + + virtual bool has_buffered_data() const = 0; + + bool is_running() const { return this->state_ == STATE_RUNNING; } + bool is_stopped() const { return this->state_ == STATE_STOPPED; } + + // Volume control is handled by a configured audio dac component. Individual speaker components can + // override and implement in software if an audio dac isn't available. + virtual void set_volume(float volume) { + this->volume_ = volume; +#ifdef USE_AUDIO_DAC + if (this->audio_dac_ != nullptr) { + this->audio_dac_->set_volume(volume); + } +#endif + }; + virtual float get_volume() { return this->volume_; } + + virtual void set_mute_state(bool mute_state) { + this->mute_state_ = mute_state; +#ifdef USE_AUDIO_DAC + if (this->audio_dac_) { + if (mute_state) { + this->audio_dac_->set_mute_on(); + } else { + this->audio_dac_->set_mute_off(); + } + } +#endif + } + virtual bool get_mute_state() { return this->mute_state_; } + +#ifdef USE_AUDIO_DAC + void set_audio_dac(audio_dac::AudioDac *audio_dac) { this->audio_dac_ = audio_dac; } +#endif + + void set_audio_stream_info(const audio::AudioStreamInfo &audio_stream_info) { + this->audio_stream_info_ = audio_stream_info; + } + + audio::AudioStreamInfo &get_audio_stream_info() { return this->audio_stream_info_; } + + /// Callback function for sending the duration of the audio written to the speaker since the last callback. + /// Parameters: + /// - Frames played + /// - System time in microseconds when the frames were written to the DAC + template void add_audio_output_callback(F &&callback) { + this->audio_output_callback_.add(std::forward(callback)); + } + + protected: + State state_{STATE_STOPPED}; + audio::AudioStreamInfo audio_stream_info_; + float volume_{1.0f}; + bool mute_state_{false}; + +#ifdef USE_AUDIO_DAC + audio_dac::AudioDac *audio_dac_{nullptr}; +#endif + + CallbackManager audio_output_callback_{}; +}; + +} // namespace esphome::speaker diff --git a/firmware/esphome/components/waveshare_box_audio/audio.h b/firmware/esphome/components/waveshare_box_audio/audio.h index 3dc09d8..6051498 100644 --- a/firmware/esphome/components/waveshare_box_audio/audio.h +++ b/firmware/esphome/components/waveshare_box_audio/audio.h @@ -29,6 +29,11 @@ class Audio : public Component, public ::openathan::Playback { bool playing() const override { return playback_ && playback_->playing(); } bool start(::openathan::Track track) override { return ready() && playback_->start(track); } void stop() override { if (playback_) playback_->stop(); } + bool stream_supported() const override { return playback_ && playback_->stream_supported(); } + uint32_t playback_epoch() const override { return playback_ ? playback_->playback_epoch() : 0; } + ::openathan::PlaybackSource source() const override { return playback_ ? playback_->source() : ::openathan::PlaybackSource::NONE; } + ::openathan::StreamState stream_state() const override { return playback_ ? playback_->stream_state() : ::openathan::StreamState::IDLE; } + bool start_stream(const std::string &url) override { return ready() && playback_->start_stream(url); } std::optional volume_percent() const override { return healthy_() ? playback_->volume_percent() : std::nullopt; } diff --git a/firmware/esphome/packages/product.yaml b/firmware/esphome/packages/product.yaml index 9248381..17fc660 100644 --- a/firmware/esphome/packages/product.yaml +++ b/firmware/esphome/packages/product.yaml @@ -10,7 +10,7 @@ external_components: - source: type: local path: ../components - components: [openathan_audio, openathan_device, improv_serial] + components: [audio, speaker, openathan_audio, openathan_device, improv_serial] logger: hardware_uart: USB_SERIAL_JTAG diff --git a/lib/openathan-core/include/openathan/playback.h b/lib/openathan-core/include/openathan/playback.h index 0f86de8..efe9aba 100644 --- a/lib/openathan-core/include/openathan/playback.h +++ b/lib/openathan-core/include/openathan/playback.h @@ -1,8 +1,11 @@ #pragma once #include "audio_image.h" #include +#include namespace openathan { +enum class PlaybackSource { NONE, ATHAN, QURAN }; +enum class StreamState { IDLE, LOADING, PLAYING, ERROR }; class Playback { public: virtual ~Playback() = default; @@ -11,6 +14,12 @@ class Playback { // Replace current audio; true means the request was accepted, not audibly played. virtual bool start(Track track) = 0; virtual void stop() = 0; + // Optional streaming capability. Epoch changes revoke outstanding requests. + virtual bool stream_supported() const { return false; } + virtual uint32_t playback_epoch() const { return 0; } + virtual PlaybackSource source() const { return playing() ? PlaybackSource::ATHAN : PlaybackSource::NONE; } + virtual StreamState stream_state() const { return StreamState::IDLE; } + virtual bool start_stream(const std::string &) { return false; } // Optional volume capability used by runtime settings, never by calculations. // Readback must report the applied control level, not a queued request. virtual std::optional volume_percent() const { return {}; } diff --git a/tests/audio_adapter_tests.cpp b/tests/audio_adapter_tests.cpp index c3fb875..1dc6623 100644 --- a/tests/audio_adapter_tests.cpp +++ b/tests/audio_adapter_tests.cpp @@ -1,7 +1,9 @@ #include "partition_audio.h" +#include "esphome/core/hal.h" #include #include #include +#include #include static bool present = true; @@ -9,7 +11,7 @@ static bool map_fails = false; static unsigned maps = 0, unmaps = 0; static esp_partition_t partition{openathan::AUDIO_PARTITION_SIZE}; static std::vector bytes; -static void check(bool condition) { if (!condition) std::abort(); } +#define check(condition) do { if (!(condition)) { std::cerr << __LINE__ << ": " << #condition << '\n'; std::abort(); } } while (false) const esp_partition_t *esp_partition_find_first(int, int, const char *) { return present ? &partition : nullptr; } int esp_partition_mmap(const esp_partition_t *, size_t, size_t, int, const void **out, esp_partition_mmap_handle_t *handle) { @@ -56,15 +58,44 @@ int main(int argc, char **argv) { check(!valid.request_volume_percent(101)); check(valid.request_volume_percent(0) && valid.volume_percent() == 0 && player.is_muted()); check(valid.request_volume_percent(70) && valid.volume_percent() == 70 && !player.is_muted()); - check(valid.play(Track::NORMAL) && valid.play(Track::FAJR)); + check(valid.play(Track::NORMAL)); + check(player.queued.empty() && valid.source() == openathan::PlaybackSource::ATHAN); + player.drain_commands(); valid.loop(); + check(player.queued.size() == 1); + check(valid.play(Track::FAJR)); + player.drain_commands(); valid.loop(); check(!valid.play(static_cast(3)) && !valid.play(static_cast(0))); check(player.queued.size() == 2 && player.queued[0] != player.queued[1]); check(player.queued[0]->data == bytes.data() + 4096 && player.queued[0]->length > 0); check(player.queued[1]->data > player.queued[0]->data && player.queued[1]->length > 0); - check(valid.play(Track::NORMAL) && player.queued[0] == player.queued[2] && unmaps == 1); + check(valid.play(Track::NORMAL)); + player.drain_commands(); valid.loop(); + check(player.queued[0] == player.queued[2] && unmaps == 1); player.commands.clear(); check(valid.start(Track::FAJR)); - check(player.commands == std::vector{"stop media", "stop announcement", "play"}); + check(player.commands == std::vector({"stop media", "stop announcement"})); + player.drain_commands(); valid.loop(); + check(player.commands == std::vector({"stop media", "stop announcement", "play"})); + auto epoch = valid.playback_epoch(); + check(!valid.start_stream("http://cdn.mp3quran.net/test/001.mp3")); + check(valid.playback_epoch() == epoch); + check(valid.start_stream("https://cdn.mp3quran.net/audio/test/001.mp3")); + check(valid.source() == openathan::PlaybackSource::QURAN); + check(valid.playback_epoch() != epoch); + player.drain_commands(); valid.loop(); + player.pipeline_state = esphome::speaker::AudioPipelineState::ERROR_DECODING; + valid.loop(); + check(valid.stream_state() == openathan::StreamState::ERROR); + player.pipeline_state = esphome::speaker::AudioPipelineState::STOPPED; + check(valid.start_stream("https://cdn.mp3quran.net/audio/test/001.mp3")); + esphome::mock_millis += 30000; + valid.loop(); + check(valid.stream_state() == openathan::StreamState::ERROR && valid.source() == openathan::PlaybackSource::NONE); + epoch = valid.playback_epoch(); + check(valid.start(Track::NORMAL)); + check(valid.source() == openathan::PlaybackSource::ATHAN && valid.playback_epoch() != epoch); + valid.stop(); + check(valid.source() == openathan::PlaybackSource::NONE); player.state = esphome::media_player::MEDIA_PLAYER_STATE_ANNOUNCING; check(valid.playing()); player.initialized = false; @@ -72,4 +103,91 @@ int main(int argc, char **argv) { check(!valid.ready() && !valid.start(Track::NORMAL)); valid.stop(); check(player.commands.empty()); + + using Source = openathan::PlaybackSource; + using Stream = openathan::StreamState; + using Pipeline = esphome::speaker::AudioPipelineState; + using State = esphome::media_player::MediaPlayerState; + const std::string url = "https://cdn.mp3quran.net/audio/test/001.mp3"; + esphome::speaker::SpeakerMediaPlayer handoff; + PartitionAudio audio; audio.set_player(&handoff); audio.setup(); + // Outgoing Quran -> stopped -> new Athan must retain its reservation. Drain + // one command at a time and independently delay each asynchronous pipeline. + handoff.pipeline_state = Pipeline::PLAYING; handoff.state = State::MEDIA_PLAYER_STATE_ANNOUNCING; + check(audio.start(Track::NORMAL)); epoch = audio.playback_epoch(); audio.loop(); + check(audio.source() == Source::ATHAN && handoff.queued.empty()); + handoff.pipeline_state = Pipeline::STOPPED; handoff.state = State::MEDIA_PLAYER_STATE_IDLE; + handoff.drain_command(); audio.loop(); check(handoff.queued.empty()); + handoff.drain_command(); handoff.media_pipeline_state = Pipeline::PLAYING; + audio.loop(); check(handoff.queued.empty() && audio.source() == Source::ATHAN); + handoff.media_pipeline_state = Pipeline::STOPPED; audio.loop(); + check(handoff.queued.size() == 1 && audio.source() == Source::ATHAN); + audio.loop(); check(audio.source() == Source::ATHAN); // New start has not played yet. + handoff.drain_commands(); handoff.pipeline_state = Pipeline::PLAYING; + handoff.state = State::MEDIA_PLAYER_STATE_ANNOUNCING; audio.loop(); + check(audio.source() == Source::ATHAN && audio.playback_epoch() == epoch); + handoff.pipeline_state = Pipeline::STOPPED; handoff.state = State::MEDIA_PLAYER_STATE_IDLE; + audio.loop(); check(audio.source() == Source::NONE); + + // Direct play() and rapid stored/stream replacements use the same boundary; + // the latest accepted recording is the only start submitted. + handoff.commands.clear(); handoff.queued.clear(); + check(audio.start_stream(url)); check(audio.play(Track::NORMAL)); check(audio.play(Track::FAJR)); + epoch = audio.playback_epoch(); + check(!audio.play(static_cast(3)) && !audio.start_stream("http://cdn.mp3quran.net/a.mp3")); + check(audio.playback_epoch() == epoch && audio.source() == Source::ATHAN); + audio.loop(); check(handoff.queued.empty()); + handoff.drain_commands(); audio.loop(); + check(handoff.queued.size() == 1 && handoff.queued[0]->data > bytes.data() + 4096); + check(handoff.commands.back() == "play"); + handoff.drain_commands(); + handoff.commands.clear(); check(audio.play(Track::NORMAL)); check(audio.start_stream(url)); + handoff.drain_commands(); audio.loop(); check(handoff.commands.back() == url); + + const auto submitted_files = handoff.queued.size(); + check(audio.start(Track::NORMAL)); epoch = audio.playback_epoch(); audio.stop(); + check(audio.source() == Source::NONE && audio.playback_epoch() != epoch); + handoff.drain_commands(); audio.loop(); check(handoff.queued.size() == submitted_files); + + // Stop revokes a pending replacement before it can submit a URL; clearing + // the old speaker state or recovering a failed speaker cannot revive it. + check(audio.start_stream(url)); epoch = audio.playback_epoch(); audio.stop(); + check(audio.source() == Source::NONE && audio.playback_epoch() != epoch); + handoff.commands.clear(); handoff.drain_commands(); audio.loop(); check(handoff.commands.empty()); + check(audio.start_stream(url)); handoff.initialized = false; audio.loop(); + check(audio.source() == Source::NONE && audio.stream_state() == Stream::ERROR); + handoff.initialized = true; handoff.commands.clear(); handoff.drain_commands(); + audio.loop(); check(handoff.commands.empty()); + check(audio.start(Track::FAJR)); handoff.initialized = false; audio.loop(); + check(audio.source() == Source::NONE); + handoff.initialized = true; handoff.commands.clear(); handoff.drain_commands(); + audio.loop(); check(handoff.commands.empty() && handoff.queued.size() == submitted_files); + + // The startup deadline includes time waiting for Stop, including when the + // outgoing session reports PLAYING. Submission does not reset that deadline. + check(audio.start_stream(url)); handoff.pipeline_state = Pipeline::PLAYING; + esphome::mock_millis += 29999; audio.loop(); check(audio.stream_state() == Stream::LOADING); + esphome::mock_millis += 1; audio.loop(); + check(audio.stream_state() == Stream::ERROR && audio.source() == Source::NONE); + handoff.pipeline_state = Pipeline::STOPPED; handoff.drain_commands(); + check(audio.start_stream(url)); esphome::mock_millis += 25000; + handoff.drain_commands(); audio.loop(); + esphome::mock_millis += 4999; audio.loop(); check(audio.stream_state() == Stream::LOADING); + esphome::mock_millis += 1; audio.loop(); check(audio.stream_state() == Stream::ERROR); + + // Old reader/decoder errors are ignored while stopping. Errors observed + // after submission end the new session and cannot restart it. + for (auto failure : {Pipeline::ERROR_READING, Pipeline::ERROR_DECODING}) { + check(audio.start_stream(url)); handoff.pipeline_state = failure; audio.loop(); + check(audio.stream_state() == Stream::LOADING && audio.source() == Source::QURAN); + handoff.drain_commands(); audio.loop(); check(audio.stream_state() == Stream::LOADING); + handoff.pipeline_state = Pipeline::STOPPED; audio.loop(); + handoff.pipeline_state = failure; audio.loop(); + check(audio.stream_state() == Stream::ERROR && audio.source() == Source::NONE); + handoff.commands.clear(); handoff.drain_commands(); handoff.pipeline_state = Pipeline::STOPPED; + audio.loop(); check(handoff.commands.empty()); + check(audio.start(Track::NORMAL)); handoff.drain_commands(); audio.loop(); + handoff.pipeline_state = failure; audio.loop(); check(audio.source() == Source::NONE); + handoff.pipeline_state = Pipeline::STOPPED; handoff.drain_commands(); + } } diff --git a/tests/device_shutdown_tests.cpp b/tests/device_shutdown_tests.cpp index b24eeb7..a921ec6 100644 --- a/tests/device_shutdown_tests.cpp +++ b/tests/device_shutdown_tests.cpp @@ -14,6 +14,10 @@ void Device::loop() { void Device::on_wifi_scan_results(const wifi::wifi_scan_vector_t&) {} void Device::on_wifi_connect_state(StringRef, std::span) {} Upgrade::~Upgrade() = default; +void Quran::shutdown() {} +void Quran::snapshot(JsonObject) {} +void Quran::catalog_snapshot(JsonObject) {} +int Quran::action(const std::string&, JsonObjectConst, std::string&) { return 503; } void Upgrade::snapshot(JsonObject) {} int Upgrade::action(const std::string& action, JsonObjectConst, std::string&) { if (action == "arm_safe_worker") { cancel_ = false; safe_ = true; return 200; } diff --git a/tests/device_ui.test.cjs b/tests/device_ui.test.cjs index d522d48..8ed170c 100644 --- a/tests/device_ui.test.cjs +++ b/tests/device_ui.test.cjs @@ -12,6 +12,10 @@ function initial(){ const times=['05:30','06:45','12:30','15:45','18:20','19:40'].map((time,i)=>({name:['Fajr','Sunrise','Dhuhr','Asr','Maghrib','Isha'][i],local:'2026-09-25 '+time,utc:Date.parse('2026-09-25T'+time+':00Z')/1000})); return {time_format:{schema:1,revision:1,hours:24,application:'applied'},schema:1,revision:1,setup:'active',application:'applied',automatic_ready:true,clock_ready:true,local_date:'2026-09-25',scheduler_fault:'none',playing:false,wifi_connected:true,hostname:'openathan-test.local',settings:{latitude:0,longitude:0,timezone:'UTC',timezone_rules:rules,method:'muslim_world_league',asr_method:'standard',high_latitude:'auto',volume:70,offsets:{fajr:0,sunrise:0,dhuhr:0,asr:0,maghrib:0,isha:0},enabled:{fajr:true,dhuhr:true,asr:true,maghrib:true,isha:true}},schedule:{state:'ready',times,conflicts:[]},next:{day:20721,prayer:2,name:'Asr',local:times[3].local,utc:times[3].utc}}; } +function stopQuran(device){ + device.playing=false;if(device.quran||'playback_source' in device)device.playback_source='none'; + if(device.quran){device.quran.state='idle';if(device.quran.playback_token){const [boot,epoch]=device.quran.playback_token.split(':');device.quran.playback_token=boot+':'+(Number(epoch)+1);}} +} async function fixture(){ const state={device:initial(),lightMutations:0,lightDrop:false,lightFailRead:false,mutations:0,drop:false,failRead:false,posts:[],authenticated:0, now:100, challenges:[],cnonces:new Set()}; @@ -91,7 +95,7 @@ async function fixture(){ if(req.url==='/api/skip'){ if(payload.expected_revision!==state.device.revision||payload.occurrence.utc!==state.device.next.utc){send(409,{error:'Occurrence changed; reload status'});return;} state.device.skip={day:payload.occurrence.day,prayer:payload.occurrence.prayer}; - }else if(req.url==='/api/cancel-skip'){delete state.device.skip;}else if(req.url==='/api/stop'&&!state.stopStillPlaying){state.device.playing=false;} + }else if(req.url==='/api/cancel-skip'){delete state.device.skip;}else if(req.url==='/api/stop'&&!state.stopStillPlaying){stopQuran(state.device);} if(state.actionDrop&&['/api/skip','/api/cancel-skip','/api/stop'].includes(req.url)){state.actionDrop=false;res.writeHead(200);res.end('{');return;} send(200,state.device); }); @@ -136,7 +140,349 @@ async function loseSkipResponseAndRestoreContact(f,page){ await page.locator('#connection-banner').waitFor({state:'hidden'}); assert.ok(await page.locator('#connection-banner').isHidden());assert.ok(await page.locator('#skip').isDisabled()); } +function quranReciters(id=1,name='Alpha',next=null,total=1){ + return {reciters:[{id,name}],surahs:[{id:18,name:'Al-Kahf'}],next_offset:next,total}; +} +async function requestStarted(signal){ + let timer; + try{await Promise.race([signal.promise,new Promise((_,reject)=>{timer=setTimeout(()=>reject(new Error('Device request did not start')),10000);})]);} + finally{clearTimeout(timer);} +} +async function quranControls(page){ + await page.locator('[data-view=quran]').click();await page.locator('#quran-load').click(); + await page.waitForFunction(()=>!document.getElementById('quran-reciter').disabled); + await page.locator('#quran-reciter').selectOption('1'); + await page.waitForFunction(()=>!document.getElementById('quran-surah').disabled); + await page.locator('#quran-surah').selectOption('18'); +} +function admitQuran(device,payload){ + if(payload.expected_playback_token!==device.quran.playback_token)return false; + stopQuran(device);Object.assign(device.quran,{state:'loading',reciter:payload.reciter,edition:payload.edition,surah:payload.surah});return true; +} +async function mockQuranCatalog(page){ + await page.route('**/api/quran/catalog',route=>route.fulfill({json:{state:'ready',request:1, + data:route.request().postDataJSON().kind==='editions'?{editions:[{id:9,name:'Edition',surahs:[18]}]}:quranReciters()}})); +} for(const browserName of (process.env.OPENATHAN_TEST_BROWSERS||'chromium').split(',')){ + for(const stopView of ['quran','settings','today'])test(`${browserName}: ${stopView} Stop invalidates a Quran Play not yet delivered`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:stopView==='quran'?'playing':'idle'}; + f.state.device.playing=stopView==='quran';f.state.device.playback_source=f.state.device.playing?'quran':'none'; + const o=await open(f,browserName,{},true),{page}=o,gate=deferred(),started=deferred();let attempts=0,admitted=0,stops=0; + try{ + await mockQuranCatalog(page); + await page.route('**/api/quran/play',async route=>{attempts++;started.resolve();await gate.promise; + if(admitQuran(f.state.device,route.request().postDataJSON())){admitted++;await route.fulfill({json:structuredClone(f.state.device)});} + else await route.fulfill({status:409,json:{error:'Playback changed; check status'}}); + }); + await page.route('**/api/stop',route=>{stops++;stopQuran(f.state.device);return route.fulfill({json:structuredClone(f.state.device)});}); + await quranControls(page);await page.locator('#quran-play').click();await requestStarted(started); + if(stopView!=='quran')await page.locator(`[data-view=${stopView}]`).click(); + const id=stopView==='quran'?'quran-stop':stopView==='settings'?'settings-stop':'stop'; + assert.ok(await page.locator('#'+id).isVisible());await page.locator('#'+id).click(); + const feedback=stopView==='quran'?'quran-feedback':stopView==='settings'?'settings-playback-feedback':'action-feedback'; + await page.waitForFunction(id=>document.getElementById(id).textContent==='Playback stopped',feedback); + gate.resolve();await requestsFinished(page); + await page.locator('[data-view=quran]').click(); + assert.equal(attempts,1);assert.equal(admitted,0);assert.equal(stops,1);assert.equal(f.state.device.quran.state,'idle'); + assert.equal(await page.locator('#quran-feedback').textContent(),'Playback stopped');assert.ok(await page.locator('#quran-play').isEnabled()); + assert.deepEqual(JSON.parse(await page.evaluate(()=>localStorage.getItem('openathan-quran'))),{reciter:'1',edition:'9',surah:'18'}); + }finally{gate.resolve();await o.close();} + }); + test(`${browserName}: Quran Play survives an idle five-second poll and a late pre-acknowledgement read`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle'}; + const o=await open(f,browserName,{},true),{page}=o,gate=deferred(),started=deferred(),oldRead=deferred(),oldStarted=deferred();let idlePolls=0,delayRead=false; + try{ + await mockQuranCatalog(page); + await page.route('**/api/quran/play',async route=>{started.resolve();await gate.promise;assert.ok(admitQuran(f.state.device,route.request().postDataJSON()));await route.fulfill({json:structuredClone(f.state.device)});}); + await page.route('**/api/status',async route=>{const raw=structuredClone(f.state.device);if(raw.quran.state==='idle')idlePolls++; + if(delayRead){delayRead=false;oldStarted.resolve();await oldRead.promise;}await route.fulfill({json:raw}); + }); + await quranControls(page);await page.locator('#quran-play').click();await requestStarted(started); + const poll=page.waitForResponse(r=>new URL(r.url()).pathname==='/api/status');await page.clock.runFor(5000);await (await poll).finished(); + assert.equal(idlePolls,1);assert.ok(await page.locator('#quran-stop').isVisible());assert.ok(await page.locator('#quran-play').isDisabled()); + delayRead=true;await page.clock.runFor(5000);await requestStarted(oldStarted); + gate.resolve();await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent.includes('Starting Quran')); + oldRead.resolve();await requestsFinished(page); + assert.equal(f.state.device.quran.state,'loading');assert.ok(await page.locator('#quran-stop').isVisible());assert.ok(await page.locator('#quran-play').isDisabled()); + assert.match(await page.locator('#quran-feedback').textContent(),/Starting Quran/); + }finally{gate.resolve();oldRead.resolve();await o.close();} + }); + for(const readFails of [false,true])test(`${browserName}: uncertain Quran Play ${readFails?'reconnects':'reads idle'} without allowing another Play`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle'}; + const o=await open(f,browserName,{},true),{page}=o;let plays=0;let original; + try{ + await mockQuranCatalog(page); + await page.route('**/api/quran/play',route=>{plays++;original=route.request().postDataJSON();f.state.failRead=readFails; + return readFails?route.abort('timedout'):route.fulfill({status:200,contentType:'application/json',body:'{'}); + }); + await quranControls(page);await page.locator('#quran-play').click(); + await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent.includes('Play unconfirmed')); + assert.ok(await page.locator('#quran-stop').isVisible());assert.ok(await page.locator('#quran-play').isDisabled()); + if(readFails){f.state.failRead=false;await refresh(page);await page.locator('[data-view=quran]').click();await page.locator('#quran-stop').waitFor({state:'visible'});} + assert.ok(await page.locator('#quran-play').isDisabled()); + f.state.actionDrop=true;await page.locator('#quran-stop').click(); + await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent==='Playback stopped · confirmed after readback'); + assert.equal(plays,1);assert.equal(admitQuran(f.state.device,original),false);assert.ok(await page.locator('#quran-play').isEnabled()); + }finally{await o.close();} + }); + for(const omitted of ['playing','quran'])test(`${browserName}: acknowledged Quran Play keeps controls pending after readback omits ${omitted}`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle'}; + const o=await open(f,browserName,{},true),{page}=o;let plays=0; + try{ + await mockQuranCatalog(page);await quranControls(page); + await page.route('**/api/quran/play',route=>{plays++;assert.ok(admitQuran(f.state.device,route.request().postDataJSON()));return route.fulfill({json:structuredClone(f.state.device)});}); + await page.route('**/api/status',route=>{const raw=structuredClone(f.state.device);delete raw[omitted];return route.fulfill({json:raw});},{times:1}); + await page.locator('#quran-play').click();await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent.includes('Play unconfirmed')); + assert.ok(await page.locator('#quran-stop').isVisible());assert.ok(await page.locator('#quran-play').isDisabled());assert.equal(plays,1); + await refresh(page);await page.locator('[data-view=quran]').click();await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent.includes('Starting Quran')); + assert.equal(plays,1);assert.equal(f.state.device.quran.state,'loading');await page.locator('#quran-stop').click(); + await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent==='Playback stopped');assert.equal(f.state.device.quran.state,'idle'); + }finally{await o.close();} + }); + test(`${browserName}: Quran Stop cannot confirm idle while the pending Play token remains valid`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle'}; + const o=await open(f,browserName,{},true),{page}=o;let stops=0; + try{ + await mockQuranCatalog(page); + await page.route('**/api/quran/play',route=>route.fulfill({status:200,contentType:'application/json',body:'{'})); + await page.route('**/api/stop',route=>{stops++;return route.fulfill({status:200,contentType:'application/json',body:'{'});}); + await quranControls(page);await page.locator('#quran-play').click();await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent.includes('Play unconfirmed')); + await page.locator('#quran-stop').click();await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent.includes('Stop unconfirmed')); + assert.ok(await page.locator('#quran-stop').isVisible());assert.ok(await page.locator('#quran-play').isDisabled()); + await page.clock.runFor(5000);await requestsFinished(page);assert.doesNotMatch(await page.locator('#quran-feedback').textContent(),/Playback stopped/); + stopQuran(f.state.device);await page.clock.runFor(5000);await requestsFinished(page); + await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent.includes('Playback stopped'));assert.equal(stops,1); + }finally{await o.close();} + }); + test(`${browserName}: obsolete Quran Play acknowledgement does not stop a newly due Athan`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle'}; + const o=await open(f,browserName,{},true),{page}=o,gate=deferred(),started=deferred();let stops=0; + try{ + await mockQuranCatalog(page); + await page.route('**/api/quran/play',async route=>{assert.ok(admitQuran(f.state.device,route.request().postDataJSON()));const raw=structuredClone(f.state.device);started.resolve();await gate.promise;await route.fulfill({json:raw});}); + await page.route('**/api/stop',route=>{stops++;stopQuran(f.state.device);return route.fulfill({json:structuredClone(f.state.device)});}); + await quranControls(page);await page.locator('#quran-play').click();await requestStarted(started);await page.locator('#quran-stop').click(); + await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent==='Playback stopped'); + stopQuran(f.state.device);f.state.device.playing=true;f.state.device.playback_source='athan'; + gate.resolve();await requestsFinished(page);await page.clock.runFor(5000);await requestsFinished(page); + await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent.includes('Athan is playing')); + assert.equal(stops,1);assert.equal(f.state.device.playback_source,'athan');assert.equal(f.state.device.playing,true);assert.ok(await page.locator('#quran-play').isDisabled()); + }finally{gate.resolve();await o.close();} + }); + test(`${browserName}: Quran clients without a playback token must reload before Play`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,state:'idle'};const o=await open(f,browserName),{page}=o; + try{await mockQuranCatalog(page);await quranControls(page);assert.ok(await page.locator('#quran-play').isDisabled());assert.match(await page.locator('#quran-feedback').textContent(),/Reload the page/);assert.equal(f.state.posts.filter(p=>p.url==='/api/quran/play').length,0);} + finally{await o.close();} + }); + + for(const failEdition of [false,true])test(`${browserName}: Quran search coalesces edits during ${failEdition?'failed':'delayed'} editions and ignores obsolete results`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle'}; + const o=await open(f,browserName,{},true),{page}=o,gate=deferred(),started=deferred();const posts=[]; + try{ + await page.route('**/api/quran/catalog',async route=>{ + const p=route.request().postDataJSON();posts.push(p); + if(p.kind==='editions'){ + started.resolve();await gate.promise; + await route.fulfill(failEdition?{status:503,json:{error:'Obsolete edition failure'}} + :{json:{state:'ready',request:2,data:{editions:[{id:9,name:'Obsolete edition',surahs:[18]}]}}}); + }else await route.fulfill({json:{state:'ready',request:posts.length,data:quranReciters(p.query?2:1,p.query||'Alpha')}}); + }); + await page.locator('[data-view=quran]').click();await page.locator('#quran-load').click(); + await page.waitForFunction(()=>!document.getElementById('quran-reciter').disabled); + await page.locator('#quran-reciter').selectOption('1');await requestStarted(started); + await page.clock.pauseAt(new Date(await page.evaluate(()=>Date.now())+1000)); + assert.ok(await page.locator('#quran-search').isEnabled()); + await page.locator('#quran-search').fill('B');await page.clock.runFor(200); + await page.locator('#quran-search').fill('Beta');await page.clock.runFor(failEdition?500:399); + assert.equal(posts.length,2);gate.resolve();await requestsFinished(page); + if(!failEdition){assert.equal(posts.length,2);await page.clock.runFor(1);} + await page.clock.resume(); + await page.waitForFunction(()=>!document.getElementById('quran-reciter').disabled&&document.querySelector('#quran-reciter option[value="2"]')); + assert.deepEqual(posts.filter(p=>p.kind==='reciters').map(p=>[p.query,p.offset]),[['',0],['Beta',0]]); + assert.equal(await page.locator('#quran-edition option[value="9"]').count(),0); + assert.doesNotMatch(await page.locator('#quran-catalog-feedback').textContent(),/Obsolete/); + assert.equal(posts.filter(p=>p.kind==='editions').length,1); + }finally{gate.resolve();await o.close();} + }); + test(`${browserName}: Quran catalog intent rejects stale responses even after returning to the same query`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle'};const o=await open(f,browserName,{},true),{page}=o; + const oldGate=deferred(),oldStarted=deferred(),latestGate=deferred(),latestStarted=deferred();const posts=[]; + try{ + await page.route('**/api/quran/catalog',async route=>{ + const p=route.request().postDataJSON();posts.push(p);const n=posts.length; + if(n===2){oldStarted.resolve();await oldGate.promise;} + if(n===3){latestStarted.resolve();await latestGate.promise;} + await route.fulfill({json:{state:'ready',request:n,data:quranReciters(n===1?1:n===2?9:2,n===2?'Obsolete result':n===3?'Latest Beta':'Alpha')}}); + }); + await page.locator('[data-view=quran]').click();await page.locator('#quran-load').click();await requestsFinished(page); + await page.locator('#quran-search').fill('Beta');await page.clock.runFor(400);await requestStarted(oldStarted); + await page.locator('#quran-search').fill('Gamma');await page.clock.runFor(400); + await page.locator('#quran-search').fill('Beta');await page.clock.runFor(400); + oldGate.resolve();await requestStarted(latestStarted); + assert.equal(await page.locator('#quran-reciter option[value="9"]').count(),0); + assert.equal(await page.locator('#quran-reciter option[value="1"]').count(),1); + latestGate.resolve();await page.waitForFunction(()=>!document.getElementById('quran-reciter').disabled); + assert.deepEqual(posts.map(p=>[p.query,p.offset]),[['',0],['Beta',0],['Beta',0]]); + assert.equal(await page.locator('#quran-reciter option[value="2"]').textContent(),'Latest Beta'); + }finally{oldGate.resolve();latestGate.resolve();await o.close();} + }); + test(`${browserName}: Quran Load more before debounce resets a changed query and replaces the list`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle'};const o=await open(f,browserName,{},true),{page}=o; + const gate=deferred(),started=deferred();const posts=[]; + try{ + await page.route('**/api/quran/catalog',async route=>{ + const p=route.request().postDataJSON();posts.push(p); + if(p.query){started.resolve();await gate.promise;} + await route.fulfill({json:{state:'ready',request:posts.length,data:p.query?quranReciters(21,'Beta'):quranReciters(1,'Alpha',20,25)}}); + }); + await page.locator('[data-view=quran]').click();await page.locator('#quran-load').click();await requestsFinished(page); + await page.locator('#quran-search').fill('Beta');await page.locator('#quran-more').click();await requestStarted(started); + await page.clock.runFor(500);assert.deepEqual(posts.map(p=>[p.query,p.offset]),[['',0],['Beta',0]]); + gate.resolve();await page.waitForFunction(()=>!document.getElementById('quran-reciter').disabled); + assert.equal(await page.locator('#quran-reciter option[value="1"]').count(),0); + assert.equal(await page.locator('#quran-reciter option[value="21"]').count(),1); + assert.ok(await page.locator('#quran-more').isHidden()); + await page.clock.runFor(1000);await requestsFinished(page);assert.equal(posts.length,2); + }finally{gate.resolve();await o.close();} + }); + test(`${browserName}: Quran failed search retries explicitly from zero and only accepted-query pagination appends`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle'};const o=await open(f,browserName,{},true),{page}=o;const posts=[]; + try{ + await page.route('**/api/quran/catalog',async route=>{ + const p=route.request().postDataJSON();posts.push(p); + if(posts.length===2){await route.fulfill({status:503,json:{error:'Search unavailable'}});return;} + await route.fulfill({json:{state:'ready',request:posts.length,data:p.query?quranReciters(p.offset?3:2,p.offset?'Beta next':'Beta',p.offset?null:20,25):quranReciters(1,'Alpha',20,25)}}); + }); + await page.locator('[data-view=quran]').click();await page.locator('#quran-load').click();await requestsFinished(page); + await page.locator('#quran-search').fill('Beta');await page.clock.runFor(400); + await page.waitForFunction(()=>document.getElementById('quran-catalog-feedback').textContent.includes('Search unavailable')); + await page.clock.runFor(2000);await requestsFinished(page);assert.equal(posts.length,2); + await page.locator('#quran-more').click();await page.waitForFunction(()=>!document.getElementById('quran-reciter').disabled); + assert.equal(await page.locator('#quran-reciter option[value="1"]').count(),0); + await page.locator('#quran-more').click();await page.waitForFunction(()=>!document.getElementById('quran-reciter').disabled); + assert.deepEqual(posts.map(p=>[p.query,p.offset]),[['',0],['Beta',0],['Beta',0],['Beta',20]]); + assert.equal(await page.locator('#quran-reciter option[value="2"]').count(),1); + assert.equal(await page.locator('#quran-reciter option[value="3"]').count(),1); + }finally{await o.close();} + }); + for(const uncertain of [false,true])test(`${browserName}: Quran search during ${uncertain?'uncertain':'pending'} Play waits for fresh status and Athan without repeating Play`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle'};const o=await open(f,browserName,{},true),{page}=o; + const gate=deferred(),started=deferred();const posts=[];let plays=0; + try{ + await page.route('**/api/quran/catalog',route=>{ + const p=route.request().postDataJSON();posts.push(p); + return route.fulfill({json:{state:'ready',request:posts.length,data:p.kind==='editions'?{editions:[{id:9,name:'Edition',surahs:[18]}]}:quranReciters(p.query?2:1,p.query||'Alpha')}}); + }); + await page.route('**/api/quran/play',async route=>{ + plays++;f.state.device.quran={supported:true,playback_token:'boot:1',state:'loading',reciter:1,edition:9,surah:18};started.resolve();await gate.promise; + await route.fulfill(uncertain?{status:200,contentType:'application/json',body:'{'}:{json:structuredClone(f.state.device)}); + }); + await quranControls(page);await page.locator('#quran-play').click();await requestStarted(started); + await page.locator('#quran-search').fill('B');await page.clock.runFor(200); + await page.locator('#quran-search').fill('Beta');await page.clock.runFor(500);assert.equal(posts.length,2); + gate.resolve();await requestsFinished(page);await page.clock.runFor(2000);await requestsFinished(page); + assert.equal(posts.length,2);assert.ok(await page.locator('#quran-search').isEnabled()); + f.state.device.quran.state='idle';f.state.device.playback_source='athan'; + await page.clock.runFor(2000);await requestsFinished(page);assert.equal(posts.length,2); + f.state.device.quran.state='playing';f.state.device.playback_source='quran';f.state.device.playing=true; + await page.clock.runFor(2000);await requestsFinished(page); + await page.waitForFunction(()=>!document.getElementById('quran-reciter').disabled); + assert.deepEqual(posts.filter(p=>p.kind==='reciters').map(p=>[p.query,p.offset]),[['',0],['Beta',0]]); + assert.equal(plays,1);assert.equal(f.state.mutations,0); + }finally{gate.resolve();await o.close();} + }); + test(`${browserName}: Quran catalog, editions, surah selection and transient playback`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle',reciter:0,edition:0,surah:0}; + const o=await open(f,browserName),{page}=o;let catalog,requestId=0,plays=0; + try{ + await page.route('**/api/quran/catalog',async route=>{ + if(route.request().method()==='POST'){ + const p=route.request().postDataJSON();requestId++; + catalog=p.kind==='reciters'?{reciters:[{id:1,name:'Example reciter'}],surahs:[{id:1,name:'Al-Fatihah'},{id:18,name:'Al-Kahf'}],total:1,next_offset:null} + :{editions:[{id:9,name:'Example edition',surahs:[18]}]}; + await route.fulfill({json:{state:'loading',request:requestId}}); + }else await route.fulfill({json:{state:'ready',request:requestId,data:catalog}}); + }); + await page.route('**/api/quran/play',async route=>{ + plays++;assert.deepEqual(route.request().postDataJSON(),{reciter:1,edition:9,surah:18,expected_playback_token:'boot:0'}); + f.state.device.quran={supported:true,playback_token:'boot:1',state:'loading',reciter:1,edition:9,surah:18}; + await route.fulfill({json:structuredClone(f.state.device)}); + }); + await page.locator('[data-view=quran]').click();await page.locator('#quran-load').click(); + await page.waitForFunction(()=>!document.getElementById('quran-reciter').disabled); + await page.locator('#quran-reciter').selectOption('1'); + await page.waitForFunction(()=>!document.getElementById('quran-surah').disabled); + assert.equal(await page.locator('#quran-surah option[value="1"]').count(),0); + await page.locator('#quran-surah').selectOption('18');await page.locator('#quran-play').click(); + await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent.includes('Starting Quran')); + assert.equal(plays,1);assert.ok(await page.locator('#quran-stop').isVisible()); + f.state.device.quran.state='playing';f.state.device.playing=true;f.state.device.playback_source='quran'; + await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent.includes('Playing Al-Kahf')); + if(process.env.OPENATHAN_QURAN_CAPTURE_DIR&&browserName==='chromium'){ + const {mkdir}=require('node:fs/promises');await mkdir(process.env.OPENATHAN_QURAN_CAPTURE_DIR,{recursive:true}); + await page.setViewportSize({width:1440,height:1000});await page.screenshot({path:join(process.env.OPENATHAN_QURAN_CAPTURE_DIR,'desktop.png'),fullPage:true}); + await page.setViewportSize({width:390,height:844});await page.screenshot({path:join(process.env.OPENATHAN_QURAN_CAPTURE_DIR,'mobile.png'),fullPage:true}); + } + f.state.device.playing=false;f.state.device.playback_source='none';f.state.device.quran.state='idle'; + await page.locator('#quran-stop').click();await page.locator('#quran-stop').waitFor({state:'hidden'}); + assert.match(await page.locator('#quran-feedback').textContent(),/Playback stopped/); + assert.doesNotMatch(await page.locator('#quran-feedback').textContent(),/requested|Checking/); + assert.equal(f.state.mutations,0);assert.equal(plays,1); + }finally{await o.close();await f.close();} + }); + test(`${browserName}: failed Quran catalog leaves retry available`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle'};const o=await open(f,browserName),{page}=o; + try{ + await page.route('**/api/quran/catalog',route=>route.fulfill({status:503,json:{error:'MP3Quran unavailable'}})); + await page.locator('[data-view=quran]').click();await page.locator('#quran-load').click(); + await page.waitForFunction(()=>document.getElementById('quran-catalog-feedback').textContent.includes('MP3Quran unavailable')); + assert.ok(await page.locator('#quran-load').isEnabled());assert.ok(await page.locator('#quran-play').isDisabled()); + }finally{await o.close();await f.close();} + }); + test(`${browserName}: Quran catalog failure stays visible during playback and completion clears request feedback`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'playing',surah:18};f.state.device.playing=true;f.state.device.playback_source='quran'; + const o=await open(f,browserName),{page}=o;const gate=deferred(); + try{ + await page.route('**/api/quran/catalog',async route=>{await gate.promise;await route.fulfill({status:503,json:{error:'MP3Quran unavailable'}});}); + await page.locator('[data-view=quran]').click();await page.locator('#quran-load').click(); + assert.match(await page.locator('#quran-catalog-feedback').textContent(),/Loading reciters/); + assert.match(await page.locator('#quran-feedback').textContent(),/Playing surah 18/); + gate.resolve();await page.waitForFunction(()=>document.getElementById('quran-catalog-feedback').textContent.includes('MP3Quran unavailable')); + assert.match(await page.locator('#quran-feedback').textContent(),/Playing surah 18/); + f.state.device.quran.state='idle';f.state.device.playing=false;f.state.device.playback_source='none'; + await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent.includes('Playback finished or stopped')); + assert.ok(await page.locator('#quran-stop').isHidden()); + }finally{gate.resolve();await o.close();await f.close();} + }); + test(`${browserName}: uncertain Quran Play is read back without repeating and Stop cancels loading`,async()=>{ + const f=await fixture();f.state.device.quran={supported:true,playback_token:'boot:0',state:'idle'};const o=await open(f,browserName),{page}=o;let plays=0; + try{ + await page.route('**/api/quran/catalog',route=>route.fulfill({json:{state:'ready',request:1,data:route.request().postDataJSON().kind==='reciters' + ?{reciters:[{id:1,name:'Example'}],surahs:[{id:18,name:'Al-Kahf'}],total:1,next_offset:null} + :{editions:[{id:9,name:'Edition',surahs:[18]}]}}})); + await page.route('**/api/quran/play',async route=>{ + plays++;f.state.device.quran={supported:true,playback_token:'boot:1',state:'loading',reciter:1,edition:9,surah:18}; + await route.fulfill({status:200,contentType:'application/json',body:'{'}); + }); + await page.route('**/api/stop',async route=>{stopQuran(f.state.device);await route.fulfill({json:structuredClone(f.state.device)});}); + await page.locator('[data-view=quran]').click();await page.locator('#quran-load').click(); + await page.waitForFunction(()=>!document.getElementById('quran-reciter').disabled);await page.locator('#quran-reciter').selectOption('1'); + await page.waitForFunction(()=>!document.getElementById('quran-surah').disabled);await page.locator('#quran-surah').selectOption('18'); + await page.locator('#quran-play').click();await requestsFinished(page); + await page.waitForFunction(()=>document.getElementById('quran-feedback').textContent.includes('Starting Quran')); + assert.equal(plays,1);assert.ok(await page.locator('#quran-play').isDisabled()); + await settings(page);assert.ok(await page.locator('#settings-stop').isVisible()); + assert.match(await page.locator('#settings-playing').textContent(),/Starting Quran/); + if(process.env.OPENATHAN_QURAN_CAPTURE_DIR&&browserName==='chromium'){ + await page.setViewportSize({width:1440,height:1000});await page.screenshot({path:join(process.env.OPENATHAN_QURAN_CAPTURE_DIR,'settings-loading-desktop.png'),fullPage:true}); + await page.setViewportSize({width:390,height:844});await page.screenshot({path:join(process.env.OPENATHAN_QURAN_CAPTURE_DIR,'settings-loading-mobile.png'),fullPage:true}); + } + await page.locator('#settings-stop').click();await page.locator('#settings-playing').waitFor({state:'hidden'}); + await page.locator('[data-view=quran]').click(); + await page.locator('#quran-stop').waitFor({state:'hidden'}); + assert.match(await page.locator('#quran-feedback').textContent(),/Playback stopped/);assert.equal(plays,1);assert.equal(f.state.mutations,0); + }finally{await o.close();await f.close();} + }); for(const stopFirst of [false,true])for(const readback of [false,true])test(`${browserName}: revision-aligned timetable survives ${readback?'Stop readback':'Stop reply'} ${stopFirst?'before':'after'} prayer-save confirmation`,async()=>{ const f=await fixture();f.state.device.playing=true;const o=await open(f,browserName,{},true),{page}=o; const saveGate=deferred(),saveStarted=deferred(),stopGate=deferred(),stopStarted=deferred(),verificationGate=deferred(),verificationStarted=deferred();let saves=0,stops=0,reads=0; @@ -149,7 +495,11 @@ for(const browserName of (process.env.OPENATHAN_TEST_BROWSERS||'chromium').split }); await page.route('**/api/status',async route=>{ reads++; - if(readback&&reads===1){const old=structuredClone(f.state.device);stopStarted.resolve();await stopGate.promise;await route.fulfill({status:200,json:old});return;} + if(reads===1){ + if(readback){const old=structuredClone(f.state.device);stopStarted.resolve();await stopGate.promise;await route.fulfill({status:200,json:old});} + else await route.continue(); + return; + } verificationStarted.resolve();await verificationGate.promise;await route.continue(); }); await page.route('**/api/stop',async route=>{ @@ -169,7 +519,7 @@ for(const browserName of (process.env.OPENATHAN_TEST_BROWSERS||'chromium').split if(!stopFirst){stopGate.resolve();await page.locator('#settings-stop').waitFor({state:'hidden'});} await requestsFinished(page);await page.waitForFunction(()=>document.getElementById('settings-playback-feedback').textContent.includes('Playback stopped'));assert.equal(await page.locator('#offset-asr').inputValue(),'60');assert.ok(await page.locator('#settings-stop').isHidden());assert.match(await page.locator('#settings-playback-feedback').textContent(),/Playback stopped/); await page.locator('[data-view="today"]').click();assert.equal(await page.locator('#next-name').textContent(),'Asr');assert.equal(await page.locator('#next-time').textContent(),'16:45');assert.equal(await page.locator('[data-prayer="asr"] dd').first().textContent(),'16:45');assert.equal(await page.locator('[aria-current="time"]').count(),1);assert.equal(await page.locator('[aria-current="time"]').getAttribute('data-prayer'),'asr');assert.equal(await page.locator('#readiness-text').textContent(),'Ready to play');assert.ok(await page.locator('#stop').isHidden());assert.equal(await page.locator('#skip').textContent(),'Skip Asr today'); - assert.equal(f.state.device.revision,2);assert.equal(saves,1);assert.equal(stops,1);assert.equal(reads,Number(readback)+Number(stopFirst));assert.equal(f.state.posts.length,1); // Only the preview uses the simulated server; routed mutations are counted above. + assert.equal(f.state.device.revision,2);assert.equal(saves,1);assert.equal(stops,1);assert.equal(reads,1+Number(stopFirst));assert.equal(f.state.posts.length,1); // Only the preview uses the simulated server; routed mutations are counted above. await page.locator('#skip').click();await page.waitForFunction(()=>document.getElementById('skip').textContent==='Restore Asr today'&&!document.getElementById('skip').disabled); const skipped=f.state.posts.find(p=>p.url==='/api/skip').payload;assert.equal(skipped.expected_revision,2);assert.equal(skipped.occurrence.utc,f.state.device.schedule.times[3].utc);assert.equal(await page.locator('[aria-current="time"]').getAttribute('data-prayer'),'asr');assert.match(await page.locator('#times [data-prayer="asr"] dt').textContent(),/Skipped/); }finally{saveGate.resolve();stopGate.resolve();verificationGate.resolve();await o.close();} @@ -404,7 +754,7 @@ for(const browserName of (process.env.OPENATHAN_TEST_BROWSERS||'chromium').split const f=await fixture();f.state.device.playing=true;f.state.stopStillPlaying=true; const o=await open(f,browserName),{page}=o,gate=deferred(),started=deferred(); try{ - await page.clock.install();await page.route('**/api/status',async route=>{started.resolve();await gate.promise;if(failure==='network')await route.abort();else await route.fulfill({status:503,json:{error:'Old status unavailable'}});}); + await page.clock.install();await page.route('**/api/status',async route=>{started.resolve();await gate.promise;if(failure==='network')await route.abort();else await route.fulfill({status:503,json:{error:'Old status unavailable'}});},{times:1}); await page.clock.runFor(5000);await started.promise;await page.locator('#stop').click();await page.waitForFunction(()=>document.getElementById('action-feedback').textContent.includes('waiting for playback')); gate.resolve();await requestsFinished(page);await page.waitForFunction(()=>document.getElementById('readiness-text').textContent==='Playing'&&!document.getElementById('stop').disabled);assert.ok(await page.locator('#connection-banner').isHidden());assert.ok(await page.locator('#stop').isEnabled());assert.equal(await page.locator('#readiness-text').textContent(),'Playing'); }finally{gate.resolve();await o.close();} @@ -1017,7 +1367,7 @@ for(const browserName of (process.env.OPENATHAN_TEST_BROWSERS||'chromium').split const f=await fixture();f.state.device.playing=true;if(restore)f.state.device.skip={day:f.state.device.next.day,prayer:f.state.device.next.prayer}; const o=await open(f,browserName),{page}=o;let release,requested,oldState,actionStarted=false;const gate=new Promise(r=>release=r),started=new Promise(r=>requested=r); try{ - if(readback)await page.route('**/api/status',async route=>{if(!actionStarted){await route.continue();return;}const raw=structuredClone(f.state.device);requested();await gate;await route.fulfill({status:200,json:raw});}); + if(readback)await page.route('**/api/status',async route=>{if(!actionStarted){await route.continue();return;}actionStarted=false;const raw=structuredClone(f.state.device);requested();await gate;await route.fulfill({status:200,json:raw});}); await page.route(restore?'**/api/cancel-skip':'**/api/skip',async route=>{ if(restore)delete f.state.device.skip;else f.state.device.skip={day:f.state.device.next.day,prayer:f.state.device.next.prayer};oldState=structuredClone(f.state.device);actionStarted=true; if(readback){await route.fulfill({status:200,body:'{'});return;}requested();await gate;await route.fulfill({status:200,json:oldState}); @@ -1136,7 +1486,7 @@ for(const browserName of (process.env.OPENATHAN_TEST_BROWSERS||'chromium').split try{ if(readback)await page.route('**/api/status',async route=>{if(!stopStarted){await route.continue();return;}const raw=structuredClone(f.state.device);requested();await gate;await route.fulfill({status:200,json:raw});}); await page.route('**/api/stop',async route=>{f.state.device.playing=false;const raw=structuredClone(f.state.device);stopStarted=true;if(readback){await route.fulfill({status:200,body:'{'});return;}requested();await gate;await route.fulfill({status:200,json:raw});}); - await page.locator('#stop').click();await started;await page.locator('#skip').click();await page.waitForFunction(restore=>document.querySelector('#skip').textContent===(restore?'Skip Asr today':'Restore Asr today')&&!document.querySelector('#skip').disabled,restore);release(); + await page.locator('#stop').click();await started;await page.locator('#skip').click();await page.waitForFunction(restore=>document.querySelector('#skip').textContent===(restore?'Skip Asr today':'Restore Asr today')&&!document.querySelector('#skip').disabled,restore);if(readback)assert.ok(await page.locator('#stop').isVisible());release(); await page.waitForFunction(()=>document.querySelector('#action-feedback').textContent.startsWith('Playback stopped'));assert.equal(await page.locator('#skip').textContent(),restore?'Skip Asr today':'Restore Asr today');assert.equal(await page.locator('#readiness-text').textContent(),restore?'Ready to play':'Athan skipped today');assert.equal(await page.locator('.time-row[aria-current]').count(),1);assert.equal(await page.locator('.time-row[aria-current] small').textContent(),restore?'Athan on':'Skipped');assert.ok(await page.locator('#stop').isHidden()); }finally{release();await o.close();} }); @@ -1188,8 +1538,8 @@ for(const browserName of (process.env.OPENATHAN_TEST_BROWSERS||'chromium').split test(`${browserName}: an older Stop reply cannot confirm a newer lost Skip outcome`,async()=>{ const f=await fixture();f.state.device.playing=true;const o=await open(f,browserName),{page}=o;let release,requested;const gate=new Promise(r=>release=r),started=new Promise(r=>requested=r); try{ - await page.route('**/api/stop',async route=>{f.state.device.playing=false;const raw=structuredClone(f.state.device);requested();await gate;await route.fulfill({status:200,json:raw});});await page.locator('#stop').click();await started;f.state.actionDrop=true;f.state.failRead=true;await page.locator('#skip').click();await page.waitForFunction(()=>document.querySelector('#action-feedback').textContent.includes('unconfirmed'));release();await page.waitForFunction(()=>document.querySelector('#settings-playback-feedback').textContent==='Playback stopped'); - assert.ok(await page.locator('#skip').isDisabled());assert.equal(await page.locator('#readiness-text').textContent(),'Skip state unconfirmed');assert.match(await page.locator('#action-feedback').textContent(),/Skip state unconfirmed/);assert.deepEqual(f.state.device.skip,{day:f.state.device.next.day,prayer:f.state.device.next.prayer}); + await page.route('**/api/stop',async route=>{f.state.device.playing=false;const raw=structuredClone(f.state.device);requested();await gate;await route.fulfill({status:200,json:raw});});await page.locator('#stop').click();await started;f.state.actionDrop=true;f.state.failRead=true;await page.locator('#skip').click();await page.waitForFunction(()=>document.querySelector('#action-feedback').textContent.includes('unconfirmed'));release();await page.waitForFunction(()=>document.querySelector('#settings-playback-feedback').textContent.includes('Stop unconfirmed')); + assert.ok(await page.locator('#skip').isDisabled());assert.ok(await page.locator('#connection-banner').isVisible());assert.match(await page.locator('#action-feedback').textContent(),/Skip state unconfirmed/);assert.deepEqual(f.state.device.skip,{day:f.state.device.next.day,prayer:f.state.device.next.prayer}); f.state.failRead=false;await refresh(page);await page.waitForFunction(()=>document.querySelector('#skip').textContent==='Restore Asr today'&&!document.querySelector('#skip').disabled);assert.equal(await page.locator('#readiness-text').textContent(),'Athan skipped today');assert.equal(f.state.posts.filter(p=>p.url==='/api/skip').length,1); }finally{release();await o.close();} }); @@ -1287,11 +1637,14 @@ for(const browserName of (process.env.OPENATHAN_TEST_BROWSERS||'chromium').split }); for(const restore of [false,true])for(const readback of [false,true]){ test(`${browserName}: an older ${restore?'Restore':'Skip'} ${readback?'readback':'reply'} keeps the newer prayer occurrence`,async()=>{ - const f=await fixture();f.state.device.playing=true;if(restore)f.state.device.skip={day:f.state.device.next.day,prayer:f.state.device.next.prayer};const o=await open(f,browserName),{page}=o;let release,requested;const gate=new Promise(r=>release=r),started=new Promise(r=>requested=r),path=restore?'cancel-skip':'skip';let reads=0; + const f=await fixture();f.state.device.playing=true;if(restore)f.state.device.skip={day:f.state.device.next.day,prayer:f.state.device.next.prayer};const o=await open(f,browserName),{page}=o;let release,requested;const gate=new Promise(r=>release=r),started=new Promise(r=>requested=r),path=restore?'cancel-skip':'skip';let reads=0,stopping=false;const stopRead=deferred(),stopReadStarted=deferred(); try{ - await page.route('**/api/'+path,async route=>{if(restore)delete f.state.device.skip;else f.state.device.skip={day:f.state.device.next.day,prayer:f.state.device.next.prayer};if(readback){await route.fulfill({status:200,contentType:'application/json',body:'{'});return;}const raw=structuredClone(f.state.device);requested();await gate;await route.fulfill({status:200,json:raw});});if(readback)await page.route('**/api/status',async route=>{if(++reads!==1){await route.continue();return;}const raw=structuredClone(f.state.device);requested();await gate;await route.fulfill({status:200,json:raw});});await page.locator('#skip').click();await started; - const next=f.state.device.schedule.times[4];f.state.device.next={day:20721,prayer:3,name:'Maghrib',local:next.local,utc:next.utc};delete f.state.device.skip;await page.locator('#stop').click();await page.waitForFunction(()=>document.getElementById('next-name').textContent==='Maghrib');release();await page.waitForFunction(()=>!actionBusy,{},{timeout:3000});assert.equal(await page.locator('#next-name').textContent(),'Maghrib');assert.equal(await page.locator('#next-time').textContent(),'18:20');assert.equal(await page.locator('#skip').textContent(),'Skip Maghrib today');assert.match(await page.locator('#action-feedback').textContent(),/current prayer/);assert.equal(await page.locator('[aria-current="time"]').count(),1);assert.equal(await page.locator('[aria-current="time"]').getAttribute('data-prayer'),'maghrib');assert.equal(await page.locator('#readiness-text').textContent(),'Ready to play');assert.ok(await page.locator('#stop').isHidden()); - }finally{release();await o.close();} + await page.route('**/api/'+path,async route=>{if(restore)delete f.state.device.skip;else f.state.device.skip={day:f.state.device.next.day,prayer:f.state.device.next.prayer};if(readback){await route.fulfill({status:200,contentType:'application/json',body:'{'});return;}const raw=structuredClone(f.state.device);requested();await gate;await route.fulfill({status:200,json:raw});});await page.route('**/api/status',async route=>{ + if(readback&&++reads===1){const raw=structuredClone(f.state.device);requested();await gate;await route.fulfill({status:200,json:raw});return;} + if(stopping){stopReadStarted.resolve();await stopRead.promise;}await route.continue(); + });await page.locator('#skip').click();await started; + const next=f.state.device.schedule.times[4];f.state.device.next={day:20721,prayer:3,name:'Maghrib',local:next.local,utc:next.utc};delete f.state.device.skip;stopping=true;await page.locator('#stop').click();await requestStarted(stopReadStarted);await page.waitForFunction(()=>document.getElementById('next-name').textContent==='Maghrib');assert.ok(await page.locator('#stop').isVisible());stopRead.resolve();await page.waitForFunction(()=>document.getElementById('stop').hidden);release();await page.waitForFunction(()=>!actionBusy,{},{timeout:3000});assert.equal(await page.locator('#next-name').textContent(),'Maghrib');assert.equal(await page.locator('#next-time').textContent(),'18:20');assert.equal(await page.locator('#skip').textContent(),'Skip Maghrib today');assert.match(await page.locator('#action-feedback').textContent(),/current prayer/);assert.equal(await page.locator('[aria-current="time"]').count(),1);assert.equal(await page.locator('[aria-current="time"]').getAttribute('data-prayer'),'maghrib');assert.equal(await page.locator('#readiness-text').textContent(),'Ready to play');assert.ok(await page.locator('#stop').isHidden()); + }finally{release();stopRead.resolve();await o.close();} }); } for(const waiting of [false,true]){ diff --git a/tests/display_tests.cpp b/tests/display_tests.cpp index 126f9b2..40efc5a 100644 --- a/tests/display_tests.cpp +++ b/tests/display_tests.cpp @@ -196,6 +196,10 @@ int main(int argc, char **argv) { CHECK(equal(present(in).detail, "Button to stop")); CHECK(equal(present(in).main, "Playing") && equal(present(in).footer, "")); capture("playback", present(in), directory); + CHECK(equal(present(in).heading, "Athan")); + in.quran = true; + CHECK(equal(present(in).heading, "Quran") && equal(present(in).main, "Playing")); + in.quran = false; in.status.playing = false; in.status.skip = key; CHECK(equal(present(in).detail, "Will be skipped")); capture("skip", present(in), directory, true); in.status.skip = EventKey{key.day, Prayer::FAJR}; diff --git a/tests/quran_service_tests.cpp b/tests/quran_service_tests.cpp new file mode 100644 index 0000000..958876b --- /dev/null +++ b/tests/quran_service_tests.cpp @@ -0,0 +1,181 @@ +#include "quran.h" +#include "../firmware/esphome/components/openathan_audio/quran_policy.h" +#include "../firmware/esphome/components/openathan_audio/quran_http.h" +#include +#include +#include +#include + +int64_t esp_timer_get_time() { return 1000000; } +using namespace esphome::openathan_device; +struct Audio : openathan::Playback { + bool active{}, available{true}; uint32_t epoch{}; std::string url; + openathan::PlaybackSource kind{openathan::PlaybackSource::NONE}; + bool ready() const override { return available; } + bool playing() const override { return active; } + bool stream_supported() const override { return true; } + uint32_t playback_epoch() const override { return epoch; } + openathan::PlaybackSource source() const override { return kind; } + bool start(openathan::Track) override { ++epoch; active = true; kind = openathan::PlaybackSource::ATHAN; return true; } + void stop() override { ++epoch; active = false; kind = openathan::PlaybackSource::NONE; } + bool start_stream(const std::string &value) override { url = value; active = true; kind = openathan::PlaybackSource::QURAN; return true; } +}; +JsonDocument input(const char *text) { JsonDocument doc; assert(!deserializeJson(doc, text)); return doc; } +int request_play(Quran &service, JsonDocument &selection, std::string &error) { + JsonDocument state; service.snapshot(state.to()); + selection["expected_playback_token"] = state["playback_token"]; + return service.action("play", selection.as(), error); +} +void execute(Quran &service) { run_worker(); service.loop(true, false); assert(live_http_clients == 0); } +int main() { + assert(quran_audio_url("https://cdn.mp3quran.net/audio/example/001.mp3")); + assert(quran_audio_url("https://server15.mp3quran.net/example/001.mp3")); + for (const char *url : {"http://cdn.mp3quran.net/a.mp3", "https://cdn.mp3quran.net.evil/a.mp3", + "https://cdn.mp3quran.net:443/a.mp3", "https://cdn.mp3quran.net/../a.mp3", + "https://cdn.mp3quran.net/%2f.mp3", "https://127.0.0.1/a.mp3", "https://server.mp3quran.net/a.mp3"}) + assert(!quran_audio_url(url)); + QuranObjectStream parser("reciters"); std::string object; + unsigned objects_seen = 0; + for (char c : std::string("{\"reciters\":[{\"name\":\"A } \\\"\"},{\"id\":2}]}")) { + assert(parser.feed(c, object)); if (!object.empty()) ++objects_seen; + } + assert(parser.complete() && objects_seen == 2); + for (const std::string text : {"{\"reciters\":[{},]}", "{\"reciters\":[{}]}x", "{\"reciters\":[", "{\"re citers\":[]}"}) { + QuranObjectStream bad("reciters"); bool accepted = true; + for (char c : text) accepted = bad.feed(c, object) && accepted; + assert(!accepted || !bad.complete()); + } + QuranObjectStream big("reciters"); bool accepted = true; + for (char c : std::string("{\"reciters\":[{\"name\":\"") + std::string(16384, 'a')) accepted = big.feed(c, object) && accepted; + assert(!accepted); + Audio audio; + esphome::openathan_component::OpenAthan athan; athan.audio = &audio; + Quran service; service.begin(&athan, "boot-a"); service.loop(true, false); + const std::string base = "https://www.mp3quran.net/api/v3/"; + const std::string entry = R"({"id":1,"name":"Example","moshaf":[{"id":9,"name":"Edition","server":"https://cdn.mp3quran.net/example/","surah_list":"1,18,114"}]})"; + http_replies[base + "reciters?language=eng&reciter=1"] = {200, "", "{\"reciters\":[" + entry + "]}"}; + std::string error; + auto play = input(R"({"reciter":1,"edition":9,"surah":18})"); + // Reject a Play that arrives after Stop without allocating a worker or + // disturbing current playback. A token is required even for older clients. + const auto attempts = task_attempts; + assert(service.action("play", play.as(), error) == 409); + JsonDocument before; service.snapshot(before.to()); + play["expected_playback_token"] = before["playback_token"]; + audio.stop(); + const auto stopped_epoch = audio.epoch; + assert(service.action("play", play.as(), error) == 409); + assert(audio.epoch == stopped_epoch && task_attempts == attempts && !pending_worker); + play["expected_playback_token"] = 1; + assert(service.action("play", play.as(), error) == 400); + play["expected_playback_token"] = std::string(60, 'x'); + assert(service.action("play", play.as(), error) == 400); + play["expected_playback_token"] = before["playback_token"]; + play["unexpected"] = true; + assert(service.action("play", play.as(), error) == 400); + play.remove("unexpected"); + audio.start(openathan::Track::NORMAL); + const auto athan_epoch = audio.epoch; + assert(service.action("play", play.as(), error) == 409); + assert(audio.epoch == athan_epoch && audio.kind == openathan::PlaybackSource::ATHAN && task_attempts == attempts); + audio.stop(); // Even after Athan ends, an older Play cannot revive. + assert(service.action("play", play.as(), error) == 409); + service.snapshot(before.to()); + play["expected_playback_token"] = before["playback_token"]; + service.begin(&athan, "boot-b"); + assert(service.action("play", play.as(), error) == 409); + audio.epoch = UINT32_MAX; + service.snapshot(before.to()); + play["expected_playback_token"] = before["playback_token"]; + audio.stop(); + assert(service.action("play", play.as(), error) == 409); + assert(audio.epoch == 0 && task_attempts == attempts); + assert(request_play(service, play, error) == 200); + athan.stop(); execute(service); assert(audio.url.empty()); // Stop while resolving. + assert(request_play(service, play, error) == 200); + service.loop(false, false); execute(service); assert(audio.url.empty()); // Reconnect cannot revive a canceled job. + assert(request_play(service, play, error) == 200); + service.shutdown(); execute(service); assert(audio.url.empty()); + assert(request_play(service, play, error) == 200); + service.loop(true, true); execute(service); assert(audio.url.empty()); // Maintenance beats late results. + assert(request_play(service, play, error) == 200); + audio.start(openathan::Track::NORMAL); execute(service); assert(audio.url.empty()); // Athan beats late result. + audio.active = false; // Athan has reserved ownership while its Stop is still draining. + assert(request_play(service, play, error) == 409); + service.loop(true, false); + assert(request_play(service, play, error) == 409); + audio.active = true; // The reservation must remain protected after playback starts. + assert(request_play(service, play, error) == 409); + audio.stop(); + assert(request_play(service, play, error) == 200); + execute(service); assert(audio.url == "https://cdn.mp3quran.net/example/018.mp3"); + service.loop(false, false);assert(!audio.active && audio.kind==openathan::PlaybackSource::NONE); + service.loop(true, false);assert(!audio.active); + assert(request_play(service, play, error) == 200);execute(service); + audio.available=false;service.loop(true, false);assert(!audio.active);audio.available=true; + audio.stop(); audio.url.clear(); + auto unavailable = input(R"({"reciter":1,"edition":9,"surah":2})"); + assert(request_play(service, unavailable, error) == 200); execute(service); + JsonDocument state; service.snapshot(state.to()); assert(state["state"] == "error" && audio.url.empty()); + service.loop(true, true); assert(request_play(service, play, error) == 409); + service.loop(false, false); assert(request_play(service, play, error) == 503); + service.loop(true, false); + task_available = false; + assert(request_play(service, play, error) == 503); task_available = true; + // The catalog is scanned entry by entry; pagination and search cover all rows. + audio.stop(); reset_http(); + std::string reciters = "{\"reciters\":["; + for (unsigned i = 1; i <= 25; ++i) { + if (i > 1) reciters += ','; + reciters += "{\"id\":" + std::to_string(i) + ",\"name\":\"Reciter " + std::to_string(i) + "\"}"; + } + reciters += "]}"; + http_replies[base + "reciters?language=eng"] = {200, "", reciters}; + std::string chapters = "{\"suwar\":["; + for (unsigned i = 1; i <= 114; ++i) { + if (i > 1) chapters += ','; + chapters += "{\"id\":" + std::to_string(i) + ",\"name\":\"Surah " + std::to_string(i) + "\"}"; + } + chapters += "]}"; + http_replies[base + "suwar?language=eng"] = {200, "", chapters}; + auto catalog = input(R"({"kind":"reciters","offset":0,"query":""})"); + assert(service.action("catalog", catalog.as(), error) == 200); execute(service); + state.clear(); service.catalog_snapshot(state.to()); + assert(state["state"] == "ready" && state["data"]["total"] == 25 && state["data"]["reciters"].size() == 20); + assert(state["data"]["surahs"].size() == 114 && state["data"]["next_offset"] == 20); + catalog["offset"] = 20; + assert(service.action("catalog", catalog.as(), error) == 200); execute(service); + state.clear(); service.catalog_snapshot(state.to()); + assert(state["data"]["reciters"].size() == 5 && state["data"]["next_offset"].isNull()); + catalog["offset"] = 0; catalog["query"] = "RECITER 25"; + audio.start_stream("https://cdn.mp3quran.net/example/018.mp3");const auto epoch=audio.epoch; + assert(service.action("catalog", catalog.as(), error) == 200); execute(service); + assert(audio.active&&audio.epoch==epoch); // Catalog browsing cannot stop current audio. + state.clear(); service.catalog_snapshot(state.to()); + assert(state["data"]["total"] == 1 && state["data"]["reciters"][0]["id"] == 25); + // Redirect destination is rejected before opening its connection. + reset_http(); + http_replies[base + "reciters?language=eng&reciter=1"] = {302, "https://evil.example/reciters", ""}; + assert(request_play(service, play, error) == 200); execute(service); + assert(http_requests.size() == 1 && last_bundle && http_requests[0].auto_redirect_disabled); + reset_http(); + const std::string audio_url = "https://cdn.mp3quran.net/example/018.mp3"; + const std::string redirected = "https://server15.mp3quran.net/example/018.mp3"; + http_replies[audio_url] = {302, redirected, ""}; + http_replies[redirected] = {200, "", "mp3"}; + { QuranHttp response; assert(response.open(audio_url, true)); } + assert(http_requests.size() == 2 && live_http_clients == 0); + { QuranHttp response; assert(response.open(redirected,true));auto *client=response.release(); + assert(client->config.user_data==nullptr); + char key[]="Content-Type",value[]="audio/mpeg"; + esp_http_client_event_t event{HTTP_EVENT_ON_HEADER,key,value,client->config.user_data}; + assert(client->config.event_handler(&event)==0);esp_http_client_cleanup(client); + } + reset_http(); + for (const auto &destination : {"http://cdn.mp3quran.net/example/018.mp3", "https://evil.example/018.mp3"}) { + http_replies[audio_url] = {302, destination, ""}; + { QuranHttp response; assert(!response.open(audio_url, true)); } + assert(http_requests.back().url == audio_url && live_http_clients == 0); + } + std::cout << "Quran parser, URL bounds, cancellation, priority, availability and HTTPS policy passed\n"; +} diff --git a/tests/quran_stubs/esp_err.h b/tests/quran_stubs/esp_err.h new file mode 100644 index 0000000..c5b407a --- /dev/null +++ b/tests/quran_stubs/esp_err.h @@ -0,0 +1,2 @@ +#pragma once +constexpr int ESP_OK = 0; diff --git a/tests/settings_adapter_tests.cpp b/tests/settings_adapter_tests.cpp index 75c1b91..76ae989 100644 --- a/tests/settings_adapter_tests.cpp +++ b/tests/settings_adapter_tests.cpp @@ -627,6 +627,37 @@ static void light_api() { nvs_test::fail_commit=false; f.device.set_light_output(nullptr); CHECK(call("POST",body()).code==404); } +static void local_api_quran() { + using namespace esphome::openathan_device; + struct Probe : QuranApi { + unsigned calls{}; int code{200}; std::string last, token; + void snapshot(JsonObject root) override { root["supported"]=true;root["state"]="loading";root["playback_token"]="boot:1"; } + void catalog_snapshot(JsonObject root) override { root["state"]="ready";root["request"]=7; } + int action(const std::string &action,JsonObjectConst input,std::string &error) override { + ++calls;last=action;token=input["expected_playback_token"] | std::string{};error="Provider unavailable";return code; + } + } probe; + Fixture f;f.begin();LocalApi api(&f.device,nullptr,0);api.set_quran(&probe); + const auto records=nvs_test::committed; + const auto writes=nvs_test::writes; + auto call=[&](const char *method,const char *uri,const char *body="") { + ApiExchange request;request.method=method;request.uri=uri;request.body=body;api.handle(request);return request; + }; + JsonDocument response; + auto catalog=call("GET","/api/quran/catalog");CHECK(catalog.code==200&&!deserializeJson(response,catalog.response)); + CHECK(response["request"]==7&&probe.calls==0); + CHECK(call("POST","/api/quran/catalog",R"({"kind":"reciters","offset":0,"query":""})").code==200&&probe.last=="catalog"); + auto play=call("POST","/api/quran/play",R"({"reciter":1,"edition":9,"surah":18,"expected_playback_token":"boot:0"})"); + CHECK(play.code==200&&probe.last=="play"&&!deserializeJson(response,play.response)); + CHECK(response["quran"]["state"]=="loading"&&response["playback_source"]=="none"); + CHECK(probe.token=="boot:0"&&response["quran"]["playback_token"]=="boot:1"); + probe.code=409;CHECK(call("POST","/api/quran/play","{}").code==409); + const auto calls=probe.calls; + CHECK(call("POST","/api/quran/play","[]").code==400&&probe.calls==calls); + CHECK(call("POST","/api/quran/play-extra","{}").code==404&&probe.calls==calls); + probe.code=503;CHECK(call("POST","/api/quran/play","{}").code==503); + CHECK(nvs_test::committed==records&&nvs_test::writes==writes); +} static void local_api_stop() { using namespace esphome::openathan_device; Fixture f; f.begin(); f.device.utc=epoch({2026,9,25},5)-1; f.device.update(); f.device.step(1); @@ -891,6 +922,6 @@ int main() { display_button_guidance(); display_integration(); light_time_and_setup(); light_integration(); updates_and_replay(); volume_and_faults(); timezones(); occurrence_identity(); setup_gate_and_preview(); maintenance_latches_writes(); #ifdef OPENATHAN_JSON_TEST - display_api(); time_format_api(); light_api(); json_transport(); coordinate_roundtrip(); local_api(); local_api_stop(); local_api_date(); local_api_coordinates(); + display_api(); time_format_api(); light_api(); json_transport(); coordinate_roundtrip(); local_api(); local_api_stop(); local_api_quran(); local_api_date(); local_api_coordinates(); #endif } diff --git a/tests/stubs/esphome/components/speaker/media_player/speaker_media_player.h b/tests/stubs/esphome/components/speaker/media_player/speaker_media_player.h index 80099d9..b715d93 100644 --- a/tests/stubs/esphome/components/speaker/media_player/speaker_media_player.h +++ b/tests/stubs/esphome/components/speaker/media_player/speaker_media_player.h @@ -4,6 +4,7 @@ #include #include #include +#include namespace esphome { namespace media_player { enum MediaPlayerState { MEDIA_PLAYER_STATE_IDLE, MEDIA_PLAYER_STATE_PLAYING, MEDIA_PLAYER_STATE_ANNOUNCING }; @@ -14,7 +15,15 @@ enum class AudioFileType { MP3 }; struct AudioFile { const uint8_t *data; size_t length; AudioFileType file_type; }; } namespace speaker { +enum class AudioPipelineState { STOPPED, PLAYING, ERROR_READING, ERROR_DECODING }; struct SpeakerMediaPlayer { + AudioPipelineState pipeline_state{AudioPipelineState::STOPPED}; + AudioPipelineState media_pipeline_state{AudioPipelineState::STOPPED}; + AudioPipelineState announcement_state() const { return pipeline_state; } + bool playback_stopped() const { + return initialized && pending_commands.empty() && pipeline_state == AudioPipelineState::STOPPED && + media_pipeline_state == AudioPipelineState::STOPPED; + } bool initialized{true}; float volume{0.7f}; bool muted{}, drop_volume{}; @@ -22,22 +31,30 @@ struct SpeakerMediaPlayer { bool is_ready() const { return initialized; } media_player::MediaPlayerState state{media_player::MEDIA_PLAYER_STATE_IDLE}; std::vector commands; + std::deque pending_commands; + // Consuming a Stop command does not synchronously stop a worker pipeline. + void drain_command() { if (!pending_commands.empty()) pending_commands.pop_front(); } + void drain_commands() { pending_commands.clear(); } + void queue(const std::string &command) { commands.push_back(command); pending_commands.push_back(command); } struct Call { SpeakerMediaPlayer *player; bool announcement{}; std::optional volume; + std::string url; void set_volume(float value) { volume = value; } void set_command(media_player::MediaPlayerCommand) {} void set_announcement(bool value) { announcement = value; } + void set_media_url(const std::string &value) { url = value; } void perform() { - if (volume) { + if (!url.empty()) player->queue(url); + else if (volume) { if (!player->drop_volume) { player->volume = *volume; player->muted = *volume == 0; } - } else player->commands.push_back(announcement ? "stop announcement" : "stop media"); + } else player->queue(announcement ? "stop announcement" : "stop media"); } }; Call make_call() { return {this}; } std::vector queued; - void play_file(audio::AudioFile *file, bool, bool) { queued.push_back(file); commands.push_back("play"); } + void play_file(audio::AudioFile *file, bool, bool) { queued.push_back(file); queue("play"); } }; } } diff --git a/tests/stubs/esphome/core/hal.h b/tests/stubs/esphome/core/hal.h index 177bb9f..63c7941 100644 --- a/tests/stubs/esphome/core/hal.h +++ b/tests/stubs/esphome/core/hal.h @@ -1,3 +1,4 @@ #pragma once #include namespace esphome { inline void delayMicroseconds(uint32_t) {} } +namespace esphome { inline uint32_t mock_millis{}; inline uint32_t millis() { return mock_millis; } } diff --git a/tests/test_quran_vendoring.py b/tests/test_quran_vendoring.py new file mode 100644 index 0000000..2e834c4 --- /dev/null +++ b/tests/test_quran_vendoring.py @@ -0,0 +1,18 @@ +"""Keep the product reader patch limited to two pinned ESPHome source files.""" +import hashlib +import json +from pathlib import Path +import unittest + +ROOT = Path(__file__).resolve().parents[1] + +class QuranVendoringTests(unittest.TestCase): + def test_pinned_unmodified_audio_sources(self): + components = ROOT / "firmware/esphome/components" + manifest = json.loads((components / "audio/upstream.json").read_text()) + self.assertEqual(manifest["version"], "2026.9.0") + self.assertIn("esphome==" + manifest["version"], (ROOT / "firmware/esphome/requirements.txt").read_text()) + modified = {"audio/audio_reader.cpp", "speaker/media_player/speaker_media_player.h"} + for name, expected in manifest["sha256"].items(): + if name not in modified: + self.assertEqual(hashlib.sha256((components / name).read_bytes()).hexdigest(), expected, name) diff --git a/tests/upgrade_stubs/esp_http_client.h b/tests/upgrade_stubs/esp_http_client.h index e2cec70..3b0e0cc 100644 --- a/tests/upgrade_stubs/esp_http_client.h +++ b/tests/upgrade_stubs/esp_http_client.h @@ -59,6 +59,7 @@ inline int esp_http_client_fetch_headers(Http *h){ return h->data.size(); } inline int esp_http_client_get_status_code(Http *h){return h->reply.status;} +inline int esp_http_client_set_user_data(Http *h,void *value){h->config.user_data=value;return 0;} inline int esp_http_client_get_content_length(Http *h){return h->data.size();} inline int esp_http_client_read(Http *h,char *out,int size){if(on_read){auto callback=on_read;on_read=nullptr;callback();}if(transfer_fail)return -1;size=std::min(size,h->data.size()-h->offset);memcpy(out,h->data.data()+h->offset,size);h->offset+=size;return size;} inline bool esp_http_client_is_complete_data_received(Http *h){return h->offset==h->data.size();} diff --git a/tests/upgrade_stubs/esphome/components/openathan/openathan.h b/tests/upgrade_stubs/esphome/components/openathan/openathan.h index fb182fd..e668657 100644 --- a/tests/upgrade_stubs/esphome/components/openathan/openathan.h +++ b/tests/upgrade_stubs/esphome/components/openathan/openathan.h @@ -14,6 +14,9 @@ class OpenAthan { Clock clock; Status sample; Settings settings; + ::openathan::Playback *audio{}; + ::openathan::Playback *playback() const { return audio; } + void stop() { if (audio) audio->stop(); } Status status() const {return sample;} Clock read() const {return clock;} Settings *settings_service(){return &settings;} diff --git a/tools/check_quran.py b/tools/check_quran.py new file mode 100644 index 0000000..3d15367 --- /dev/null +++ b/tools/check_quran.py @@ -0,0 +1,27 @@ +"""Exercise the production Quran worker against deterministic HTTP/task adapters.""" +import os +from pathlib import Path +import subprocess +import tempfile +import argparse + +ROOT = Path(__file__).resolve().parents[1] + +def check(arduinojson): + with tempfile.TemporaryDirectory() as directory: + binary = Path(directory) / "quran-tests" + subprocess.run([os.environ.get("CXX", "c++"), "-std=c++20", "-fsanitize=undefined", + "-fno-sanitize-recover=all", "-I" + str(ROOT / "tests/upgrade_stubs"), + "-I" + str(ROOT / "tests/quran_stubs"), + "-I" + str(ROOT / "tests/stubs"), "-I" + arduinojson, + "-I" + str(ROOT / "lib/openathan-core/include"), + "-I" + str(ROOT / "firmware/esphome/components/openathan_device"), + str(ROOT / "tests/quran_service_tests.cpp"), + str(ROOT / "firmware/esphome/components/openathan_device/quran.cpp"), + "-o", str(binary)], check=True) + subprocess.run([str(binary)], check=True) + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--arduinojson", required=True) + check(parser.parse_args().arduinojson) diff --git a/web/device-ui/README.md b/web/device-ui/README.md index 443ce08..6eeb522 100644 --- a/web/device-ui/README.md +++ b/web/device-ui/README.md @@ -18,6 +18,9 @@ highlight stays on that occurrence when skipped. An occurrence outside today’s timetable has a dated hero and no highlighted row. Connection loss freezes the last observed information and marks it stale; it does not assert playback stopped. Stop stays reachable from Today and Settings while playback is reported active. +The optional Quran view also exposes Stop while a recording is resolving/loading. +All three views keep Stop reachable during pending or unconfirmed Play requests, +including while the device still reports idle. The current row matches the device's next prayer and UTC instant, rather than prayer name or browser time alone. Only the confirmed Today timetable has a highlight; preview and setup-review tables do not. @@ -30,11 +33,14 @@ clients tolerate its absence and hide the header rather than use the browser's date. No playing-prayer identity, playback history or device-accurate countdown is inferred. Skip/Restore keep the reported day/prayer key and settings revision. -Stop bypasses the preference-write scheduler. An HTTP-successful response that -still reports `playing:true` remains pending; only authoritative `playing:false` -confirms completion. A failed response triggers readback. Failed readback reports -an unconfirmed Stop and warns that playback may remain active. Today and Settings -share that pending, confirmed or unconfirmed feedback. +Stop bypasses the preference-write scheduler and supersedes pending Play feedback. +Every Play or Stop response, including an uncertain failure, triggers a new status +read. Earlier polls cannot settle the command or overwrite its subsequent playback +observation. `playing:true` or Quran loading keeps Stop pending; authoritative +`playing:false` without Quran loading confirms completion only after any pending +Play token is invalidated. Failed readback reports an unconfirmed Stop and warns that +playback may remain active. Today and Settings +share that pending, confirmed or unconfirmed feedback with the Quran view. ### Everyday preferences @@ -183,8 +189,18 @@ credential-recovery tool, with a separately gated preserving USB updater for capable firmware. While a USB request is active, the local Updates group displays transfer, interruption or owner power-handoff status and hides network cancellation; firmware rejects competing mutations. See the [USB protocol](../../docs/development/usb-firmware-updates.md). -Quran/adhkar and owner-facing audio replacement remain -future work. +The development Quran view loads MP3Quran's live reciters with search and paging, +then recording editions and available surahs. Catalog and playback feedback stay +independent. Browser storage remembers selections; Play and Stop do not change +prayer settings. One surah plays on the speaker even after the page closes; +Athan interrupts without automatic resume. Play carries the current device playback +token, so a delayed request cannot restart audio after Stop or reboot. An uncertain +Play is read back rather than automatically repeated; idle with an unchanged token +keeps Play blocked and Stop reachable. Tokens remain in memory, while browser +storage remembers selection IDs only. A missing token requires page reload and +firmware compatibility checking. See the [streaming contract](../../docs/development/quran-streaming.md) +for APIs, bounds and physical qualification gates. Adhkar and owner-facing stored +audio replacement remain future work. The [redesign validation report](../../docs/development/device-ui-redesign-validation-2026-10-05.md) retains integration evidence. The [holistic review report](../../docs/development/device-ui-holistic-review-2026-10-06.md) diff --git a/web/device-ui/app.js b/web/device-ui/app.js index f900d89..7116d84 100644 --- a/web/device-ui/app.js +++ b/web/device-ui/app.js @@ -37,6 +37,8 @@ let actionBusy = false, actionOrder = 0; let stopBusy = false, stopAwaiting = false; +let playbackIntent = null, + playbackReadFloor = 0; let timezonesReady = false, timezonesLoading = false, proposedLocation; @@ -362,8 +364,9 @@ function projectSnapshot() { for (const key of ["display", "lights", "time_format"]) snapshot[key] = domainData(key); snapshot.firmware = firmware.data; } -function acceptResponse({ data, order }, resource = "status") { +function acceptResponse({ data, order }, resource = "status", allowPlayback = true) { const accepted = {}; + let playbackAccepted = false; const initial = !snapshot, wasFault = settingsFault(); if (resource === "status") { @@ -390,18 +393,23 @@ function acceptResponse({ data, order }, resource = "status") { time_format, firmware: update, playing, + quran, + playback_source, ...operational } = data; - observedStatus.data = { ...operational, playing: observedStatus.data?.playing }; + observedStatus.data = { ...operational, playing: observedStatus.data?.playing, + quran: observedStatus.data?.quran, playback_source: observedStatus.data?.playback_source }; observedStatus.order = order; if (actionUncertain && order >= actionOrder) { actionUncertain = false; status("action", "Skip state checked · review the current prayer"); } } - if (order >= observedStatus.playbackOrder) { - observedStatus.data = { ...observedStatus.data, playing: data.playing }; + if (allowPlayback && order >= Math.max(observedStatus.playbackOrder, playbackReadFloor)) { + observedStatus.data = { ...observedStatus.data, playing: data.playing, + quran: data.quran, playback_source: data.playback_source }; observedStatus.playbackOrder = order; + playbackAccepted = true; } } else if (resource !== "firmware") accepted[resource] = acceptDomain(resource, data, order); const update = resource === "firmware" ? data : resource === "status" ? data.firmware : null; @@ -416,7 +424,7 @@ function acceptResponse({ data, order }, resource = "status") { connected = true; projectSnapshot(); if (snapshot) { - if (stopAwaiting && snapshot.playing === false) stoppedFeedback(); + if (playbackAccepted) settlePlaybackIntent(); if (firstRun && snapshot.setup === "active" && !draftDirty && snapshot.settings) { fillDraft(snapshot.settings); status("prayer", "Saved prayer settings", "success"); @@ -454,6 +462,7 @@ function acceptResponse({ data, order }, resource = "status") { render(); applyPendingLocation(); queueMicrotask(pump); + queueMicrotask(drainQuranSearch); return accepted; } function acceptSavedResponse(key, response) { @@ -640,6 +649,7 @@ function renderTimes(id, schedule, settings, isPreview = false) { } } function render() { + renderQuran(); const state = snapshot, next = state?.clock_ready && state?.setup === "active" ? state.next : null, skipped = isSkipped(next, state?.skip); @@ -680,7 +690,7 @@ function render() { : !connected ? "Connection lost · stale" : state.playing - ? "Playing" + ? state.playback_source === "quran" ? "Quran playing" : "Playing" : settingsFault() ? "Saved storage unavailable" : actionUncertain @@ -724,13 +734,16 @@ function render() { text("state-detail", detail); $("state-detail").hidden = !detail; for (const id of ["stop", "settings-stop"]) { - $(id).hidden = !state?.playing; + $(id).hidden = !playbackControlsNeeded(); $(id).disabled = !connected || stopBusy; } - $("settings-playing").hidden = !state?.playing; + $("settings-playing").hidden = !playbackControlsNeeded(); + $("settings-playing").querySelector("strong").textContent = playbackIntent ? "Checking playback" + : state?.quran?.state === "loading" ? "Starting Quran" : "Playing"; $("settings-playing").querySelector(".hint").textContent = connected - ? "Playback is active on the speaker." - : "Last observed playing · connection lost."; + ? playbackIntent ? quran.playMessage : state?.quran?.state === "loading" ? "Quran is starting on the speaker." + : state?.playback_source === "quran" ? "Quran is playing on the speaker." : "Playback is active on the speaker." + : "Connection lost · playback may still be active."; const skipKey = state?.skip, skipName = skipKey ? title(prayers[skipKey.prayer]) : next?.name; const actionDate = @@ -767,6 +780,7 @@ function showView(view, focus = true) { if (firstRun) view = "settings"; $("today-view").hidden = view !== "today"; $("settings-view").hidden = view !== "settings"; + $("quran-view").hidden = view !== "quran"; document.querySelectorAll("[data-view]").forEach((b) => { if (b.dataset.view === view) b.setAttribute("aria-current", "page"); else b.removeAttribute("aria-current"); @@ -776,6 +790,260 @@ function showView(view, focus = true) { scrollTo(0, 0); } } + +// Quran sessions are transient; saved prayer settings never acquire media URLs. +const quran = { reciters: [], surahs: [], editions: [], next: null, total: 0, + busy: false, loaded: false, requestedQuery: "", acceptedQuery: "", intent: 0, + pendingSearch: false, searchReady: false, searchTimer: null, + message: "Load the catalog to choose a recording.", tone: "", selection: {}, + playMessage: "", playTone: "", observedState: "idle" }; +try { + const saved = JSON.parse(localStorage.getItem("openathan-quran") || "{}"); + if (saved && typeof saved === "object" && !Array.isArray(saved)) + for (const key of ["reciter", "edition", "surah"]) + if (/^[1-9][0-9]*$/.test(String(saved[key]))) quran.selection[key] = String(saved[key]); +} catch {} +function quranPlaybackFeedback(message, tone = "") { + quran.playMessage = message; quran.playTone = tone; +} +function playbackActive() { return !!snapshot?.playing || snapshot?.quran?.state === "loading"; } +function playbackControlsNeeded() { return playbackActive() || !!playbackIntent || stopBusy || stopAwaiting; } +function playbackToken() { + const token = snapshot?.quran?.playback_token; + return typeof token === "string" && token.length > 0 && token.length <= 59 ? token : null; +} +function beginPlaybackIntent(kind) { + const intent = { kind, token: playbackToken(), settled: false, uncertain: true, + cancelToken: playbackIntent?.kind === "play" ? playbackIntent.token : playbackIntent?.cancelToken }; + playbackIntent = intent; + // Reads already in flight cannot settle a command that has not replied yet. + playbackReadFloor = Infinity; + return intent; +} +function stopFeedback(message, tone = "warning") { + status("action", message + (actionUncertain ? " · Skip state unconfirmed; refresh to check." : ""), tone); + text("settings-playback-feedback", message); + quranPlaybackFeedback(message, tone); +} +function settlePlaybackIntent() { + const intent = playbackIntent; + if (!intent?.settled || intent.readbackPending) return; + const token = playbackToken(); + if (intent.kind === "play") { + if (!token || typeof snapshot.playing !== "boolean" || + !["idle", "loading", "playing", "error"].includes(snapshot.quran?.state) || + intent.uncertain && token === intent.token) { + quranPlaybackFeedback("Play unconfirmed · use Stop before trying again.", "warning"); + return; + } + playbackIntent = null; + quranPlaybackFeedback(intent.error ? intent.error + ". Check playback before trying again." + : snapshot.quran?.state === "idle" ? "Playback finished or stopped." + : "Quran requested. Checking speaker playback…", intent.error ? "error" : ""); + } else if (snapshot.playing === false && !["loading", "playing"].includes(snapshot.quran?.state) && + (!intent.cancelToken || token && token !== intent.cancelToken)) { + playbackIntent = null; + stoppedFeedback(intent.uncertain); + } else stopFeedback(playbackActive() + ? intent.uncertain ? "Playback is still active. Try Stop again." : "Stop requested · waiting for playback to stop" + : "Stop unconfirmed · try Stop again before Play."); +} +async function readPlaybackIntent(intent) { + if (playbackIntent !== intent) return; + intent.settled = true; + intent.readbackPending = true; + // Fence every read issued before the command settled, including a later + // numbered poll that the device answered before admitting Play or Stop. + playbackReadFloor = requestOrder + 1; + try { + const response = await request("/api/status"); + acceptResponse(response, "status", playbackIntent === intent); + } catch (error) { + if (playbackIntent !== intent) return; + connectionFailure(error); + if (intent.kind === "stop") stopFeedback("Stop unconfirmed · reconnect to check. Playback may still be active."); + else quranPlaybackFeedback("Play unconfirmed · reconnect and use Stop before trying again.", "warning"); + } finally { + intent.readbackPending = false; + // A concurrent settings or Skip reply may carry a newer observation. Finish + // this readback with that state, retaining both its data and Stop feedback. + if (playbackIntent === intent && observedStatus.playbackOrder >= playbackReadFloor) + settlePlaybackIntent(); + } +} +function rememberQuran() { + quran.selection = { reciter: $("quran-reciter").value, edition: $("quran-edition").value, + surah: $("quran-surah").value }; + try { localStorage.setItem("openathan-quran", JSON.stringify(quran.selection)); } catch {} +} +function quranOptions(id, entries, placeholder, selected) { + const input = $(id); + input.replaceChildren(); + if (placeholder) input.add(new Option(placeholder, "")); + for (const entry of entries) input.add(new Option(entry.name, String(entry.id))); + if (entries.some((entry) => String(entry.id) === String(selected))) input.value = String(selected); +} +function filterQuranReciters() { + const value = $("quran-reciter").value || quran.selection.reciter; + quranOptions("quran-reciter", quran.reciters, "Choose a reciter", value); + renderQuran(); +} +function quranSurahs() { + const edition = quran.editions.find((entry) => String(entry.id) === $("quran-edition").value); + const available = new Set(edition?.surahs || []); + quranOptions("quran-surah", quran.surahs.filter((entry) => available.has(entry.id)), + "Choose a surah", quran.selection.surah); + renderQuran(); +} +function renderQuran() { + const state = snapshot?.quran; + if (connected && state) { + if (state.state === "idle" && ["playing", "loading"].includes(quran.observedState) && !playbackIntent && !stopBusy && !stopAwaiting) + quranPlaybackFeedback("Playback finished or stopped."); + quran.observedState = state.state; + } + const surahName = quran.surahs.find((entry) => entry.id === state?.surah)?.name; + $("quran-nav").hidden = !state?.supported || firstRun; + const disabled = !connected || firstRun || !state?.supported; + const catalogBusy = quran.busy || quran.pendingSearch || !!playbackIntent; + $("quran-load").disabled = disabled || quran.busy || !!playbackIntent; + $("quran-load").textContent = quran.reciters.length ? "Refresh catalog" : "Load reciters"; + $("quran-search").disabled = disabled || !quran.loaded; + $("quran-reciter").disabled = disabled || catalogBusy || !quran.reciters.length; + $("quran-more").hidden = quran.next === null; + $("quran-more").disabled = disabled || quran.busy || !!playbackIntent; + $("quran-edition").disabled = disabled || catalogBusy || !quran.editions.length; + $("quran-surah").disabled = disabled || catalogBusy || !$("quran-edition").value; + $("quran-play").disabled = disabled || catalogBusy || stopBusy || stopAwaiting || !playbackToken() || !$("quran-surah").value || + snapshot?.playback_source === "athan" || state?.state === "loading"; + $("quran-stop").hidden = !playbackControlsNeeded(); + $("quran-stop").disabled = !connected || stopBusy; + text("quran-count", quran.reciters.length ? `${quran.reciters.length} of ${quran.total} reciters loaded` : ""); + text("quran-catalog-feedback", quran.message); + $("quran-catalog-feedback").className = "feedback " + quran.tone; + const message = !connected ? (playbackIntent || stopAwaiting) && quran.playMessage || "Connection lost · playback may still be active." + : playbackIntent || stopBusy || stopAwaiting ? quran.playMessage : state?.state === "loading" + ? "Starting Quran on the speaker…" : state?.state === "playing" ? `Playing ${surahName || "surah " + state.surah} on the speaker` + : snapshot?.playback_source === "athan" ? "Athan is playing on the speaker." + : state?.state === "error" ? state.error : state?.supported && !playbackToken() + ? "Reload the page and check device firmware before pressing Play." : quran.playMessage; + text("quran-feedback", message); + $("quran-feedback").hidden = !message; + $("quran-feedback").className = "feedback " + (state?.state === "error" ? "error" : + ["playing", "loading"].includes(state?.state) && !stopAwaiting ? "" : quran.playTone); +} +async function quranCatalog(payload) { + const initial = (await request("/api/quran/catalog", payload)).data; + const id = initial.request; + const deadline = Date.now() + 75000; + let result = initial; + while (result.state === "loading" && Date.now() < deadline) { + await new Promise((resolve) => setTimeout(resolve, 500)); + result = (await request("/api/quran/catalog")).data; + if (result.request !== id) throw new Error("Another page requested a catalog. Try again."); + } + if (result.state !== "ready" || !result.data) throw new Error(result.error || "Catalog loading timed out. Try again."); + return result.data; +} +function quranCatalogAvailable() { + return !quran.busy && !playbackIntent && !stopBusy && !stopAwaiting && connected && !firstRun && snapshot?.quran?.supported && + snapshot.quran.state !== "loading" && snapshot.playback_source !== "athan"; +} +function drainQuranSearch() { + if (quran.pendingSearch && quran.searchReady && quranCatalogAvailable()) + void requestQuranReciters(false); +} +function loadQuran(more = false) { + clearTimeout(quran.searchTimer); quran.searchTimer = null; + const query = $("quran-search").value.trim(); + if (query !== quran.requestedQuery) ++quran.intent; + quran.requestedQuery = query; + quran.pendingSearch = true; quran.searchReady = true; + if (!quranCatalogAvailable()) { renderQuran(); return; } + // An offset belongs to the last accepted query, never to the editable input. + return requestQuranReciters(more && quran.loaded && query === quran.acceptedQuery); +} +async function requestQuranReciters(more) { + const query = quran.requestedQuery, intent = quran.intent; + quran.pendingSearch = false; quran.searchReady = false; + quran.busy = true; quran.message = "Loading reciters from MP3Quran…"; quran.tone = ""; + let accepted = false; + renderQuran(); + try { + const data = await quranCatalog({ kind: "reciters", offset: more ? quran.next : 0, query }); + if (intent !== quran.intent) return; + if (!Array.isArray(data.reciters) || !Array.isArray(data.surahs)) throw new Error("Catalog response is incomplete"); + quran.reciters = more ? [...new Map([...quran.reciters, ...data.reciters].map((entry) => [entry.id, entry])).values()] : data.reciters; + quran.surahs = data.surahs; quran.next = data.next_offset; quran.total = data.total; + quran.loaded = true; quran.acceptedQuery = query; accepted = true; + if (!more) { quran.editions = []; quranOptions("quran-edition", [], "", ""); quranSurahs(); } + filterQuranReciters(); + quran.message = data.total ? "Choose a reciter. Load more to browse all results." : "No reciters match this search."; + } catch (error) { + if (intent === quran.intent) { quran.message = error.message; quran.tone = "error"; } + } finally { + quran.busy = false; renderQuran(); + if (quran.pendingSearch) drainQuranSearch(); + else if (accepted && intent === quran.intent && $("quran-reciter").value && !quran.editions.length) + void loadQuranEditions(); + } +} +async function loadQuranEditions() { + if (quran.busy || quran.pendingSearch) return; + quran.editions = []; quranOptions("quran-edition", [], "", ""); quranSurahs(); + const reciter = Number($("quran-reciter").value); + if (!reciter) return; + const intent = quran.intent; + quran.busy = true; quran.message = "Loading this reciter’s recordings…"; quran.tone = ""; + renderQuran(); + try { + const data = await quranCatalog({ kind: "editions", reciter }); + if (intent !== quran.intent || reciter !== Number($("quran-reciter").value)) return; + if (!Array.isArray(data.editions)) throw new Error("Recording response is incomplete"); + quran.editions = data.editions; + quranOptions("quran-edition", data.editions, data.editions.length > 1 ? "Choose an edition" : "", + quran.selection.reciter === String(reciter) ? quran.selection.edition : ""); + $("quran-edition-label").hidden = data.editions.length <= 1; + quranSurahs(); + quran.message = data.editions.length ? "Choose a surah, then press Play." : "This reciter has no available recordings."; + } catch (error) { + if (intent === quran.intent) { quran.message = error.message; quran.tone = "error"; } + } finally { quran.busy = false; renderQuran(); drainQuranSearch(); } +} +$("quran-load").addEventListener("click", () => loadQuran()); +$("quran-more").addEventListener("click", () => loadQuran(true)); +$("quran-search").addEventListener("input", () => { + clearTimeout(quran.searchTimer); + ++quran.intent; quran.requestedQuery = $("quran-search").value.trim(); + quran.pendingSearch = true; quran.searchReady = false; + quran.searchTimer = setTimeout(() => { + quran.searchTimer = null; quran.searchReady = true; drainQuranSearch(); + }, 400); + renderQuran(); +}); +$("quran-reciter").addEventListener("change", loadQuranEditions); +$("quran-edition").addEventListener("change", () => { quranSurahs(); rememberQuran(); }); +$("quran-surah").addEventListener("change", () => { rememberQuran(); renderQuran(); }); +$("quran-stop").addEventListener("click", stopPlayback); +$("quran-play").addEventListener("click", async () => { + if (quran.busy || $("quran-play").disabled) return; + const intent = beginPlaybackIntent("play"); + quranPlaybackFeedback("Requesting Quran playback…"); + rememberQuran(); render(); + try { + const response = await request("/api/quran/play", { reciter: Number(quran.selection.reciter), + edition: Number(quran.selection.edition), surah: Number(quran.selection.surah), expected_playback_token: intent.token }); + acceptResponse(response, "status", false); + intent.uncertain = false; + } catch (error) { + intent.error = error.message; + intent.uncertain = !error.status; + } + if (playbackIntent !== intent) return; + quranPlaybackFeedback("Checking speaker playback…"); + await readPlaybackIntent(intent); + if (playbackIntent === intent || !playbackIntent) { render(); drainQuranSearch(); } +}); function focusStep() { const id = setupStep === 0 @@ -1273,45 +1541,34 @@ async function runAction(action) { render(); } } -function stoppedFeedback() { +function stoppedFeedback(readback = false) { stopAwaiting = false; + const message = "Playback stopped" + (readback ? " · confirmed after readback" : ""); status( "action", - "Playback stopped" + (actionUncertain ? " · Skip state unconfirmed; refresh to check." : ""), + message + (actionUncertain ? " · Skip state unconfirmed; refresh to check." : ""), actionUncertain ? "warning" : "success" ); - text("settings-playback-feedback", "Playback stopped"); + text("settings-playback-feedback", message); + quranPlaybackFeedback(message, "success"); } async function stopPlayback() { if (stopBusy || !connected) return; + const intent = beginPlaybackIntent("stop"); stopBusy = true; stopAwaiting = true; status("action", "Requesting Stop…"); text("settings-playback-feedback", "Requesting Stop…"); + quranPlaybackFeedback("Requesting Stop…"); $("action-feedback").hidden = false; $("settings-playback-feedback").hidden = false; render(); try { - acceptResponse(await request("/api/stop", {})); - if (snapshot.playing) { - stopAwaiting = true; - status("action", "Stop requested · waiting for playback to stop", "warning"); - text("settings-playback-feedback", "Stop requested · waiting for playback to stop"); - } else stoppedFeedback(); - } catch { - try { - acceptResponse(await request("/api/status")); - const message = snapshot.playing - ? "Playback is still active. Try Stop again." - : "Playback stopped · confirmed after readback"; - status("action", message, snapshot.playing ? "warning" : "success"); - text("settings-playback-feedback", message); - } catch (error) { - connectionFailure(error); - const message = "Stop unconfirmed · reconnect to check. Playback may still be active."; - status("action", message, "warning"); - text("settings-playback-feedback", message); - } + acceptResponse(await request("/api/stop", {}), "status", false); + intent.uncertain = false; + } catch { /* Never infer that an uncertain Stop took effect. */ } + try { + await readPlaybackIntent(intent); } finally { stopBusy = false; $("action-feedback").hidden = false; diff --git a/web/device-ui/index.html b/web/device-ui/index.html index cd12f76..5b29dc2 100644 --- a/web/device-ui/index.html +++ b/web/device-ui/index.html @@ -12,6 +12,7 @@
@@ -65,6 +66,23 @@

Updates


Keep your phone on the same network as your speaker. Wi-Fi or password changed? Connect by USB to recover access.

+
diff --git a/web/device-ui/style.css b/web/device-ui/style.css index 84adcbe..a9ed625 100644 --- a/web/device-ui/style.css +++ b/web/device-ui/style.css @@ -1,4 +1,5 @@ :root{color-scheme:dark;--bg:#101820;--surface:#19252f;--raised:#22323f;--fg:#f3f6f8;--muted:#b6c5d1;--amber:#f0bb67;--upcoming-bg:#2b2c29;--border:#7a8d9c;--rule:#354854;--good:#8ed8b5;--bad:#ffabab;--font:system-ui,-apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif} +#quran-view{max-width:48rem;margin:2rem auto}#quran-view h2{font-size:1.75rem;margin-bottom:.5rem}#quran-view select,#quran-view input[type=search]{width:100%}#quran-view label{display:block;margin-top:1.25rem}#quran-view .actions{margin-top:1.5rem}#quran-more{margin-top:1rem} *{box-sizing:border-box}body{margin:0;background:var(--bg);color:var(--fg);font:1rem/1.5 var(--font);caret-color:var(--amber);scrollbar-color:var(--border) var(--bg)}::selection{color:var(--bg);background:var(--amber)}[hidden]{display:none!important}button,input,select{font:inherit}button,a,input,select,summary{-webkit-tap-highlight-color:transparent}button{cursor:pointer;min-height:44px;border:1px solid var(--border);border-radius:8px;padding:9px 16px;background:var(--surface);color:var(--fg);font-weight:600;line-height:1.4}button:hover{background:var(--raised)}button:disabled{cursor:default;opacity:.55}button.primary{background:var(--amber);border-color:var(--amber);color:var(--bg)}button.primary:hover{background:#ffcf86}a{color:var(--amber);text-underline-offset:4px}a:hover{text-decoration-thickness:2px}:focus-visible{outline:3px solid var(--amber);outline-offset:3px}h1,h2,h3,p{margin:0}h1{font-size:1.5rem;letter-spacing:-.02em;line-height:1.2}h2{font-size:1.25rem;line-height:1.3}h3{font-size:1.125rem}p{overflow-wrap:anywhere}.device{max-width:1120px;margin:auto;padding-bottom:84px}.device-header{display:flex;align-items:flex-end;justify-content:space-between;gap:12px;border-bottom:1px solid var(--rule);padding:16px 20px}.device-header p{font-size:.8125rem;color:var(--muted);margin-top:3px}.device-header .date{white-space:nowrap;font-size:.8125rem;padding-bottom:1px}.next-and-controls{padding:24px 20px 18px}.next-section h2{font-size:1.125rem;font-weight:400}.next-time{font-size:3.75rem;line-height:1.1;font-weight:700;letter-spacing:-.03em;font-variant-numeric:tabular-nums;margin:8px 0 12px;white-space:nowrap}.readiness{display:flex;align-items:center;gap:10px;color:var(--good);font-size:1.125rem;font-weight:500}.state-dot{width:12px;height:12px;flex:0 0 12px;background:currentColor;border-radius:50%}.audio{margin-top:20px}.range-label{display:flex;justify-content:space-between;gap:12px;font-weight:500}.range-label output{font-variant-numeric:tabular-nums}input[type=range]{width:100%;height:44px;margin:0;accent-color:var(--amber);cursor:pointer}.feedback{font-size:.875rem;min-height:21px;color:var(--muted);margin:0 0 12px}.feedback.success{color:var(--good)}.feedback.error,.error{color:var(--bad)}.feedback.warning,.warning{color:var(--amber)}.hint{font-size:.875rem;color:var(--muted);margin-top:8px}.audio>button{width:100%;margin-top:2px}.audio>button+button{margin-top:10px}.audio #action-feedback{margin-top:10px;margin-bottom:0}.timetable{padding:16px 20px 20px;border-top:1px solid var(--rule)}.timetable h2{margin-bottom:10px}.timetable dl{margin:0}.time-row{display:grid;grid-template-columns:minmax(0,1fr) auto;align-items:center;gap:14px;min-height:52px;padding:10px 0;border-top:1px solid var(--rule);background:transparent}.time-row:last-child{border-bottom:1px solid var(--rule)}.time-row dt{font-weight:600}.time-row dt small{display:block;font-size:.8125rem;line-height:1.4;font-weight:400;color:var(--muted)}.time-row dd{margin:0;font-size:1.125rem;font-weight:600;white-space:nowrap;font-variant-numeric:tabular-nums}.navigation{display:grid;grid-template-columns:1fr 1fr;position:fixed;bottom:0;left:0;right:0;background:var(--bg);border-top:1px solid var(--rule);padding:8px 20px max(8px,env(safe-area-inset-bottom));z-index:5}.navigation button{display:flex;flex-direction:column;gap:4px;align-items:center;border:none;border-radius:0;background:transparent;color:var(--muted);font-size:.875rem;min-height:58px;padding:2px 8px 8px;position:relative}.navigation button[aria-current]{color:var(--amber)}.navigation button[aria-current]::after{content:"";position:absolute;bottom:0;left:0;right:0;height:2px;background:var(--amber)}svg{width:24px;height:24px;fill:none;stroke:currentColor;stroke-width:1.8;stroke-linecap:round;stroke-linejoin:round}.skip-link{position:fixed;top:8px;left:8px;transform:translateY(-160%);padding:12px;background:var(--amber);color:var(--bg);z-index:20}.skip-link:focus{transform:none}.actions{display:flex;gap:8px;flex-wrap:wrap} @media(max-width:767px){.device-header{padding:12px 20px}.device-header h1{font-size:1.25rem}.device-header p,.device-header .date{font-size:.75rem}.next-time{font-size:min(3.875rem,25vw);margin-top:0;margin-bottom:8px}.readiness{line-height:1.25}.audio{margin-top:12px}.audio .feedback{margin-bottom:4px}.next-and-controls{padding-bottom:10px}.timetable h2{margin-bottom:4px}.time-row{padding:4px 0}} @media(max-width:350px){.device-header{align-items:flex-start;flex-direction:column;gap:4px}.device-header .date{font-size:.8125rem}.next-time{font-size:min(3.5rem,25vw)}.next-and-controls,.timetable{padding-left:16px;padding-right:16px}} @@ -16,3 +17,4 @@ html{scroll-padding-bottom:8rem}button,input,select,summary{scroll-margin-bottom .helper-link{display:inline-flex;align-items:center;min-height:44px;padding:9px 16px;border:1px solid var(--border);border-radius:8px;background:var(--surface);font-weight:600}#firmware-section p{margin-bottom:12px}#firmware-section .actions{margin-top:12px}.next-date{font-size:.875rem;color:var(--muted);margin-bottom:8px}.prayer-actions button:disabled{opacity:.55} select{overflow:hidden;text-overflow:ellipsis}.time-row dt{overflow-wrap:anywhere} label:has(>select){overflow:hidden}label:has(>select:focus){outline:3px solid var(--amber);outline-offset:3px} +@media(max-width:767px){.navigation:has(#quran-nav:not([hidden])){grid-template-columns:repeat(3,minmax(0,1fr))}#quran-view{padding:0 20px}}