Repository navigation
feat(audio-fx)!: add live rhythm evidence contracts - #858
Merged
Merged
Conversation
Bundle ReportChanges will decrease total bundle size by 20.87MB (-36.89%) ⬇️. This is within the configured threshold ✅ Detailed changes
Affected Assets, Files, and Routes:view changes for bundle: exo-full-iife-min-Exo-iifeAssets Changed:
Files in
view changes for bundle: exo-full-iife-Exo-iifeAssets Changed:
Files in
view changes for bundle: exojs-audio-fx-esmAssets Changed:
Files in
view changes for bundle: site-server-esmAssets Changed:
App Routes Affected:
|
Exoridus
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Beat events previously mixed grid timing with the current emission hop's accent, bar information was forced during ambiguity, and predictions could survive silence or a source change. This change separates matched onset evidence, pulse confidence, meter interpretation and forecast validity in the existing live pipeline.
Draft: the automatic meter candidate fails real-music qualification. On the frozen 60-recording comparison, its downbeat F1 after five seconds is 0.0000 versus 0.1520 before this PR. It emits no downbeats, with only 0.0473% time-weighted anchored meter coverage. Beat F1 remains 0.4533. The candidate must not be promoted as a qualified automatic detector or released on the strength of synthetic tests.
Implementation
BeatInfo.energyretains emission-hop novelty, while optionalonsetEnergyandonsetAudioTimedescribe the matched onset or null.beatConfidenceexcludes the legacy meter factor. Legacyconfidence,gridStabilityand lock thresholds retain their existing contract. The old bar tracker is isolated, not deleted.meterEstimateis an immutable view with nullableselected, at most two distinct-cycle alternatives, an explicit abstention reason, audio-clock evidence/validity and estimated/manual/convention origin. A selected model has pulse period, cycle, independently nullable phase and nullable notated denominator.meterKnownrequires a valid cycle and phase; automatic inference only considers three or four pulses and never infers a notated denominator.manualMetervalidates and snapshots cycles 1..32, optional phase and optional denominator. Changing it starts a new analysis epoch; configuration persists across resets, history does not. Disabling time-signature detection retains an explicit four-pulse counting convention. Eleven-pulse tests cover downbeats outside the eight-entry forecast horizon.scheduleexposes immutable epoch/revision, generation/evidence/expiry times, actual forecast pulse spacing, horizon, generation status and futurenextBeat/nextDownbeator null. Audio-clock expiry works without further port messages. Meter expiry removes bar labels while valid beat predictions can remain. Late accent finalization may revise predictions but does not emit a retrospective beat.reset(), source identity and input/frame gaps invalidate history and stale epochs.notifyDiscontinuity()handles sourceChange, seek, loop, starvation, explicitPause, resume, eof and reset. EOF/pause stops analysis until explicitly resumed/reset or a new source is assigned. Reentrant reset suppresses old follow-up bar events; lifetime counters survive analysis resets.Migration: automatic
onDownbeat/onBarStartno longer fire during acquisition, ambiguous phase/cycle or expired meter. UseonBeatfor pulse effects and inspectmeterEstimate.selected, phase and origin for bar-dependent UI. Legacy time-signature getters remain counting/display conventions. Signal seeks/loops/EOF on a shared source explicitly. Prediction callbacks update a revisioned schedule; they are not one guaranteed gameplay trigger per beat. README, guide, evaluation contract and generated API cover these changes.Model and quality evidence
The bounded candidate fits standardized log-relative power/band observations to repeating three-/four-position means against a common two-position null, with equal cycle priors and explicit phase abstention. Its fixed reference parameters are untrained. This is not a Krebs/madmom reproduction, an offline neural analyzer, or general 5/7/9/11-meter recognition.
A separate development manifest excludes the comparison recordings and exact audio duplicates: 70 training and 26 validation clips. A compact parameter-grid calibration had negative utility on both splits; a private learned Gaussian observation prototype selected abstention and supplied no qualified gain. Neither failed selection was adopted as a production winner.
After implementation and review were frozen at
2192ad19f15609b8625eaa2fc64acf612bff8913, 53 authored PCM controls were evaluated through the actual built minified worklet. The 47 identifiable cases yield canonical mir_eval 0.8.2 downbeat F1 after five seconds of 0.8823 versus 0.3053, and CMLt 0.8222 versus 0.0799. Six neutral/alias/out-of-model controls abstain. Slow isolated 50/60 BPM cases emit downbeats across every starting phase; both default-range 50 BPM cases still fail because of pulse-level selection. Correct phase after a synthetic cycle switch takes about 13.2/14.2 seconds; recovery after a break takes about 5.7 seconds. These are development controls, not musical generalization evidence.The subsequent fixed 60-recording GTZAN comparison uses 30 triple and 30 quadruple recordings, independent pinned beat/downbeat annotations, independently indexed audio MD5s, identical 48 kHz PCM and 128-sample blocks, default options, and no output-based retuning. Full and after-five-second all/provisional/locked precision, recall, F1, canonical CML/AML/Cemgil, abstention, phase decisions and censored cases are retained. Duplicate predictions are retained; five nonmonotonic beat timestamps occur in each version.
The paired downbeat F1 difference is -0.1520, fixed-seed 10,000-recording bootstrap 95% interval [-0.1970, -0.1138]. This comparison was observed in an earlier exploratory run and is not a fresh blind holdout. Artist/composition disjointness and two-annotator agreement have not been established. The musical gate fails independently of those limitations.
Frozen candidate worklet SHA256:
0289cc442d70fdb5a4013bb51c5e7b4de1f4dbdd3ef76640765799b00275bb36. Fixed selection SHA256:2a0b3523c14f5ce62b815e9d2c89f786b17bf1cdaf7b51e74d8ae3a243418aa9. Audio revision:d2146561ecc7df707d9e6b8318885fe6a39668a2; annotation revision:890407d158078527ab396b49fea3c8a83e5734ee.Engineering validation
Targeted owner/worklet, evidence, lifecycle, schedule, confidence, cost and full-PCM tests pass. Ten native Chromium AudioWorklet cases pass at 44.1/48 kHz, including actual slow-tempo beat/downbeat/bar-start emissions, reset and silence expiry. Independent full-branch review findings were reproduced red, corrected and verified without remaining findings.
Audio package typechecks/builds, changed-file ESLint/Prettier, source hygiene, 566 file-symbol checks, generated API synchronization, 46 guide snippet files and Site typechecking pass. The full package qualification chain passes: size, exports, declaration imports/semantics, version/release consistency, all 16 pack/publint checks and external consumers. Full minified gzip size is 646.01 kB. Its cap is explicitly adjusted from 643 to 647 kB for the added public contracts and bounded worklet; other budgets are unchanged. This budget change is intentional, not a claim that the original cap passed.
The unchanged normal pre-push hook passed every selected lane: unit (15,761 tests passed, one existing opt-in rendering sweep skipped; separate allocation13 and physics2 passed), WebGL, WebGPU, audio (38 tests/10 files), tilemap, benchmark and Site/examples smoke. Remote CI39514695902 also passed completely on2192ad19, including package qualification and verdict. No merge or release is proposed.
Native runtime qualification: measured, target not met
A subsequent isolated native run used600seconds per version,48kHz/128frames, first300seconds quiet and next300seconds with64full-screen WebGL2draws per RAF at512x512. Platform: Chromium153.0.8010.12, Windows build26200, Ryzen7 3700X, ANGLE/Vulkan SwiftShader software renderer. This is one sequential pair with tracing/observer overhead, not a hardware-GPU or paired per-callback causal comparison.
Both runs have complete start/graphics/end trace markers, no trace loss, zero processor errors and1201beat events;224985/224997callbacks were captured. Each has one startup callback over the2.667ms quantum; neither has one after30seconds. No audible-output dropout guarantee is inferred from a detector with no audio output.
The proposed totalP99<=0.667ms/maximum<=1.333ms budget fails in both versions. Observed sequential quiet P99 difference is+0.335ms and maximum difference+0.726ms, so the proposed added0.10/0.25ms budget is not established either. Graphics P99 difference is+0.002ms and maximum difference+0.692ms. Thermal/OS/JIT/tracing effects have not been separated from algorithm cost. These targets were not loosened to call the run a pass.
Audio-thread top-level GC collections78/101, total pauses21.205/31.138ms, maximum pause1.210/3.593ms. Post30s after-GC heap slopes are51.7/137.6B/s. Those JS-heap samples establish neither allocation-free operation, stable total memory nor an unbounded leak; the128KiB additional-total-memory target remains unqualified. Unique retained ArrayBuffers increase40000->44374B(+4374B), a partial inventory only.
State transport is18.75Hz and average context-clock state delivery lag9.831/9.791ms. Native state+beat JSON-size proxy grows25.09->40.84kB/s(+15.38KiB/s). A separate60s functional payload inventory after timing includes every message type and observes+14.73KiB/s mean/+15.34KiB/s steady JSON-size proxy. This is not measured structured-clone wire size or a bound for arbitrary inputs. Callback allocation rates and total peak memory were not measured. The PR remains Draft for failed musical qualification and unmet/unqualified native budgets despite green engineering CI.
Candidate storage optimization
Follow-up head
ebe7a3379d6c1c974a83e831319178704fc999fdreplaces allocating ACF peak/filter/scored lists and stable sorts with independent three-slot stable/fast candidate pools, active counts and an internal scorer. Final-score/raw-peak/lag tie ordering and independent public snapshots are preserved. Meter parameters, tempo behavior, defaults and transport cadence are unchanged.All 60 existing music recordings reproduce exactly the same 37,817 public messages and 3,542 beats against frozen2192; this is behavioral equivalence, not a successful meter qualification. Identity/reset tests reproduced the original storage failures and pass after the change. Added tie/snapshot and canonical candidate parity tests also pass, alongside full-PCM regressions, owner/worklet neighbors, package typechecks/builds and ten native AudioWorklet cases. The normal pre-push hook passed all selected static, unit and audio lanes. Full minified gzip size is646.23kB under the unchanged647kB cap.
In the targeted Node/V8 sampling workload (128-byte interval including collected objects,12s warmup/48s audio, preallocated driver inputs and no-op postMessage), scorer-attributed sample weight decreases1,756,432->2,048B. This is sampled attribution rather than exact total allocation bytes; state/meter/message allocation remains. The optimized built worklet SHA256 is
0118006b8a82f99efee0747f0d1cd63814eb7fd76a0bbbe2abfa8d1956810192.The frozen native A/B/B/A comparison is complete: four300s blocks, each half quiet/graphics, total600s per version. A is2192 and B is this storage refactor; this is not the pre-PR baseline and does not replace the historical continuous600s traces above. Chromium153/Windows26200/Ryzen7 3700X, SwiftShader software graphics and tracing/JSON observers remain measurement limitations. No concurrent local builds/tests were run during timing.
Across600s/version, GC collections are118->114, total GC pause37.428->34.042ms, largest observed GC pause1.730->1.134ms. These are modest observed reductions in one ABBA sequence with only two blocks/version, not an established general GC benefit. There is no robust P99 gain: reference quiet P99 itself varies0.946->1.345ms, and B2 graphics is slower. Quantiles are reported per run, not averaged into a pooled P99. Overall P99 is1.401/1.409/1.817/1.429ms; startup-inclusive maxima are3.840/4.784/4.996/5.303ms. The proposed absolute runtime budgets remain failed; added-cost and total-memory budgets remain unqualified.
All four blocks have valid boundaries, no trace loss, zero processor errors and601beats each. Every block has one startup callback above2.667ms; none after30s. This is callback-wall-time evidence, not proof of audible-output deadline safety. Post30s after-GC heap slopes are313/312/291/364B/s; two five-minute blocks do not prove continuous ten-minute stability or stable total memory. All-message JSON-size proxy remains about40.87kB/s and state rate18.75Hz.
CI39528848071 passes completely on ebe7a33, including package verification and verdict. The normal local hook passed1,439audio-fx unit tests and38native browser audio tests. Musical qualification remains failed and the PR remains Draft. Blink consumer migration is prepared only; no Blink source or dependency upgrade is included. Audio-to-photon latency remains unmeasured.