diff --git a/package.json b/package.json index 846387d0c..e9bfdc83d 100644 --- a/package.json +++ b/package.json @@ -224,7 +224,7 @@ }, { "path": "dist/exo.full.iife.min.js", - "limit": "643 KB", + "limit": "647 KB", "gzip": true }, { diff --git a/packages/exojs-audio-fx/README.md b/packages/exojs-audio-fx/README.md index 9ae735435..114d4d3a4 100644 --- a/packages/exojs-audio-fx/README.md +++ b/packages/exojs-audio-fx/README.md @@ -44,6 +44,18 @@ Effects, analyser nodes, detectors, and playing voices have distinct lifetimes. Beat detection estimates tempo and phase. Polling windows such as `justBeat` are not one-shot events, and asynchronous messages are not hard real-time delivery. A rhythm-game scoring timeline needs an explicit authoritative timing design. Analysis values are not a calibrated acoustic loudness measurement. +Assigning a different `BeatDetector.source` clears analysis history and pending predictions. Call `detector.reset()` after a seek or another playback discontinuity on the same source; reassigning that same source preserves continuity. Disconnected inputs and gaps in the context-frame stream also reset analysis. Recorded silence is processed as audio and expires predictions without assuming a source change. Use `notifyDiscontinuity('eof')` or `notifyDiscontinuity('explicitPause')` to stop analysis explicitly; `resume`, `reset()` or another source identity starts a new epoch. Seek, loop and starvation notifications clear history while preserving an explicit stop. Changing manual context also preserves that stop. + +`predictionValidUntil` is the audio-context time at which a retained prediction schedule must be discarded, or 0 when unavailable. Without renewed onset support, validity lasts two tracked beat periods plus one FFT window. `lookahead` becomes empty after expiry, and unavailable `nextBeatTime` and `nextDownbeatTime` return 0. Later times in the eight-beat extrapolation remain conditional; retaining an array does not extend its validity. `onBeatPredicted` announces each received revision at most once and does not coalesce queued revisions. The immutable `schedule` view adds epoch/revision, generation/evidence times, conditional horizon, pulse period and the trust status at generation. It is null when expired. Its `nextBeat` and `nextDownbeat` are future entries or null, including when a manual boundary lies beyond the horizon. A retained snapshot containing downbeats expires with its meter interpretation; a fresh view may retain beat-only predictions until their separate deadline. + +`meterEstimate` is an immutable view with `selected`, at most two `alternatives`, an uncalibrated `score`, `reason`, `origin` and audio-context validity times. `selected: null` means no supported cycle; `selected.phasePulses: null` means a cycle without an anchored downbeat. The selected model contains `cyclePulses`, `pulsePeriodSeconds`, `phasePulses` and `denominator`. Automatic detection considers three/four pulses and never infers a notated denominator. `meterKnown` requires both a valid cycle and phase. Unknown or expired phase suppresses automatic downbeats, bar-start events and lookahead downbeat flags; beat effects can continue. + +Use `manualMeter: { cyclePulses: 3, phasePulses: 0, denominator: 4 }` to provide counting context, anchored at the first tracked beat after reset (`origin: 'manual'`). Omit phase to keep the cycle unanchored. The runtime `manualMeter` setter snapshots and validates its input, resets analysis when changed and accepts `null` to resume automatic detection. Configuration survives reset/source changes; the phase anchor is established anew. `enableTimeSignatureDetection: false` instead supplies a fixed four-beat convention (`origin: 'convention'`). Manual cycles support 1..32 pulses; an upcoming boundary outside the eight-entry lookahead remains unavailable rather than becoming the next beat. + +`barPosition`, `barLength` and `timeSignature` are legacy display values when meter is unknown. Their denominator defaults to the convention 4 unless provided manually. `beatConfidence` separates the pulse score from the legacy meter factor; `confidence` and its lock behavior retain that factor, and `gridStability` remains its legacy alias. None is a calibrated probability. `BeatInfo.energy` retains raw novelty from the emission hop; the separate `onsetEnergy` and `onsetAudioTime` describe the associated onset or are null without a match. Cached rhythm getters expire against the audio-context clock even without another worklet message. + +The [meter model and interpretation contract](docs/meter-estimation.md) describes the compact candidate and its limits. Automatic cycle/downbeat quality is experimental; a locked tempo does not establish musical bar accuracy. + ## Documentation [Audio basics](https://exoridus.github.io/ExoJS/en/guide/audio/audio-basics/) · [Effects and routing](https://exoridus.github.io/ExoJS/en/guide/audio/audio-effects/) · [Beat detection](https://exoridus.github.io/ExoJS/en/guide/audio/beat-detection/) · [Audio-reactive visuals](https://exoridus.github.io/ExoJS/en/guide/audio/audio-reactive-visualization/) · [AudioAnalyser API](https://exoridus.github.io/ExoJS/en/api/audio-analyser/) diff --git a/packages/exojs-audio-fx/docs/meter-estimation.md b/packages/exojs-audio-fx/docs/meter-estimation.md new file mode 100644 index 000000000..18b1c6178 --- /dev/null +++ b/packages/exojs-audio-fx/docs/meter-estimation.md @@ -0,0 +1,19 @@ +# Meter interpretation + +The live detector keeps pulse tracking and meter interpretation separate. FFT, mel novelty, two tempo windows and the PLL still determine beats. A compact experimental consumer evaluates three/four-pulse cycles on the tracked grid. It is not a joint tempo/bar-pointer decoder or a neural downbeat model, and it does not identify notated denominators, arbitrary groupings or simultaneous meters. + +Each matched onset retains raw novelty and normalized strength with its identity and sample position. Its accent is the maximum PCM power over the following FFT window. Three positive-novelty band sums are accumulated from the existing mel pass over the same window. A later unrelated emission hop cannot supply that onset's accent. Missing onsets contribute no fabricated feature vector. New meter fitting and band accumulation are disabled for manual/fixed counting; the legacy tracker still contributes to legacy confidence and lock behavior. + +The consumer retains at most 32 beat observations. It transforms each available channel with `log1p(value / windowMean)` and standardizes channels with sufficient contrast. Three/four-position repeating means are compared against a common two-position null, so uniform accents or a two-pulse alias do not establish a bar. Both cycle models use the same observations and channel priors; an extra-mean penalty limits the advantage of the larger model. Channel evidence is marginalized so a noisy band cannot veto a consistent attack channel. The highest supported cycle and its phase anchor are evaluated separately: two equally plausible anchor positions leave phase unknown even when repetition supports a cycle. + +This is an untrained periodicity/accent reference with explicit abstention. Its scores are heuristic, and its thresholds do not establish calibrated confidence or representative real-music accuracy. Synthetic correctness tests prove the tested associations, phase rules, transitions and event timing; they do not qualify the automatic method for a musical application. A release-quality automatic-method decision also requires independent annotated recordings and native runtime measurements. No model is selected merely because it ranks first among unsuccessful candidates. + +Support lasts two tracked pulse periods plus one FFT finalization window after the latest supporting observation. This covers one missing pulse and its successor's complete attack without a tempo-independent timeout. Expiry removes model/phase availability, including in cached main-thread getters. It does not backdate a newly inferred downbeat event. Lookahead revisions can update after feature finalization without emitting another beat. + +`MeterEstimate.selected` is null without a supported cycle. A selected model's `phasePulses` can independently be null. Alternatives describe competing cycle models rather than simultaneous meters. Reasons distinguish acquisition, insufficient evidence, low contrast, ambiguous cycle, ambiguous phase, availability and expiry. `meterKnown` requires a current selection and non-null phase; only then may downbeats and bar starts be emitted. Legacy display counters remain readable while meter is unknown. + +Manual context is caller-owned musical/counting knowledge, copied at assignment. Phase refers to the first tracked beat of the new analysis epoch, not an automatically found acoustic bar. Configuration persists across reset/source changes; observations and anchors do not. Fixed four-pulse counting has origin `convention`; manual context has origin `manual`; audio-derived hypotheses have origin `estimated`. A score of 1 for a chosen rule describes that rule, not certainty about the music. + +A forecast revision captures generation time, consumed-evidence time, grid period and provisional/locked status when generated. Later state delivery does not rewrite that metadata. The public immutable `schedule` view bounds the lifetime of all included interpretations; a snapshot containing downbeat flags cannot outlive meter support. A fresh beat-only view can remain available until pulse support expires. Regular state transport runs no faster than 20 Hz, rounded to whole analysis hops; beat, tempo and immediate reset notices remain separate. + +Explicit EOF/pause notifications invalidate the epoch and stop audio consumption by the detector. Resume, reset or another source identity starts fresh. Notifications for seek, loop or starvation preserve an existing stop, as does manual reconfiguration. The tap does not infer these owner-level playback events from recorded silence. diff --git a/packages/exojs-audio-fx/docs/mir-evaluation.md b/packages/exojs-audio-fx/docs/mir-evaluation.md index 1cde2fd84..23b659ec4 100644 --- a/packages/exojs-audio-fx/docs/mir-evaluation.md +++ b/packages/exojs-audio-fx/docs/mir-evaluation.md @@ -66,3 +66,13 @@ The existing provisional path improves first delivery by 1.440 seconds on 120 BP A separate low-latency onset path is deferred. This evaluation establishes an existing startup benefit but does not demonstrate an incremental benefit from another detector, and provisional beats still have substantial posting delay and false positives on some inputs. No onset API, detector algorithm, threshold, or worklet option is changed by the harness. Any future candidate should be compared against these unchanged fixtures and a broader independently annotated music corpus before expanding the public API. The sandbox measures deterministic algorithm behavior, not main-thread message delivery, speaker latency, animation-frame alignment, browser scheduling, CPU cost, or real-time deadline reliability. It covers one sample rate and synthetic mono signals. The kit is a simplified synthesis, swing annotations reflect an explicit beat convention, and ramp beat positions use the existing generator's forward-step approximation. Results do not establish music-genre generalization, calibrated confidence, downbeat accuracy, or rhythm-game scoring suitability. + +## Pulse-cycle evidence + +Run the named `test/worklets/meter-evidence.test.ts` and `test/worklets/beat-detector-meter.test.ts` files in the same package project for the automatic cycle model. The first tests finalized onset power, every initial three-/four-pulse phase across gains, missing observations, skipped grid pulses, validity through slow pulses, two-pulse ambiguity, equal accents and seeded random accent noise. The second passes authored PCM through the production FFT/onset/tempo/PLL pipeline without preparing a meter state: 50/60 BPM event regressions and 90/120/180 BPM controls, three-/four-pulse accents, quiet attacks with another spectrum, equal accents, both cycle-switch directions and explicit four-beat counting. Authored downbeat times are independent of the detector's output. Cycle and phase checks after ten seconds describe these controls; they do not establish zero startup delay or general music accuracy. + +The named `test/browser/beat-detector.audio.test.ts` file in `browser-audio-chromium` additionally exercises three-/four-pulse and equal-accent audio at 44.1 and 48 kHz, reset acknowledgement and expiry during silence in a native AudioWorklet. These functional tests do not measure render deadlines, allocations or performance under graphics contention. + +Unknown is a normal result, including when a beat grid is locked. The model requires at least twelve finalized matched onset observations, sufficient contrast against equal accents and separation between three- and four-pulse explanations. Its bounded history can delay recognition after a cycle change. The score is uncalibrated, and a pulse cycle does not infer a notated denominator. Representative independently annotated real-music meter/downbeat evaluation and controlled native timing/allocation qualification remain required before claiming general reliability. + +Prediction retention follows the tracked pulse interval: two periods plus one FFT window after the last supporting observation. The finalization allowance keeps an established cycle available while the next attack is being finalized, including at 50 BPM. One missing pulse can be bridged; without renewed support, predictions still expire. Explicit reset and source/input discontinuities invalidate immediately. diff --git a/packages/exojs-audio-fx/src/BeatAnalysisDiscontinuity.ts b/packages/exojs-audio-fx/src/BeatAnalysisDiscontinuity.ts new file mode 100644 index 000000000..6c100630a --- /dev/null +++ b/packages/exojs-audio-fx/src/BeatAnalysisDiscontinuity.ts @@ -0,0 +1 @@ +export type BeatAnalysisDiscontinuity = 'sourceChange' | 'seek' | 'loop' | 'starvation' | 'explicitPause' | 'resume' | 'eof' | 'reset'; diff --git a/packages/exojs-audio-fx/src/BeatDetector.ts b/packages/exojs-audio-fx/src/BeatDetector.ts index 5592e1bf6..7d03605f2 100644 --- a/packages/exojs-audio-fx/src/BeatDetector.ts +++ b/packages/exojs-audio-fx/src/BeatDetector.ts @@ -10,6 +10,9 @@ import { } from '@codexo/exojs'; import { AudioTap } from './AudioTap'; +import type { BeatAnalysisDiscontinuity } from './BeatAnalysisDiscontinuity'; +import type { BeatSchedule, UpcomingBeat } from './beatScheduleTypes'; +import type { ManualMeter, MeterEstimate } from './meterTypes'; import { ReadyGate } from './ReadyGate'; import beatDetectorWorkletSource from './worklets/beat-detector.worklet.ts?worklet'; @@ -74,10 +77,12 @@ export interface BeatDetectorOptions { /** Number of mel filterbank bands. Default 24. */ melBands?: number; /** - * When true (default), the worklet runs parallel 3/4 and 4/4 posteriors and - * switches active time signature via hysteresis. Set false to lock to 4/4. + * When true (default), estimates three- or four-pulse accent cycles and leaves + * ambiguous meter unknown. Set false for a fixed four-beat counting convention. */ enableTimeSignatureDetection?: boolean; + /** Overrides automatic meter with explicit counting context; changing it resets analysis. */ + manualMeter?: ManualMeter | null; /** * Optional initial source. Equivalent to constructing then assigning * `detector.source = value`; provided for ergonomic one-shot construction. @@ -106,34 +111,34 @@ export interface BeatInfo { audioTime: number; /** BPM at this beat. */ tempo: number; - /** Confidence 0..1. */ + /** Legacy combined confidence, 0..1; not a calibrated probability. */ confidence: number; + /** Pulse confidence without the meter factor, 0..1 and uncalibrated. */ + beatConfidence?: number; /** Phase within beat (0 = start). */ beatPhase: number; - /** Novelty/onset strength at the beat. */ + /** Raw spectral novelty of the emission hop, which may follow the matched onset. */ energy: number; + /** Matched onset novelty, or null when this beat has no associated onset. */ + onsetEnergy?: number | null; + /** Matched onset time on the audio-context clock, or null without a match. */ + onsetAudioTime?: number | null; /** Is this beat the first in a bar? */ isDownbeat: boolean; + /** True when the cycle and its phase are available; false leaves beatInBar as display counting only. */ + meterKnown?: boolean; /** Beat position within the bar (1..N). */ beatInBar: number; /** * Detection trust level: * - `'provisional'` - an early, low-latency beat emitted before the tempo grid * is fully settled. Good for visual reactivity (a "blink"); may be revised. - * - `'locked'` - the tempo grid is settled and trustworthy. Safe for - * sync-critical use. Each stable segment yields exactly one provisional→locked - * transition. + * - `'locked'` - the tempo grid passed the detector's settling heuristic. + * This does not establish meter accuracy or calibrated confidence. */ status: 'provisional' | 'locked'; } -export interface UpcomingBeat { - audioTime: number; - tempo: number; - isDownbeat: boolean; - beatInBar: number; -} - export interface BarInfo { audioTime: number; tempo: number; @@ -161,10 +166,45 @@ export interface BandEnergy { const workletName = 'exojs-beat-detector'; +const snapshotManualMeter = (value: ManualMeter | null): ManualMeter | null => { + if (value === null) { + return null; + } + + const { cyclePulses } = value; + const phasePulses = value.phasePulses ?? null; + const denominator = value.denominator ?? null; + + if ( + !Number.isInteger(cyclePulses) || + cyclePulses < 1 || + cyclePulses > 32 || + (phasePulses !== null && (!Number.isInteger(phasePulses) || phasePulses < 0 || phasePulses >= cyclePulses)) || + (denominator !== null && + (!Number.isInteger(denominator) || denominator < 1 || denominator > 128 || (denominator & (denominator - 1)) !== 0)) + ) { + throw new RangeError('Invalid manual meter cycle, phase or denominator'); + } + + return Object.freeze({ cyclePulses, phasePulses, denominator }); +}; + +const acquiringMeter: MeterEstimate = Object.freeze({ + selected: null, + alternatives: Object.freeze([]), + score: 0, + reason: 'acquiring', + origin: 'estimated', + evidenceThroughAudioTime: 0, + validUntilAudioTime: 0, +}); +const acquiringManualMeter: MeterEstimate = Object.freeze({ ...acquiringMeter, origin: 'manual' }); +const acquiringConventionMeter: MeterEstimate = Object.freeze({ ...acquiringMeter, origin: 'convention' }); + /** * Real-time tempo + beat tracker. Splits work between the audio-rendering * thread (an AudioWorklet that runs onset detection, tempogram analysis, - * and parallel 3/4 and 4/4 posterior estimation) and the main thread (this + * and three-/four-pulse accent-cycle estimation) and the main thread (this * class - receives beats, fires Signals, handles configuration and source * routing). * @@ -182,7 +222,8 @@ const workletName = 'exojs-beat-detector'; * then promote to `status: 'locked'` once the tempo grid is trustworthy - a * single {@link BeatDetector.onBeat} signal carries both via {@link BeatInfo.status}. * Set `emitProvisionalBeats: false` to receive only locked beats. Time-signature - * detection (3/4 vs 4/4) is on by default; lock to 4/4 by setting + * detection is on by default; ambiguous meter leaves beats available without + * downbeats. Use a fixed four-beat convention with * `enableTimeSignatureDetection: false`. */ export class BeatDetector { @@ -192,13 +233,11 @@ export class BeatDetector { public readonly onDownbeat = new Signal<[BeatInfo]>(); public readonly onBarStart = new Signal<[BarInfo]>(); /** - * Dispatched with the first entry of each new {@link lookahead} schedule. + * Dispatched with the first future entry of each new {@link lookahead} schedule. * - * The detector computes a schedule once per tracked beat, so a listener hears - * about every upcoming beat once, not on every state update. A newer schedule - * supersedes the times an earlier one predicted for the same beats, and one - * replaced before the main thread received it is skipped. Nothing is - * dispatched while the detector coasts through silence. + * Each revision is announced at most once; queued revisions are not coalesced. + * An expired schedule or one without a future entry produces no notice. + * A newer revision supersedes earlier predictions for the same beats. */ public readonly onBeatPredicted = new Signal<[UpcomingBeat]>(); @@ -211,6 +250,7 @@ export class BeatDetector { private readonly _tap = new AudioTap(); private readonly _ready = new ReadyGate('The beat detector was destroyed before it became ready.'); private _destroyed = false; + private _epoch = 0; private readonly _onAudioContextReady = (ctx: AudioContext): void => { onAudioContextReady.remove(this._onAudioContextReady); this._setup(ctx); @@ -221,6 +261,7 @@ export class BeatDetector { private _beatPhase = 0; private _nextBeatTime = 0; private _confidence = 0; + private _beatConfidence = 0; private _phaseConfidence = 0; private _gridStability = 0; private _analysisTime = 0; @@ -234,7 +275,16 @@ export class BeatDetector { private _timeSignature: TimeSignature = { numerator: 4, denominator: 4 }; private _nextDownbeatTime = 0; private _lookahead: readonly UpcomingBeat[] = Object.freeze([]); + private _beatOnlyLookahead: readonly UpcomingBeat[] = Object.freeze([]); private _lookaheadRevision = 0; + private _scheduleBase: Omit | null = null; + private _scheduleView: BeatSchedule | null = null; + private _stopReason: 'eof' | 'explicitPause' | null = null; + private _predictionValidUntil = 0; + private _meterEstimate: MeterEstimate = acquiringMeter; + private _expiredMeterEstimate: MeterEstimate = acquiringMeter; + private readonly _emptyTempoCandidates: readonly TempoCandidate[] = Object.freeze([]); + private readonly _emptyLookahead: readonly UpcomingBeat[] = Object.freeze([]); /** * Half-life in seconds for the {@link pulse} envelope. Mutable; default 0.15. @@ -261,6 +311,7 @@ export class BeatDetector { emitProvisionalBeats: options?.emitProvisionalBeats ?? true, melBands: options?.melBands ?? 24, enableTimeSignatureDetection: options?.enableTimeSignatureDetection ?? true, + manualMeter: snapshotManualMeter(options?.manualMeter ?? null), // Visual-state options aren't part of worklet config; cached in the public // fields below but kept in the Required<> shape for type completeness. source: options?.source ?? null, @@ -272,6 +323,7 @@ export class BeatDetector { this.pulseHalfLife = this._options.pulseHalfLife; this.barPulseHalfLife = this._options.barPulseHalfLife; this.justBeatWindow = this._options.justBeatWindow; + this._clearState(); if (isAudioContextReady()) { this._setup(getAudioContext()); @@ -293,9 +345,39 @@ export class BeatDetector { } public set source(value: BeatDetectorSource) { + if (this._destroyed || value === this._tap.source) { + return; + } + + this.notifyDiscontinuity('sourceChange'); this._tap.source = value; } + public get manualMeter(): ManualMeter | null { + return this._options.manualMeter; + } + + /** Snapshots the value and resets analysis. An identical context preserves continuity. */ + public set manualMeter(value: ManualMeter | null) { + if (this._destroyed) { + return; + } + + const next = snapshotManualMeter(value); + const current = this._options.manualMeter; + + if ( + next?.cyclePulses === current?.cyclePulses && + next?.phasePulses === current?.phasePulses && + next?.denominator === current?.denominator + ) { + return; + } + + this._options.manualMeter = next; + this.notifyDiscontinuity(this._stopReason ?? 'reset'); + } + // ----------------------------------------------------------------------- // Ready promise // ----------------------------------------------------------------------- @@ -315,38 +397,45 @@ export class BeatDetector { // ----------------------------------------------------------------------- public get tempo(): number { - return this._tempo; + return this._predictionsValid() ? this._tempo : 0; } public get beatPhase(): number { - return this._beatPhase; + return this._predictionsValid() ? this._beatPhase : 0; } + /** Next predicted future beat on the audio-context clock, or 0 without a valid prediction. */ public get nextBeatTime(): number { - return this._nextBeatTime; + if (this._nextBeatTime <= 0 || !this._predictionsValid()) { + return 0; + } + + const now = getAudioContext().currentTime; + + return this._nextBeatTime > now ? this._nextBeatTime : (this._lookahead.find(beat => beat.audioTime > now)?.audioTime ?? 0); } public get confidence(): number { - return this._confidence; + return this._predictionsValid() ? this._confidence : 0; } /** - * How well recent onsets support the beat grid's phase, 0 to 1, or 0 before - * the grid locks. - * - * Distinct from {@link confidence}, which reports how sure the detector is of - * the tempo. A metronomic loop at a wrong-by-an-octave tempo reads high - * confidence and high phase confidence; a rubato passage at a known tempo - * reads high confidence and low phase confidence. Gate anything that must - * land exactly on the beat - a scored hit, a quantised trigger - on this - * rather than on {@link confidence}. + * Pulse-grid confidence without the legacy meter factor, 0..1, or 0 without + * current support. Available during provisional tracking and uncalibrated; + * it does not change the legacy lock threshold or establish meter accuracy. */ + public get beatConfidence(): number { + return this._predictionsValid() ? this._beatConfidence : 0; + } + + /** Current onset support for the tracked phase, 0..1 and uncalibrated. */ public get phaseConfidence(): number { - return this._phaseConfidence; + return this._predictionsValid() ? this._phaseConfidence : 0; } + /** Legacy alias of confidence, not an independent measure of phase stability. */ public get gridStability(): number { - return this._gridStability; + return this._predictionsValid() ? this._gridStability : 0; } /** @@ -389,36 +478,106 @@ export class BeatDetector { } public get tempoCandidates(): readonly TempoCandidate[] { - return this._tempoCandidates; + return this._predictionsValid() ? this._tempoCandidates : this._emptyTempoCandidates; } // ----------------------------------------------------------------------- // Stage 2 state accessors // ----------------------------------------------------------------------- + /** Display counting only while {@link meterKnown} is false. */ public get barPosition(): number { return this._barPosition; } + /** Display cycle length; use {@link meterEstimate} for an available estimate. */ public get barLength(): number { return this._barLength; } + /** Legacy display counting; the denominator is manual or defaults to the convention 4. */ public get timeSignature(): TimeSignature { return this._timeSignature; } + /** + * Immutable interpretation view, including reasons and competing models while + * abstaining. A null selected model means no cycle; a null selected phase + * means no downbeat anchor. The notated denominator is never inferred. + */ + public get meterEstimate(): MeterEstimate { + const estimate = this._meterEstimate; + + return estimate.validUntilAudioTime > 0 && (!this._predictionsValid() || estimate.validUntilAudioTime <= getAudioContext().currentTime) + ? this._expiredMeterEstimate + : estimate; + } + + public get meterKnown(): boolean { + const selected = this.meterEstimate.selected; + + return selected !== null && selected.phasePulses !== null; + } + + /** Current immutable conditional forecast, or null after expiry/reset. Future entries are checked at access time. */ + public get schedule(): BeatSchedule | null { + if (!this._predictionsValid() || !this._scheduleBase) { + return null; + } + + const beats = this.lookahead; + const now = getAudioContext().currentTime; + const nextBeat = beats.find(beat => beat.audioTime > now) ?? null; + const nextDownbeat = beats.find(beat => beat.isDownbeat && beat.audioTime > now) ?? null; + const validUntilAudioTime = this.meterKnown + ? Math.min(this._scheduleBase.validUntilAudioTime, this.meterEstimate.validUntilAudioTime) + : this._scheduleBase.validUntilAudioTime; + const view = this._scheduleView; + + if (view?.beats === beats && view.nextBeat === nextBeat && view.nextDownbeat === nextDownbeat) { + return view; + } + + this._scheduleView = Object.freeze({ ...this._scheduleBase, validUntilAudioTime, beats, nextBeat, nextDownbeat }); + + return this._scheduleView; + } + + /** Next predicted future bar start on the audio-context clock, or 0 when unavailable. */ public get nextDownbeatTime(): number { - return this._nextDownbeatTime; + if (this._nextDownbeatTime <= 0 || !this.meterKnown) { + return 0; + } + + const now = getAudioContext().currentTime; + + return this._nextDownbeatTime > now + ? this._nextDownbeatTime + : (this._lookahead.find(beat => beat.isDownbeat && beat.audioTime > now)?.audioTime ?? 0); + } + + /** + * Audio-context time when the current prediction schedule must be discarded. + * Returns 0 without a current schedule. Later extrapolated beat times remain + * conditional on the rhythm continuing; this is not a delivery deadline. + */ + public get predictionValidUntil(): number { + return this._predictionsValid() ? this._predictionValidUntil : 0; } /** * The next eight beats, extrapolated from the last tracked beat at the current - * tempo and numbered from the bar position that follows it. Predictions, not - * analysed audio: a tempo change or break is not anticipated. + * tempo and numbered from the bar position that follows it. Entries may have + * passed since delivery. Predictions are conditional on the rhythm continuing; + * a tempo change or break is not anticipated. Empty after prediction expiry. + * Unknown or expired meter clears downbeat flags while retaining beat times. */ public get lookahead(): readonly UpcomingBeat[] { - return this._lookahead; + if (!this._predictionsValid()) { + return this._emptyLookahead; + } + + return this.meterKnown ? this._lookahead : this._beatOnlyLookahead; } // ----------------------------------------------------------------------- @@ -430,7 +589,7 @@ export class BeatDetector { * and {@link tempo}. Returns 0 when the detector hasn't locked yet. */ public get secondsSinceLastBeat(): number { - if (this._tempo === 0) { + if (this.tempo === 0) { return 0; } @@ -443,7 +602,7 @@ export class BeatDetector { * with a single multiplication: `sprite.scale = 1 + clock.pulse * 0.3`. */ public get pulse(): number { - if (this._tempo === 0) { + if (this.tempo === 0) { return 0; } @@ -456,12 +615,13 @@ export class BeatDetector { * each bar (e.g. brighter flash on "1" vs "2,3,4"). */ public get barPulse(): number { - if (this._tempo === 0 || this._barLength === 0) { + const selected = this.meterEstimate.selected; + + if (this.tempo === 0 || selected?.phasePulses == null || this._nextBeatTime <= 0) { return 0; } - const secondsPerBeat = 60 / this._tempo; - const lastDownbeat = this._nextDownbeatTime - this._barLength * secondsPerBeat; + const lastDownbeat = this._nextBeatTime - (selected.phasePulses + 1) * selected.pulsePeriodSeconds; const elapsed = Math.max(0, getAudioContext().currentTime - lastDownbeat); return Math.pow(0.5, elapsed / this.barPulseHalfLife); @@ -473,7 +633,7 @@ export class BeatDetector { * sample retrigger). Default window 30ms covers a typical 60fps frame. */ public get justBeat(): boolean { - return this._tempo > 0 && this.secondsSinceLastBeat < this.justBeatWindow; + return this.tempo > 0 && this.secondsSinceLastBeat < this.justBeatWindow; } /** @@ -489,21 +649,67 @@ export class BeatDetector { return 0; } - return (this._beatPhase * division) % 1; + return (this.beatPhase * division) % 1; } // ----------------------------------------------------------------------- // Lifecycle // ----------------------------------------------------------------------- + /** + * Discards rhythm history and pending predictions. Call after a seek, loop + * discontinuity or explicit playback interruption on the same source. + * Assigning a different source resets automatically. + */ + public reset(): void { + this.notifyDiscontinuity('reset'); + } + + /** + * Discards current rhythm evidence immediately. EOF and explicit pause stop + * analysis until resume, reset or a source identity change. Loops reset phase. + * Manual configuration persists; the next epoch establishes a new anchor. + */ + public notifyDiscontinuity(reason: BeatAnalysisDiscontinuity): void { + if (this._destroyed) { + return; + } + + this._epoch++; + + if (reason === 'eof' || reason === 'explicitPause') { + this._stopReason = reason; + } else if (reason === 'resume' || reason === 'reset' || reason === 'sourceChange') { + this._stopReason = null; + } + + this._clearState(); + this._workletNode?.port.postMessage({ + type: 'reset', + epoch: this._epoch, + manualMeter: this._options.manualMeter, + stopped: this._stopReason !== null, + }); + } + public destroy(): void { + if (this._destroyed) { + return; + } + onAudioContextReady.remove(this._onAudioContextReady); this._destroyed = true; + this._clearState(); this._tap.destroy(); // Disconnecting alone leaves the processor rendering every quantum until // the AudioContext closes; the message lets it return false and be released. this._workletNode?.port.postMessage({ type: 'destroy' }); + + if (this._workletNode) { + this._workletNode.port.onmessage = null; + } + this._workletNode?.disconnect(); this._workletNode = null; this._ready.abort(); @@ -533,6 +739,7 @@ export class BeatDetector { numberOfInputs: 1, numberOfOutputs: 0, processorOptions: { + epoch: this._epoch, fftSize: opts.fftSize, hopSize: opts.hopSize, minBpm: opts.minBpm, @@ -544,6 +751,8 @@ export class BeatDetector { stableTempoWindowSec: opts.stableTempoWindowSec, acfIntervalHops: opts.acfIntervalHops, enableTimeSignatureDetection: opts.enableTimeSignatureDetection, + manualMeter: opts.manualMeter, + stopped: this._stopReason !== null, }, }); @@ -566,9 +775,41 @@ export class BeatDetector { private _onWorkletMessage(event: MessageEvent): void { const message = event.data as Record; + const epoch = (message.epoch as number | undefined) ?? 0; + + if (this._destroyed || epoch !== this._epoch || (this._stopReason !== null && message.type !== 'reset')) { + return; + } switch (message.type) { + case 'reset': + this._clearState(); + break; + case 'state': + this._predictionValidUntil = (message.predictionValidUntil as number) ?? 0; + + if (message.meterEstimate) { + const estimate = message.meterEstimate as MeterEstimate; + this._meterEstimate = Object.freeze({ + ...estimate, + selected: estimate.selected ? Object.freeze(estimate.selected) : null, + alternatives: Object.freeze( + estimate.alternatives.map(candidate => Object.freeze({ ...candidate, model: Object.freeze(candidate.model) })), + ), + }); + this._expiredMeterEstimate = Object.freeze({ + ...this._meterEstimate, + selected: null, + alternatives: acquiringMeter.alternatives, + score: 0, + reason: 'expired', + }); + } else { + this._meterEstimate = acquiringMeter; + this._expiredMeterEstimate = acquiringMeter; + } + this._analysisTime = (message.analysisTime as number) ?? 0; this._analysisLatency = Math.max(0, (this._workletNode?.context.currentTime ?? this._analysisTime) - this._analysisTime); this._tempo = (message.tempo as number) ?? 0; @@ -576,6 +817,7 @@ export class BeatDetector { this._nextBeatTime = (message.nextBeatTime as number) ?? 0; this._nextDownbeatTime = (message.nextDownbeatTime as number) ?? 0; this._confidence = (message.confidence as number) ?? 0; + this._beatConfidence = (message.beatConfidence as number) ?? 0; this._phaseConfidence = (message.phaseConfidence as number) ?? 0; this._gridStability = (message.gridStability as number) ?? 0; this._tempoCandidates = Object.freeze((message.tempoCandidates as TempoCandidate[]) ?? []); @@ -589,11 +831,27 @@ export class BeatDetector { { const lookahead = (message.lookahead as UpcomingBeat[]) ?? []; const revision = (message.lookaheadRevision as number) ?? 0; - this._lookahead = Object.freeze(lookahead); - - if (revision !== this._lookaheadRevision && lookahead.length > 0) { + this._lookahead = Object.freeze(lookahead.map(beat => Object.freeze(beat))); + this._beatOnlyLookahead = Object.freeze(lookahead.map(beat => Object.freeze({ ...beat, isDownbeat: false, meterKnown: false }))); + this._scheduleBase = { + epoch, + revision, + generatedAtAudioTime: (message.predictionGeneratedAtAudioTime as number) ?? this._analysisTime, + evidenceThroughAudioTime: (message.predictionEvidenceThroughAudioTime as number) ?? this._analysisTime, + validUntilAudioTime: this._predictionValidUntil, + horizonAudioTime: lookahead.at(-1)?.audioTime ?? 0, + pulsePeriodSeconds: (message.predictionPulsePeriodSeconds as number) ?? 0, + status: (message.predictionStatus as BeatSchedule['status']) ?? 'provisional', + }; + this._scheduleView = null; + + if (revision !== this._lookaheadRevision) { this._lookaheadRevision = revision; - this.onBeatPredicted.dispatch(lookahead[0]!); + const upcoming = this.lookahead.find(beat => beat.audioTime > getAudioContext().currentTime); + + if (upcoming) { + this.onBeatPredicted.dispatch(upcoming); + } } } @@ -604,15 +862,19 @@ export class BeatDetector { audioTime: message.audioTime as number, tempo: message.tempo as number, confidence: message.confidence as number, + beatConfidence: (message.beatConfidence as number) ?? 0, + onsetEnergy: (message.onsetEnergy as number | null) ?? null, + onsetAudioTime: (message.onsetAudioTime as number | null) ?? null, beatPhase: message.beatPhase as number, energy: message.energy as number, + meterKnown: (message.meterKnown as boolean | undefined) ?? false, isDownbeat: message.isDownbeat as boolean, beatInBar: message.beatInBar as number, status: (message.status as 'provisional' | 'locked' | undefined) ?? 'locked', }; this.onBeat.dispatch(bi); - if (bi.isDownbeat) { + if (bi.isDownbeat && !this._destroyed && epoch === this._epoch) { this.onDownbeat.dispatch(bi); } @@ -636,6 +898,45 @@ export class BeatDetector { } } + private _clearState(): void { + this._meterEstimate = acquiringMeter; + + if (this._options.manualMeter) { + this._meterEstimate = acquiringManualMeter; + } else if (!this._options.enableTimeSignatureDetection) { + this._meterEstimate = acquiringConventionMeter; + } + + this._expiredMeterEstimate = this._meterEstimate; + this._predictionValidUntil = 0; + this._tempo = 0; + this._beatPhase = 0; + this._nextBeatTime = 0; + this._nextDownbeatTime = 0; + this._confidence = 0; + this._beatConfidence = 0; + this._phaseConfidence = 0; + this._gridStability = 0; + this._analysisTime = 0; + this._analysisLatency = 0; + this._tempoCandidates = []; + this._rms = 0; + this._onsetStrength = 0; + this._bandEnergy = { low: 0, mid: 0, high: 0 }; + this._barPosition = 1; + this._barLength = 4; + this._timeSignature = { numerator: 4, denominator: 4 }; + this._lookahead = []; + this._beatOnlyLookahead = []; + this._lookaheadRevision = 0; + this._scheduleBase = null; + this._scheduleView = null; + } + + private _predictionsValid(): boolean { + return !this._destroyed && this._predictionValidUntil > 0 && this._predictionValidUntil > getAudioContext().currentTime; + } + // ----------------------------------------------------------------------- // Private helpers - source tap // ----------------------------------------------------------------------- diff --git a/packages/exojs-audio-fx/src/beatScheduleTypes.ts b/packages/exojs-audio-fx/src/beatScheduleTypes.ts new file mode 100644 index 000000000..10f78f2b8 --- /dev/null +++ b/packages/exojs-audio-fx/src/beatScheduleTypes.ts @@ -0,0 +1,28 @@ +export interface UpcomingBeat { + audioTime: number; + tempo: number; + isDownbeat: boolean; + beatInBar: number; + meterKnown?: boolean; +} + +export interface BeatSchedule { + readonly epoch: number; + readonly revision: number; + /** Time this forecast revision was generated, not when it reached the caller. */ + readonly generatedAtAudioTime: number; + /** End of the audio consumed by this forecast revision. */ + readonly evidenceThroughAudioTime: number; + /** Discard the forecast at this context time without renewed evidence. */ + readonly validUntilAudioTime: number; + /** Last conditional target; this does not extend validity. */ + readonly horizonAudioTime: number; + /** Period used to extrapolate this forecast, not a notated note value. */ + readonly pulsePeriodSeconds: number; + readonly status: 'provisional' | 'locked'; + /** Retained revision entries can include times that have since passed. */ + readonly beats: ReadonlyArray>; + readonly nextBeat: Readonly | null; + /** Null without valid anchored meter or a future boundary within the horizon. */ + readonly nextDownbeat: Readonly | null; +} diff --git a/packages/exojs-audio-fx/src/index.ts b/packages/exojs-audio-fx/src/index.ts index 47ab6d3b4..784d99212 100644 --- a/packages/exojs-audio-fx/src/index.ts +++ b/packages/exojs-audio-fx/src/index.ts @@ -4,17 +4,10 @@ export type { AudioAnalyserOptions, AudioAnalyserSource, SpectrumMappingOptions } from './AudioAnalyser'; export { AudioAnalyser } from './AudioAnalyser'; -export type { - BandEnergy, - BarInfo, - BeatDetectorOptions, - BeatDetectorSource, - BeatInfo, - TempoCandidate, - TimeSignature, - UpcomingBeat, -} from './BeatDetector'; +export type { BeatAnalysisDiscontinuity } from './BeatAnalysisDiscontinuity'; +export type { BandEnergy, BarInfo, BeatDetectorOptions, BeatDetectorSource, BeatInfo, TempoCandidate, TimeSignature } from './BeatDetector'; export { BeatDetector } from './BeatDetector'; +export type { BeatSchedule, UpcomingBeat } from './beatScheduleTypes'; export { AutoWahEffect, type AutoWahEffectOptions } from './effects/AutoWahEffect'; export { BitCrusherEffect, type BitCrusherEffectOptions } from './effects/BitCrusherEffect'; export { ChorusEffect, type ChorusEffectOptions } from './effects/ChorusEffect'; @@ -34,4 +27,5 @@ export { ReverbEffect, type ReverbEffectOptions } from './effects/ReverbEffect'; export { RingModulatorEffect, type RingModulatorEffectOptions } from './effects/RingModulatorEffect'; export { TremoloEffect, type TremoloEffectOptions } from './effects/TremoloEffect'; export { VocoderEffect, type VocoderEffectOptions } from './effects/VocoderEffect'; +export type { ManualMeter, MeterCandidate, MeterEstimate, MeterEstimateReason, MeterModel } from './meterTypes'; export { PcmStreamSource, type PcmStreamSourceOptions, type PcmStreamState } from './PcmStreamSource'; diff --git a/packages/exojs-audio-fx/src/meterTypes.ts b/packages/exojs-audio-fx/src/meterTypes.ts new file mode 100644 index 000000000..7fc18841b --- /dev/null +++ b/packages/exojs-audio-fx/src/meterTypes.ts @@ -0,0 +1,42 @@ +export interface MeterModel { + /** Pulses per cycle of the tracked grid, not necessarily a notated numerator. */ + readonly cyclePulses: number; + readonly pulsePeriodSeconds: number; + /** Zero-based position of the last tracked beat, or null when unanchored. */ + readonly phasePulses: number | null; + /** Not inferred from audio; null unless explicitly supplied. */ + readonly denominator: number | null; +} + +export interface MeterCandidate { + readonly model: MeterModel; + /** Uncalibrated model strength, not a probability. */ + readonly score: number; +} + +export type MeterEstimateReason = + 'acquiring' | 'insufficientEvidence' | 'lowContrast' | 'ambiguousCycle' | 'ambiguousPhase' | 'available' | 'expired'; + +export interface MeterEstimate { + readonly selected: MeterModel | null; + /** At most two distinct cycle models, ordered by score. */ + readonly alternatives: readonly MeterCandidate[]; + /** Uncalibrated strength of the leading model, not a probability. */ + readonly score: number; + readonly reason: MeterEstimateReason; + readonly origin: 'estimated' | 'manual' | 'convention'; + /** Audio-context time through which observations were processed. */ + readonly evidenceThroughAudioTime: number; + /** Audio-context time at which this view loses support without new evidence. */ + readonly validUntilAudioTime: number; +} + +/** Manual counting context; phase anchors at the first tracked beat after reset. */ +export interface ManualMeter { + /** Integer from 1 through 32. */ + readonly cyclePulses: number; + /** Null or omitted leaves phase unknown. */ + readonly phasePulses?: number | null; + /** A power of two from 1 through 128, or null when unspecified. */ + readonly denominator?: number | null; +} diff --git a/packages/exojs-audio-fx/src/worklets/LegacyBarTracker.ts b/packages/exojs-audio-fx/src/worklets/LegacyBarTracker.ts new file mode 100644 index 000000000..2fd0a8cb0 --- /dev/null +++ b/packages/exojs-audio-fx/src/worklets/LegacyBarTracker.ts @@ -0,0 +1,92 @@ +export class LegacyBarTracker { + public readonly posterior4 = new Float32Array([0.25, 0.25, 0.25, 0.25]); + public readonly posterior3 = new Float32Array([1 / 3, 1 / 3, 1 / 3]); + public ts4Confidence = 0.5; + public ts3Confidence = 0.5; + public activeTs: '3/4' | '4/4' = '4/4'; + public sustainCounter = 0; + public barPosition = 1; + public beatsSinceStart = 0; + + public constructor(private readonly enableTsDetection: boolean) {} + + public reset(): void { + this.posterior4.fill(0.25); + this.posterior3.fill(1 / 3); + this.ts4Confidence = 0.5; + this.ts3Confidence = 0.5; + this.activeTs = '4/4'; + this.sustainCounter = 0; + this.barPosition = 1; + this.beatsSinceStart = 0; + } + + public update(likelihood: number): void { + this.beatsSinceStart++; + + const p4 = this.posterior4; + const p3 = this.posterior3; + const max4 = this._advance(p4, likelihood); + const max3 = this._advance(p3, likelihood); + const alpha = 0.1; + this.ts4Confidence = (1 - alpha) * this.ts4Confidence + alpha * max4; + this.ts3Confidence = (1 - alpha) * this.ts3Confidence + alpha * max3; + + const favorThree = this.activeTs === '4/4'; + const favored = favorThree ? this.ts3Confidence > this.ts4Confidence * 1.4 : this.ts4Confidence > this.ts3Confidence * 1.4; + + if (this.enableTsDetection && this.beatsSinceStart > 8 && favored) { + if (++this.sustainCounter >= (favorThree ? 12 : 16)) { + this.activeTs = favorThree ? '3/4' : '4/4'; + this.sustainCounter = 0; + } + } else { + this.sustainCounter = 0; + } + + const barLen = this.activeTs === '3/4' ? 3 : 4; + const posterior = this.activeTs === '3/4' ? p3 : p4; + + if (this.beatsSinceStart >= barLen) { + let maxP = -1; + let maxI = 0; + + for (let i = 0; i < barLen; i++) { + if (posterior[i]! > maxP) { + maxP = posterior[i]!; + maxI = i; + } + } + + this.barPosition = maxI + 1; + } else { + this.barPosition = (this.barPosition % barLen) + 1; + } + } + + private _advance(posterior: Float32Array, likelihood: number): number { + let previous = posterior[posterior.length - 1]!; + let sum = 0; + + for (let i = 0; i < posterior.length; i++) { + const next = posterior[i]!; + posterior[i] = previous * (likelihood + (i === 0 ? 0.3 : 0)); + sum += posterior[i]!; + previous = next; + } + + let max = 0; + + for (let i = 0; i < posterior.length; i++) { + if (sum > 0) { + posterior[i]! /= sum; + } + + if (posterior[i]! > max) { + max = posterior[i]!; + } + } + + return max; + } +} diff --git a/packages/exojs-audio-fx/src/worklets/MeterEvidence.ts b/packages/exojs-audio-fx/src/worklets/MeterEvidence.ts new file mode 100644 index 000000000..58ceb7e0c --- /dev/null +++ b/packages/exojs-audio-fx/src/worklets/MeterEvidence.ts @@ -0,0 +1,354 @@ +import type { MeterEstimateReason } from '../meterTypes'; +import { type MeterModelParameters, meterModelParameters } from './meterModelParameters'; +import type { OnsetEvidenceRing } from './OnsetEvidenceRing'; + +export class MeterEvidence { + public cyclePulses: 0 | 3 | 4 = 0; + public phasePulses: number | null = null; + public score = 0; + public reason: MeterEstimateReason = 'acquiring'; + public readonly candidateScores = new Float64Array(2); + public readonly candidatePhases = new Int8Array(2).fill(-1); + public evidenceThroughSample = 0; + public validUntilSample = 0; + + private readonly _indices = new Float64Array(32); + private readonly _samples = new Float64Array(32); + private readonly _onsetIds = new Float64Array(32); + private readonly _features = new Float64Array(128).fill(NaN); + private readonly _values = new Float64Array(128).fill(NaN); + private readonly _scratch = new Float64Array(4); + private readonly _anchors = new Float64Array(32); + private readonly _anchorWeights = new Float64Array(32); + private readonly _channelScores = new Float64Array(8); + private readonly _featureCounts = new Uint8Array(4); + private readonly _nullErrors = new Float64Array(4); + private readonly _active = new Uint8Array(4); + private readonly _phaseSums = new Float64Array(4); + private readonly _phaseCounts = new Uint8Array(4); + private _count = 0; + private _position = 0; + private _index = -1; + private _lastGrid = -1; + private _period = 0; + private _offset = 0; + private _dirty = false; + + public constructor(private readonly _parameters: MeterModelParameters = meterModelParameters) {} + + public reset(): void { + this.cyclePulses = 0; + this.phasePulses = null; + this.score = 0; + this.reason = 'acquiring'; + this.candidateScores.fill(0); + this.candidatePhases.fill(-1); + this.evidenceThroughSample = 0; + this.validUntilSample = 0; + this._count = 0; + this._position = 0; + this._index = -1; + this._lastGrid = -1; + this._period = 0; + this._offset = 0; + this._dirty = false; + this._features.fill(NaN); + } + + public addBeat(sample: number, period: number, onsetId: number): void { + if (this._lastGrid >= 0 && sample - this._lastGrid < period * 0.5) { + return; + } + + if (this._lastGrid >= 0 && (sample - this._lastGrid > period * 2.5 || Math.abs(period / this._period - 1) > 0.15)) { + this.reset(); + } + + this._index += this._lastGrid < 0 ? 1 : Math.max(1, Math.round((sample - this._lastGrid) / period)); + this._lastGrid = sample; + this._period = period; + this._indices[this._position] = this._index; + this._samples[this._position] = sample; + this._onsetIds[this._position] = onsetId; + this._features.fill(NaN, this._position * 4, this._position * 4 + 4); + this._position = (this._position + 1) % this._indices.length; + this._count = Math.min(this._count + 1, this._indices.length); + this._dirty = true; + } + + public update(sample: number, onsets: OnsetEvidenceRing, window: number): boolean { + const previousCycle = this.cyclePulses; + const previousOffset = this._offset; + const previousPhaseKnown = this.phasePulses !== null; + this.evidenceThroughSample = sample; + + if (this.validUntilSample > 0 && sample >= this.validUntilSample) { + this.cyclePulses = 0; + this.score = 0; + this.validUntilSample = 0; + this.reason = 'expired'; + this.candidateScores.fill(0); + this.candidatePhases.fill(-1); + } + + for (let i = 0; i < this._count; i++) { + if ( + this._onsetIds[i]! > 0 && + !Number.isFinite(this._features[i * 4]!) && + onsets.accentFeatures(this._onsetIds[i]!, sample, window, this._scratch) + ) { + this._features.set(this._scratch, i * 4); + this._dirty = true; + } + } + + if (this._dirty) { + this._fit(sample, window); + this._dirty = false; + } + + this.phasePulses = + this.cyclePulses > 0 && this.candidatePhases[this.cyclePulses - 3]! >= 0 + ? (((this._index - this._offset) % this.cyclePulses) + this.cyclePulses) % this.cyclePulses + : null; + + return previousCycle !== this.cyclePulses || previousOffset !== this._offset || previousPhaseKnown !== (this.phasePulses !== null); + } + + public candidatePhase(cycle: number): number | null { + const offset = this.candidatePhases[cycle - 3]!; + + return offset < 0 ? null : (((this._index - offset) % cycle) + cycle) % cycle; + } + + private _fit(sample: number, window: number): void { + let count = 0; + let latestSupport = 0; + + for (let i = 0; i < this._count; i++) { + if (Number.isFinite(this._features[i * 4]!)) { + count++; + latestSupport = Math.max(latestSupport, this._samples[i]!); + } + } + + this.cyclePulses = 0; + this.score = 0; + this.candidateScores.fill(0); + this.candidatePhases.fill(-1); + // Cover a missing pulse and the complete attack window of its successor. + this.validUntilSample = count > 0 ? latestSupport + 2 * this._period + window : 0; + + if (this.validUntilSample > 0 && this.validUntilSample <= sample) { + this.validUntilSample = 0; + this.reason = 'expired'; + + return; + } + + if (count < 12) { + this.reason = 'insufficientEvidence'; + + return; + } + + let active = 0; + this._active.fill(0); + + for (let feature = 0; feature < 4; feature++) { + let sum = 0; + let n = 0; + + for (let i = 0; i < this._count; i++) { + const value = this._features[i * 4 + feature]!; + + if (Number.isFinite(value)) { + sum += value; + n++; + } + } + + if (n < 12 || sum <= 0) { + continue; + } + + const mean = sum / n; + sum = 0; + let squares = 0; + + for (let i = 0; i < this._count; i++) { + const raw = this._features[i * 4 + feature]!; + const value = Number.isFinite(raw) ? Math.log1p(raw / mean) : NaN; + this._values[i * 4 + feature] = value; + + if (Number.isFinite(value)) { + sum += value; + squares += value * value; + } + } + + const error = Math.max(0, squares - (sum * sum) / n); + + if (error < (0.01 * sum * sum) / n) { + continue; + } + + const center = sum / n; + const scale = Math.sqrt(error / n); + + for (let i = 0; i < this._count; i++) { + const index = i * 4 + feature; + this._values[index] = (this._values[index]! - center) / scale; + } + + this._active[feature] = 1; + this._featureCounts[feature] = n; + // A repeated two-pulse pattern is an alias, not evidence of a four-pulse cycle. + const parityError = this._error(2, feature); + // Unseen parity positions cannot support a two-position null; use the constant null. + this._nullErrors[feature] = Number.isFinite(parityError) ? parityError : n; + + if (this._nullErrors[feature]! < 0.01 * n) { + this._active[feature] = 0; + } else { + active++; + } + } + + if (active === 0) { + this.reason = 'lowContrast'; + + return; + } + + let bestCycle: 0 | 3 | 4 = 0; + let bestScore = 0; + let secondScore = 0; + + for (let cycle = 3; cycle <= 4; cycle++) { + let likelihood = 0; + let prior = 0; + + for (let feature = 0; feature < 4; feature++) { + if (this._active[feature]) { + const n = this._featureCounts[feature]!; + const error = this._error(cycle, feature); + const strength = Number.isFinite(error) ? 1 - error / this._nullErrors[feature]! - ((cycle - 2) * Math.log(n)) / n : -Infinity; + this._channelScores[(cycle - 3) * 4 + feature] = strength; + likelihood += this._parameters.channelPriors[feature]! * Math.exp((strength * count) / 2); + prior += this._parameters.channelPriors[feature]!; + } + } + + // Marginalize channels so an uninformative band cannot veto a consistent attack. + // Both cycle models use identical channel priors and the same two-pulse null. + const score = Math.max(0, (Math.log(likelihood / prior) * 2) / count); + this.candidateScores[cycle - 3] = score; + this.candidatePhases[cycle - 3] = score > 0 ? this._anchor(cycle) : -1; + + if (score > bestScore) { + secondScore = bestScore; + bestScore = score; + bestCycle = cycle as 3 | 4; + } else { + secondScore = Math.max(secondScore, score); + } + } + + this.score = bestScore; + this.reason = bestScore < this._parameters.minimumScore ? 'lowContrast' : 'ambiguousCycle'; + + if (bestScore >= this._parameters.minimumScore && bestScore - secondScore >= this._parameters.cycleMargin) { + this.cyclePulses = bestCycle; + this._offset = this.candidatePhases[bestCycle - 3]!; + this.reason = this._offset >= 0 ? 'available' : 'ambiguousPhase'; + } + } + + private _error(cycle: number, feature: number): number { + this._phaseSums.fill(0); + this._phaseCounts.fill(0); + let squares = 0; + + for (let i = 0; i < this._count; i++) { + const value = this._values[i * 4 + feature]!; + + if (Number.isFinite(value)) { + const phase = this._indices[i]! % cycle; + this._phaseSums[phase] = this._phaseSums[phase]! + value; + this._phaseCounts[phase] = this._phaseCounts[phase]! + 1; + squares += value * value; + } + } + + for (let phase = 0; phase < cycle; phase++) { + const n = this._phaseCounts[phase]!; + + if (n < 3) { + return Infinity; + } + + squares -= this._phaseSums[phase]! ** 2 / n; + } + + return Math.max(0, squares); + } + + private _anchor(cycle: number): number { + this._anchors.fill(0); + this._anchorWeights.fill(0); + let maximum = -Infinity; + + for (let feature = 0; feature < 4; feature++) { + if (this._active[feature]) { + maximum = Math.max(maximum, this._channelScores[(cycle - 3) * 4 + feature]!); + } + } + + for (let feature = 0; feature < 4; feature++) { + if (this._active[feature]) { + const weight = + this._parameters.channelPriors[feature]! * + Math.exp(((this._channelScores[(cycle - 3) * 4 + feature]! - maximum) * this._count) / 2); + + for (let i = 0; i < this._count; i++) { + const value = this._values[i * 4 + feature]!; + + if (Number.isFinite(value)) { + this._anchors[i] = this._anchors[i]! + weight * value; + this._anchorWeights[i] = this._anchorWeights[i]! + weight; + } + } + } + } + + this._phaseSums.fill(0); + this._phaseCounts.fill(0); + + for (let i = 0; i < this._count; i++) { + if (this._anchorWeights[i]! > 0) { + const phase = this._indices[i]! % cycle; + this._phaseSums[phase] = this._phaseSums[phase]! + this._anchors[i]! / this._anchorWeights[i]!; + this._phaseCounts[phase] = this._phaseCounts[phase]! + 1; + } + } + + let best = -Infinity; + let second = -Infinity; + let offset = -1; + + for (let phase = 0; phase < cycle; phase++) { + const mean = this._phaseSums[phase]! / this._phaseCounts[phase]!; + + if (mean > best) { + second = best; + best = mean; + offset = phase; + } else { + second = Math.max(second, mean); + } + } + + return best - second >= this._parameters.phaseMargin ? offset : -1; + } +} diff --git a/packages/exojs-audio-fx/src/worklets/OnsetEvidenceRing.ts b/packages/exojs-audio-fx/src/worklets/OnsetEvidenceRing.ts new file mode 100644 index 000000000..6b96f0177 --- /dev/null +++ b/packages/exojs-audio-fx/src/worklets/OnsetEvidenceRing.ts @@ -0,0 +1,116 @@ +/** Keeps raw and normalized onset observations together without per-query allocation. */ +export class OnsetEvidenceRing { + /** Borrowed view, overwritten by the next query or reset. */ + public readonly selected = { id: 0, sample: 0, rawFlux: 0, strength: 0 }; + public count = 0; + private readonly _samples = new Float64Array(16); + private readonly _flux = new Float64Array(16); + private readonly _strengths = new Float32Array(16); + private readonly _ids = new Float64Array(16); + private readonly _powers = new Float64Array(16); + private readonly _bands = new Float64Array(48).fill(NaN); + private _position = 0; + private _id = 0; + + public push(sample: number, rawFlux: number, strength: number): void { + this._samples[this._position] = sample; + this._flux[this._position] = rawFlux; + this._strengths[this._position] = strength; + this._ids[this._position] = ++this._id; + this._powers[this._position] = 0; + this._bands.fill(NaN, this._position * 3, this._position * 3 + 3); + this._position = (this._position + 1) % this._samples.length; + this.count = Math.min(this.count + 1, this._samples.length); + } + + public observePower(sample: number, power: number, window: number, bands?: Float64Array): void { + for (let i = 0; i < this.count; i++) { + if (sample >= this._samples[i]! && sample - this._samples[i]! <= window) { + this._powers[i] = Math.max(this._powers[i]!, power); + + if (bands) { + for (let band = 0; band < 3; band++) { + const index = i * 3 + band; + this._bands[index] = Math.max(Number.isFinite(this._bands[index]!) ? this._bands[index]! : 0, bands[band]!); + } + } + } + } + } + + public accent(id: number, sample: number, window: number): number | null { + for (let i = 0; i < this.count; i++) { + if (this._ids[i] === id && sample - this._samples[i]! >= window) { + return this._powers[i]!; + } + } + + return null; + } + + public accentFeatures(id: number, sample: number, window: number, into: Float64Array): boolean { + for (let i = 0; i < this.count; i++) { + if (this._ids[i] === id && sample - this._samples[i]! >= window) { + into[0] = this._powers[i]!; + + for (let band = 0; band < 3; band++) { + into[band + 1] = this._bands[i * 3 + band]!; + } + + return true; + } + } + + return false; + } + + public nearest(target: number, window: number): number { + let index = -1; + let distance = window + 1; + + for (let i = 0; i < this.count; i++) { + const next = Math.abs(this._samples[i]! - target); + + if (next <= window && next < distance) { + distance = next; + index = i; + } + } + + return this._select(index); + } + + public bootstrap(now: number, maxAge: number): number { + let index = -1; + let strength = -1; + let sample = -1; + + for (let i = 0; i < this.count; i++) { + const nextSample = this._samples[i]!; + const nextStrength = this._strengths[i]!; + + if (now - nextSample <= maxAge && (nextStrength > strength || (nextStrength === strength && nextSample > sample))) { + sample = nextSample; + strength = nextStrength; + index = i; + } + } + + return this._select(index); + } + + public reset(): void { + this.count = 0; + this._position = 0; + this._select(-1); + } + + private _select(index: number): number { + this.selected.id = index >= 0 ? this._ids[index]! : 0; + this.selected.sample = index >= 0 ? this._samples[index]! : 0; + this.selected.rawFlux = index >= 0 ? this._flux[index]! : 0; + this.selected.strength = index >= 0 ? this._strengths[index]! : 0; + + return index >= 0 ? this.selected.sample : -1; + } +} diff --git a/packages/exojs-audio-fx/src/worklets/TempoCandidateScorer.ts b/packages/exojs-audio-fx/src/worklets/TempoCandidateScorer.ts new file mode 100644 index 000000000..00dc434e1 --- /dev/null +++ b/packages/exojs-audio-fx/src/worklets/TempoCandidateScorer.ts @@ -0,0 +1,95 @@ +import { + acfAtLag, + candidateEdgeTolerance, + combPenaltyDouble, + combPenaltyTriple, + combWeightFundamental, + combWeightHalf, + combWeightThird, + defaultPriorMu, + defaultPriorSigma, + parabolicPeakOffset, + type TempoCandidateResult, + tempoPrior, +} from '../dsp/tempogram'; + +export class TempoCandidateScorer { + private readonly _rawScores = new Float64Array(3); + + public constructor( + private readonly _sampleRate: number, + private readonly _hopSize: number, + private readonly _minBpm: number, + private readonly _maxBpm: number, + ) {} + + public scoreInto(acf: Float32Array, minLag: number, into: TempoCandidateResult[]): number { + const numLags = acf.length; + const lastIdx = numLags - 1; + const loBpm = this._minBpm * (1 - candidateEdgeTolerance); + const hiBpm = this._maxBpm * (1 + candidateEdgeTolerance); + let count = 0; + + if (numLags <= 1) { + return count; + } + + for (let i = 0; i < numLags; i++) { + const raw = acf[i]!; + + if (raw <= 0 || (i > 0 && !(raw > acf[i - 1]!)) || (i < lastIdx && !(raw > acf[i + 1]!))) { + continue; + } + + const lag = minLag + i + (i > 0 && i < lastIdx ? parabolicPeakOffset(acf[i - 1]!, raw, acf[i + 1]!) : 0); + const bpm = (60 * this._sampleRate) / (lag * this._hopSize); + + if (bpm < loBpm || bpm > hiBpm) { + continue; + } + + const aF = acfAtLag(acf, minLag, lag); + const aHalf = acfAtLag(acf, minLag, lag * 2); + const aThird = acfAtLag(acf, minLag, lag * 3); + const aDouble = acfAtLag(acf, minLag, lag / 2); + const aTriple = acfAtLag(acf, minLag, lag / 3); + const support = combWeightFundamental * aF + combWeightHalf * aHalf + combWeightThird * aThird; + // Energy above the BPM ceiling is a subdivision, not a competing beat. + const penDbl = bpm * 2 <= hiBpm ? combPenaltyDouble * aDouble : 0; + const penTrip = bpm * 3 <= hiBpm ? combPenaltyTriple * aTriple : 0; + let comb = support - penDbl - penTrip; + + if (comb < 0) { + comb = 0; + } + + const score = comb * tempoPrior(bpm, defaultPriorMu, defaultPriorSigma); + let position = 0; + + // Final-score ties retain the raw-peak ranking of the canonical stable sorts. + while ( + position < count && + (into[position]!.score > score || (into[position]!.score === score && this._rawScores[position]! >= raw)) + ) { + position++; + } + + if (position < into.length) { + for (let j = Math.min(count, into.length - 1); j > position; j--) { + into[j]!.bpm = into[j - 1]!.bpm; + into[j]!.score = into[j - 1]!.score; + into[j]!.lag = into[j - 1]!.lag; + this._rawScores[j] = this._rawScores[j - 1]!; + } + + into[position]!.bpm = bpm; + into[position]!.score = score; + into[position]!.lag = lag; + this._rawScores[position] = raw; + count = Math.min(count + 1, into.length); + } + } + + return count; + } +} diff --git a/packages/exojs-audio-fx/src/worklets/beat-detector.worklet.ts b/packages/exojs-audio-fx/src/worklets/beat-detector.worklet.ts index 37cf56ba0..b71aedf00 100644 --- a/packages/exojs-audio-fx/src/worklets/beat-detector.worklet.ts +++ b/packages/exojs-audio-fx/src/worklets/beat-detector.worklet.ts @@ -14,36 +14,28 @@ // The DSP this file drives - FFT, mel filterbank, tempogram - lives in `../dsp/` // and is imported, not copied. What stays here is the real-time orchestration // around it: preallocated scratch buffers, ring-buffer bookkeeping, and the PLL -// beat tracker. The audio thread cannot allocate, so this file calls the -// out-parameter primitives (`computeAcfInto`) rather than the allocating -// convenience wrappers built on them. +// beat tracker. Candidate scoring reuses buffers through the out-parameter +// primitives (`computeAcfInto`); message publication has separate snapshot +// lifetimes. import { fft } from '../dsp/fft'; import { buildMelFilterbank, computeMelBands, type MelBand } from '../dsp/mel'; -import { - acfAtLag, - acfExtendedMinLag, - candidateEdgeTolerance, - combPenaltyDouble, - combPenaltyTriple, - combWeightFundamental, - combWeightHalf, - combWeightThird, - computeAcfInto, - defaultPriorMu, - defaultPriorSigma, - isOctaveRelated, - parabolicPeakOffset, - type TempoCandidateResult, - tempoPrior, -} from '../dsp/tempogram'; +import { acfExtendedMinLag, computeAcfInto, isOctaveRelated, type TempoCandidateResult } from '../dsp/tempogram'; +import type { ManualMeter, MeterEstimate } from '../meterTypes'; +import { captureBeatEvidence, createBeatEvidence, resetBeatEvidence } from './beatEvidence'; import { DisposableProcessor } from './disposable-processor'; +import { LegacyBarTracker } from './LegacyBarTracker'; +import { createMeterEstimate } from './meterEstimateView'; +import { MeterEvidence } from './MeterEvidence'; +import { OnsetEvidenceRing } from './OnsetEvidenceRing'; +import { TempoCandidateScorer } from './TempoCandidateScorer'; interface UpcomingBeatInline { audioTime: number; tempo: number; isDownbeat: boolean; beatInBar: number; + meterKnown: boolean; } interface BandEnergyInline { @@ -53,6 +45,7 @@ interface BandEnergyInline { } interface BeatDetectorProcessorOptions { + epoch?: number; fftSize?: number; hopSize?: number; minBpm?: number; @@ -63,6 +56,8 @@ interface BeatDetectorProcessorOptions { fastTempoWindowSec?: number; stableTempoWindowSec?: number; enableTimeSignatureDetection?: boolean; + manualMeter?: ManualMeter | null; + stopped?: boolean; acfIntervalHops?: number; } @@ -98,7 +93,6 @@ const ONSET_PEAK_DECAY = 0.999; // per-hop decay of the running flux peak (noise const ONSET_MIN_REFRACTORY_SEC = 0.1; // minimum spacing between detected onsets (~100 ms) const ONSET_REFRACTORY_IBI_FRAC = 0.4; // once locked, refractory = max(min, frac × IBI) const ONSET_BEAT_COAST_IBI = 2; // suppress beat emission after this many IBIs without an onset -const ONSET_RING_SIZE = 16; // recent onsets retained for the PLL's nearestOnset() // ---- PLL beat-phase tracker ---- // A bounded phase-locked loop replaces the old constant-IBI predictor + buggy snap. Each @@ -162,6 +156,8 @@ class BeatDetectorProcessor extends DisposableProcessor { private readonly _fastTempoWindowSec: number; private readonly _stableTempoWindowSec: number; private readonly _enableTsDetection: boolean; + private _manualMeter: ManualMeter | null; + private _stopped: boolean; private readonly _real: Float32Array; private readonly _imag: Float32Array; @@ -173,6 +169,7 @@ class BeatDetectorProcessor extends DisposableProcessor { private readonly _melBandFilters: readonly MelBand[]; private readonly _melOut: Float32Array; + private readonly _meterBands = new Float64Array(3); private readonly _fluxWindow: Float32Array; private readonly _linFlux: Float32Array; private _fluxWritePos: number; @@ -192,7 +189,10 @@ class BeatDetectorProcessor extends DisposableProcessor { private _bestBpm: number; private _bestScore: number; - private _candidates: TempoCandidateResult[]; + private readonly _candidates: TempoCandidateResult[]; + private _candidateCount: number; + private readonly _fastCandidates: TempoCandidateResult[]; + private readonly _candidateScorer: TempoCandidateScorer; private _firstLockSample: number; private _driftBpm: number; @@ -204,17 +204,14 @@ class BeatDetectorProcessor extends DisposableProcessor { private readonly _ibiHistory: Float32Array; private _ibiIdx: number; - private readonly _posterior4: Float32Array; - private readonly _posterior3: Float32Array; - private _ts4Confidence: number; - private _ts3Confidence: number; - private _activeTs: '4/4' | '3/4'; - private _sustainCounter: number; - private _barPosition: number; - private _barNumber: number; - private _beatsSinceStart: number; + private readonly _legacyBars: LegacyBarTracker; + private readonly _meter: MeterEvidence; + private _barPosition = 1; + private _conventionPhase = 0; + private _barNumber = 0; private _confidence: number; + private _beatConfidence = 0; private _phaseConfidence: number; /** @@ -224,12 +221,15 @@ class BeatDetectorProcessor extends DisposableProcessor { * clock by itself. */ private _frameOffset: number; + private _epoch: number; + private _expectedFrame = -1; private readonly _stateInterval: number; private _hopsSinceState: number; private readonly _minEmitHops: number; private _authLocked: boolean; + private _fullWindowAttempted = false; private _locked: boolean; private _beatsSinceAuthLock: number; @@ -237,6 +237,10 @@ class BeatDetectorProcessor extends DisposableProcessor { // State messages repeat a schedule until the next beat replaces it. The // revision, not a comparison of its times, tells a new schedule from a repeat. private _lookaheadRevision: number; + private _predictionValidUntilSample = 0; + private _predictionGeneratedAtSample = 0; + private _predictionPeriodSamples = 0; + private _predictionStatus: 'provisional' | 'locked' = 'provisional'; private _rms: number; private _onsetStrength: number; @@ -250,10 +254,9 @@ class BeatDetectorProcessor extends DisposableProcessor { private _onsetPrev1: number; private _onsetHopCount: number; private _lastOnsetSample: number; - private readonly _onsetRingSamples: Float32Array; - private readonly _onsetRingStrengths: Float32Array; - private _onsetRingPos: number; - private _onsetRingCount: number; + private readonly _onsets = new OnsetEvidenceRing(); + private readonly _matchedOnset = this._onsets.selected; + private readonly _beatEvidence = createBeatEvidence(); private readonly _lowBandEnd: number; private readonly _midBandEnd: number; @@ -261,6 +264,7 @@ class BeatDetectorProcessor extends DisposableProcessor { public constructor(options?: unknown) { super(); const opts = (options as { processorOptions?: BeatDetectorProcessorOptions } | undefined)?.processorOptions ?? {}; + this._epoch = opts.epoch ?? 0; this._sampleRate = sampleRate; this._fftSize = opts.fftSize || 2048; this._hopSize = opts.hopSize || 512; @@ -278,6 +282,9 @@ class BeatDetectorProcessor extends DisposableProcessor { this._fastTempoWindowSec = opts.fastTempoWindowSec || 2.5; this._stableTempoWindowSec = opts.stableTempoWindowSec || 8; this._enableTsDetection = opts.enableTimeSignatureDetection !== false; + this._manualMeter = opts.manualMeter ?? null; + this._stopped = opts.stopped ?? false; + this._conventionPhase = this._manualMeter?.phasePulses ?? 0; const numBins = this._fftSize >> 1; this._real = new Float32Array(this._fftSize); @@ -325,7 +332,10 @@ class BeatDetectorProcessor extends DisposableProcessor { // Tempo state this._bestBpm = 0; this._bestScore = 0; - this._candidates = []; + this._candidates = Array.from({ length: 3 }, () => ({ bpm: 0, score: 0, lag: 0 })); + this._candidateCount = 0; + this._fastCandidates = Array.from({ length: 3 }, () => ({ bpm: 0, score: 0, lag: 0 })); + this._candidateScorer = new TempoCandidateScorer(this._sampleRate, this._hopSize, this._minBpm, this._maxBpm); this._firstLockSample = -1; // sample index of first lock (for hysteresis grace window) // Drift-tracking state. The fast window must point at a different, non-octave tempo @@ -340,16 +350,8 @@ class BeatDetectorProcessor extends DisposableProcessor { this._ibiHistory = new Float32Array(4); // last 4 inter-beat intervals this._ibiIdx = 0; - // Bar position state - parallel posteriors for 4/4 and 3/4 - this._posterior4 = new Float32Array([0.25, 0.25, 0.25, 0.25]); - this._posterior3 = new Float32Array([1 / 3, 1 / 3, 1 / 3]); - this._ts4Confidence = 0.5; - this._ts3Confidence = 0.5; - this._activeTs = '4/4'; - this._sustainCounter = 0; - this._barPosition = 1; // 1-indexed - this._barNumber = 0; - this._beatsSinceStart = 0; + this._legacyBars = new LegacyBarTracker(this._enableTsDetection); + this._meter = new MeterEvidence(); // Confidence this._confidence = 0; @@ -357,7 +359,7 @@ class BeatDetectorProcessor extends DisposableProcessor { this._frameOffset = 0; // State snapshot cadence (~20 Hz) - const STATE_INTERVAL_HOPS = Math.round(this._sampleRate / this._hopSize / 20); + const STATE_INTERVAL_HOPS = Math.ceil(this._sampleRate / this._hopSize / 20); this._stateInterval = Math.max(1, STATE_INTERVAL_HOPS); this._hopsSinceState = 0; @@ -365,7 +367,7 @@ class BeatDetectorProcessor extends DisposableProcessor { // the ACF may run (derived from minSettlingMs); _locked latches once the full-window // evidence + beat persistence + confidence floor are met (see LOCK_PROMOTE_* and // _emitBeat). _onGridStreak counts consecutive onset-matched beats toward promotion. - this._minEmitHops = Math.max(1, Math.round((this._minSettlingMs * 0.001 * this._sampleRate) / this._hopSize)); + this._minEmitHops = Math.max(1, Math.ceil((this._minSettlingMs * 0.001 * this._sampleRate) / this._hopSize)); // _authLocked flips true the first time the flux ring holds a full slowest-tempo period // (fluxCount ≥ maxLag+1) - the AUTHORITATIVE lock, identical to the original first lock. // Before it, any ACF tempo is an early estimate that drives provisional beats only and is @@ -396,16 +398,30 @@ class BeatDetectorProcessor extends DisposableProcessor { // _lastOnsetSample = most recent confirmed onset (sub-hop precise); the ring holds // the last ONSET_RING_SIZE onsets so the PLL can pick nearestOnset(). this._lastOnsetSample = -1; - this._onsetRingSamples = new Float32Array(ONSET_RING_SIZE); - this._onsetRingStrengths = new Float32Array(ONSET_RING_SIZE); - this._onsetRingPos = 0; - this._onsetRingCount = 0; // Band energy (for state messages) const LOW_BANDS = Math.round(this._melBands * 0.25); const MID_BANDS = Math.round(this._melBands * 0.6); this._lowBandEnd = LOW_BANDS; this._midBandEnd = MID_BANDS; + + const disposeMessage = this.port.onmessage; + + this.port.onmessage = event => { + const message = event.data as { type?: string; epoch?: number; manualMeter?: ManualMeter | null; stopped?: boolean }; + + if (message.type === 'reset' && typeof message.epoch === 'number' && message.epoch > this._epoch) { + this._stopped = message.stopped ?? false; + + if ('manualMeter' in message) { + this._manualMeter = message.manualMeter ?? null; + } + + this._resetAnalysis(message.epoch); + } else if (message.type === 'destroy') { + disposeMessage?.call(this.port, event); + } + }; } public override process(inputs: Float32Array[][], _outputs: Float32Array[][], _parameters: Record): boolean { @@ -413,17 +429,30 @@ class BeatDetectorProcessor extends DisposableProcessor { return false; } + if (this._stopped) { + return true; + } + const input = inputs[0]; - if (!input || input.length === 0) { + if (!input || !input[0]?.length) { + if (this._expectedFrame >= 0) { + this._resetAnalysis(this._epoch); + } + return true; } - const left = input[0] ?? new Float32Array(0); + if (this._expectedFrame >= 0 && currentFrame !== this._expectedFrame) { + this._resetAnalysis(this._epoch); + } + + const left = input[0]; const right = input[1] ?? left; const blockLen = left.length; this._frameOffset = currentFrame - this._sampleCount; + this._expectedFrame = currentFrame + blockLen; for (let s = 0; s < blockLen; s++) { // Mono downmix @@ -448,6 +477,70 @@ class BeatDetectorProcessor extends DisposableProcessor { return true; } + private _resetAnalysis(epoch: number): void { + this._epoch = epoch; + this._expectedFrame = -1; + this._frameOffset = 0; + this._ringBuffer.fill(0); + this._real.fill(0); + this._imag.fill(0); + this._mag.fill(0); + this._ringWritePos = 0; + this._sampleCount = 0; + this._hopAccum = 0; + this._melOut.fill(0); + + for (const frame of this._prevMelFrames) { + frame.fill(0); + } + + this._prevMelFrameIdx = 0; + this._fluxWindow.fill(0); + this._fluxWritePos = 0; + this._fluxCount = 0; + this._hopsSinceACF = 0; + this._fullWindowAttempted = false; + this._bestBpm = 0; + this._bestScore = 0; + this._candidateCount = 0; + this._firstLockSample = -1; + this._driftBpm = 0; + this._driftHops = 0; + this._lastBeatSample = -1; + this._lastBeatContextTime = 0; + this._ibiSamples = 0; + this._ibiHistory.fill(0); + this._ibiIdx = 0; + this._legacyBars.reset(); + this._meter.reset(); + this._barPosition = 1; + this._conventionPhase = this._manualMeter?.phasePulses ?? 0; + this._confidence = 0; + this._beatConfidence = 0; + this._phaseConfidence = 0; + this._hopsSinceState = 0; + this._authLocked = false; + this._locked = false; + this._beatsSinceAuthLock = 0; + this._lookahead = []; + this._predictionValidUntilSample = 0; + this._predictionGeneratedAtSample = 0; + this._predictionPeriodSamples = 0; + this._predictionStatus = 'provisional'; + this._rms = 0; + this._onsetStrength = 0; + this._onsetWin.fill(0); + this._onsetWinPos = 0; + this._onsetWinCount = 0; + this._fluxPeak = 0; + this._onsetPrev1 = 0; + this._onsetHopCount = 0; + this._lastOnsetSample = -1; + this._onsets.reset(); + resetBeatEvidence(this._beatEvidence, epoch); + this.port.postMessage({ type: 'reset', epoch }); + } + /** Audio-context seconds of an analysed sample position from the current render quantum. */ private _contextTimeAt(sample: number): number { return (sample + this._frameOffset) / this._sampleRate; @@ -489,6 +582,10 @@ class BeatDetectorProcessor extends DisposableProcessor { const K = this._prevMelFrames.length; let flux = 0; + if (this._automaticMeter) { + this._meterBands.fill(0); + } + for (let b = 0; b < this._melBands; b++) { let localMax = -Infinity; @@ -504,6 +601,11 @@ class BeatDetectorProcessor extends DisposableProcessor { if (diff > 0) { flux += diff; + + if (this._automaticMeter) { + const band = b < this._lowBandEnd ? 0 : Number(b >= this._midBandEnd) + 1; + this._meterBands[band] = this._meterBands[band]! + diff; + } } } @@ -528,20 +630,34 @@ class BeatDetectorProcessor extends DisposableProcessor { // of the ACF/tempo path, which still consumes the raw _fluxWindow above). this._detectOnset(flux); + if (this._automaticMeter) { + this._onsets.observePower(this._sampleCount, this._rms * this._rms, this._fftSize, this._meterBands); + } + + if (this._automaticMeter && this._meter.update(this._sampleCount, this._onsets, this._fftSize) && this._lastBeatSample >= 0) { + this._barPosition = this._meter.phasePulses !== null ? this._meter.phasePulses + 1 : this._legacyBars.barPosition; + this._updateLookahead(this._lastBeatContextTime); + } + // Tempogram: compute ACF periodically. The warm-up gate is lowered from a full slowest- // tempo period (_maxLag+1, ~1.2 s at minBpm) to _minEmitHops (~0.4 s) so a fast/core tempo // can lock - and a provisional beat can fire - far earlier. The ACF over a short span only // resolves tempos that already have ≳1 period of evidence in it, so slow tempos still wait // (physically) for their period; promotion to 'locked' still requires the full window. this._hopsSinceACF++; - const dueForAcf = this._hopsSinceACF >= this._acfInterval && this._fluxCount >= this._minEmitHops; + const dueForAcf = this._hopsSinceACF >= this._acfInterval && this._sampleCount >= this._minEmitHops * this._hopSize; // Force the AUTHORITATIVE compute the instant the full window first fills (independent of // the earlier provisional ACF cadence), so the authoritative lock + phase re-bootstrap land // on exactly the evidence the original detector used for its first lock. That makes every // LOCKED beat identical to the original output (same tempo, same grid phase). - const firstFullWindow = !this._authLocked && this._fluxCount >= this._maxLag + 1; + const firstFullWindow = + !this._fullWindowAttempted && this._fluxCount >= this._maxLag + 1 && this._sampleCount >= this._minEmitHops * this._hopSize; if (dueForAcf || firstFullWindow) { + if (firstFullWindow) { + this._fullWindowAttempted = true; + } + this._hopsSinceACF = 0; this._computeACFAndCandidates(); } @@ -566,19 +682,15 @@ class BeatDetectorProcessor extends DisposableProcessor { // Score tempo candidates over the most-recent spanHops of the flux ring (capped at the // available count). Mirrors src/dsp/tempogram.computeTempoCandidates over that span: ACF → // positive peaks (parabola-refined interior, raw endpoints) → comb − super-harmonic penalty - // × tempo prior → top-3 sorted by score. The STABLE span (whole ring) is the parity-checked - // candidate set. Shares the _linFlux / _acf scratch buffers, so the caller must consume the - // returned array before the next call. - private _scoreSpan(spanHops: number): TempoCandidateResult[] { + // × tempo prior → top-3 sorted by score. Stable and fast results need separate storage: + // both must remain available after the shared ACF scratch is overwritten. + private _scoreSpan(spanHops: number, into: TempoCandidateResult[]): number { const buf = this._fluxWindow; const wp = this._fluxWritePos; const len = buf.length; const n = spanHops < this._fluxCount ? spanHops : this._fluxCount; const minLag = this._acfMinLag; // extended ACF base (see constructor) const maxLag = this._maxLag; - const numLags = maxLag - minLag + 1; - const loBpm = this._minBpm * (1 - candidateEdgeTolerance); - const hiBpm = this._maxBpm * (1 + candidateEdgeTolerance); // Linearise the most-recent n hops of the flux ring (oldest first) into a scratch array // so the ACF operates on the same layout src/dsp/tempogram.computeAcf expects. @@ -592,80 +704,17 @@ class BeatDetectorProcessor extends DisposableProcessor { const acf = this._acf; computeAcfInto(lin, n, minLag, maxLag, acf); - // Raw positive peaks (interior + endpoints) - mirrors findTempoPeaks. - const peaks: TempoCandidateResult[] = []; - const lastIdx = numLags - 1; - - if (numLags > 1 && acf[0]! > acf[1]! && acf[0]! > 0) { - peaks.push({ bpm: (60 * this._sampleRate) / (minLag * this._hopSize), score: acf[0]!, lag: minLag }); - } - - for (let i = 1; i < lastIdx; i++) { - if (acf[i]! > acf[i - 1]! && acf[i]! > acf[i + 1]! && acf[i]! > 0) { - const lagI = minLag + i + parabolicPeakOffset(acf[i - 1]!, acf[i]!, acf[i + 1]!); - peaks.push({ bpm: (60 * this._sampleRate) / (lagI * this._hopSize), score: acf[i]!, lag: lagI }); - } - } - - if (numLags > 1 && acf[lastIdx]! > acf[lastIdx - 1]! && acf[lastIdx]! > 0) { - const lagL = minLag + lastIdx; - peaks.push({ bpm: (60 * this._sampleRate) / (lagL * this._hopSize), score: acf[lastIdx]!, lag: lagL }); - } - - peaks.sort((a, b) => b.score - a.score); - - // Filter to BPM range (with edge tolerance) - mirrors computeTempoCandidates filter. - const inRange: TempoCandidateResult[] = []; - - for (let pf = 0; pf < peaks.length; pf++) { - if (peaks[pf]!.bpm >= loBpm && peaks[pf]!.bpm <= hiBpm) { - inRange.push(peaks[pf]!); - } - } - - // Comb (own sub-multiples) − super-harmonic penalty, × tempo prior. - // Mirrors scoreTempoHypotheses. The penalty is what defeats the octave-down bias. - // The penalty is SUBDIVISION-AWARE: a super-harmonic kf only demotes f when kf is - // itself a plausible beat (kf <= maxBpm). Energy above maxBpm is a subdivision (hats - // on 8ths over a 180 kick), not a competing fundamental, so it must not demote f. - const superHi = hiBpm; - const scored: TempoCandidateResult[] = []; - - for (let q = 0; q < inRange.length; q++) { - const lag = inRange[q]!.lag; - const aF = acfAtLag(acf, minLag, lag); - const aHalf = acfAtLag(acf, minLag, lag * 2); - const aThird = acfAtLag(acf, minLag, lag * 3); - const aDouble = acfAtLag(acf, minLag, lag / 2); - const aTriple = acfAtLag(acf, minLag, lag / 3); - const support = combWeightFundamental * aF + combWeightHalf * aHalf + combWeightThird * aThird; - const penDbl = inRange[q]!.bpm * 2 <= superHi ? combPenaltyDouble * aDouble : 0; - const penTrip = inRange[q]!.bpm * 3 <= superHi ? combPenaltyTriple * aTriple : 0; - let comb = support - penDbl - penTrip; - - if (comb < 0) { - comb = 0; - } - - const w = tempoPrior(inRange[q]!.bpm, defaultPriorMu, defaultPriorSigma); - scored.push({ bpm: inRange[q]!.bpm, score: comb * w, lag }); - } - - scored.sort((a, b) => b.score - a.score); - - return scored.slice(0, 3); + return this._candidateScorer.scoreInto(acf, minLag, into); } private _computeACFAndCandidates(): void { // Dual ACF windows over the SAME flux ring. The long STABLE span holds the grid (and // is the parity-checked candidate set the state message reports); the short FAST span // detects a genuine tempo change quickly. Both feed the octave-aware hysteresis below. - const stable = this._scoreSpan(this._stableWindowHops); - const fast = this._scoreSpan(this._fastWindowHops); + this._candidateCount = this._scoreSpan(this._stableWindowHops, this._candidates); + const fastCount = this._scoreSpan(this._fastWindowHops, this._fastCandidates); - this._candidates = stable; - - if (this._candidates.length === 0) { + if (this._candidateCount === 0) { return; } @@ -709,7 +758,7 @@ class BeatDetectorProcessor extends DisposableProcessor { // Fresh score of the currently-tracked tempo from this frame. let currentScore = 0; - for (let c = 0; c < this._candidates.length; c++) { + for (let c = 0; c < this._candidateCount; c++) { if (Math.abs(this._candidates[c]!.bpm / this._bestBpm - 1) < 0.03 && this._candidates[c]!.score > currentScore) { currentScore = this._candidates[c]!.score; } @@ -742,14 +791,14 @@ class BeatDetectorProcessor extends DisposableProcessor { // Only fire tempoChange if > 5% different if (Math.abs(this._bestBpm - oldBpm) / oldBpm > 0.05) { - this.port.postMessage({ type: 'tempoChange', newTempo: this._bestBpm, oldTempo: oldBpm }); + this.port.postMessage({ type: 'tempoChange', epoch: this._epoch, newTempo: this._bestBpm, oldTempo: oldBpm }); } } else { this._bestScore = currentScore; } // --- Fast-window drift adaptation: follows a genuine, sustained DJ drift. --- - this._trackDrift(fast); + this._trackDrift(this._fastCandidates, fastCount); } this._updateConfidence(); @@ -761,8 +810,8 @@ class BeatDetectorProcessor extends DisposableProcessor { // consecutive ACF hops, the grid follows it. Octave-scale and hard-jump disagreements are // left to the stable hysteresis above (it carries the octave guards); single noisy hops are // filtered out by the persistence streak - so a static tempo is never made nervous. - private _trackDrift(fast: TempoCandidateResult[]): void { - if (fast.length === 0) { + private _trackDrift(fast: TempoCandidateResult[], count = fast.length): void { + if (count === 0) { this._driftHops = 0; this._driftBpm = 0; @@ -799,7 +848,7 @@ class BeatDetectorProcessor extends DisposableProcessor { this._driftHops = 0; // restart the streak; the next step must re-confirm if (Math.abs(this._bestBpm - oldBpm) / oldBpm > 0.05) { - this.port.postMessage({ type: 'tempoChange', newTempo: this._bestBpm, oldTempo: oldBpm }); + this.port.postMessage({ type: 'tempoChange', epoch: this._epoch, newTempo: this._bestBpm, oldTempo: oldBpm }); } } } @@ -905,13 +954,7 @@ class BeatDetectorProcessor extends DisposableProcessor { const onsetSample = this._sampleCount - this._hopSize + frac * this._hopSize; this._lastOnsetSample = onsetSample; - this._onsetRingSamples[this._onsetRingPos] = onsetSample; - this._onsetRingStrengths[this._onsetRingPos] = norm; - this._onsetRingPos = (this._onsetRingPos + 1) % this._onsetRingSamples.length; - - if (this._onsetRingCount < this._onsetRingSamples.length) { - this._onsetRingCount++; - } + this._onsets.push(onsetSample, flux, norm); } // Shift history for the next hop. @@ -925,57 +968,11 @@ class BeatDetectorProcessor extends DisposableProcessor { // louder neighbour (e.g. a bright off-beat hat) must not be allowed to yank the grid. // Returns the onset sample, or -1 if none. private _nearestOnset(targetSample: number, windowSamples: number): number { - let best = -1; - let bestDist = windowSamples + 1; - const cnt = this._onsetRingCount; - - for (let i = 0; i < cnt; i++) { - const s = this._onsetRingSamples[i]!; - - if (s < 0) { - continue; - } - - let d = s - targetSample; - - if (d < 0) { - d = -d; - } - - if (d <= windowSamples && d < bestDist) { - bestDist = d; - best = s; - } - } - - return best; + return this._onsets.nearest(targetSample, windowSamples); } - // Strongest onset no older than maxAgeSamples (ties → most recent) - the phase anchor for - // the first beat. Anchoring to the strongest recent transient, rather than merely the - // latest, biases the bootstrap toward a real beat onset. Always a real onset - never the old - // arbitrary settling boundary. Returns the anchor sample, or -1 if the ring is empty. private _bootstrapOnset(maxAgeSamples: number): number { - let best = -1; - let bestStrength = -1; - const cnt = this._onsetRingCount; - - for (let i = 0; i < cnt; i++) { - const s = this._onsetRingSamples[i]!; - - if (s < 0 || this._sampleCount - s > maxAgeSamples) { - continue; - } - - const st = this._onsetRingStrengths[i]!; - - if (st > bestStrength || (st === bestStrength && s > best)) { - bestStrength = st; - best = s; - } - } - - return best; + return this._onsets.bootstrap(this._sampleCount, maxAgeSamples); } // Emit exactly one beat at beatSample (samples). Centralises the bar/confidence/lookahead @@ -984,7 +981,13 @@ class BeatDetectorProcessor extends DisposableProcessor { // and the bar grid stays consistent even while provisional beats are suppressed); only the port // messages are gated. private _emitBeat(beatSample: number, flux: number): void { + if (this._sampleCount < this._minEmitHops * this._hopSize) { + return; + } + const beatTime = this._contextTimeAt(beatSample); + const evidence = this._beatEvidence; + captureBeatEvidence(evidence, this._epoch, beatSample, this._sampleCount, this._ibiSamples, this._matchedOnset); this._lastBeatContextTime = beatTime; @@ -992,10 +995,15 @@ class BeatDetectorProcessor extends DisposableProcessor { this._ibiHistory[this._ibiIdx] = this._ibiSamples; this._ibiIdx = (this._ibiIdx + 1) & 3; - this._beatsSinceStart++; + if (this._automaticMeter) { + this._meter.addBeat(beatSample, this._ibiSamples, this._matchedOnset.id); + this._meter.update(this._sampleCount, this._onsets, this._fftSize); + } + this._updateBarPosition(flux); this._updateConfidence(); this._updateLookahead(beatTime); + evidence.beatConfidence = this._beatConfidence; // Provisional→locked promotion (latched, one transition per stable segment). Locked needs // the SAME evidence the old detector waited for before emitting at all - the full stable @@ -1015,6 +1023,12 @@ class BeatDetectorProcessor extends DisposableProcessor { } const status = this._locked ? 'locked' : 'provisional'; + const meterKnown = this._meterKnown; + const isDownbeat = meterKnown && this._barPosition === 1; + + if (isDownbeat) { + this._barNumber++; + } // Provisional gating: when emitProvisionalBeats is false, suppress the message until the // beat is locked (grid bookkeeping above already ran). Locked beats always post. @@ -1022,15 +1036,19 @@ class BeatDetectorProcessor extends DisposableProcessor { return; } - const isDownbeat = this._barPosition === 1; this.port.postMessage({ type: 'beat', + epoch: this._epoch, audioTime: beatTime, tempo: this._bestBpm, confidence: this._confidence, beatPhase: 0, energy: flux, + beatConfidence: this._beatConfidence, + onsetEnergy: evidence.rawFlux, + onsetAudioTime: evidence.onsetSample !== null ? this._contextTimeAt(evidence.onsetSample) : null, isDownbeat, + meterKnown, beatInBar: this._barPosition, status, }); @@ -1038,6 +1056,7 @@ class BeatDetectorProcessor extends DisposableProcessor { if (isDownbeat) { this.port.postMessage({ type: 'barStart', + epoch: this._epoch, audioTime: beatTime, tempo: this._bestBpm, confidence: this._confidence, @@ -1170,139 +1189,37 @@ class BeatDetectorProcessor extends DisposableProcessor { } private _updateBarPosition(flux: number): void { - const likelihood = this._computeBeatLikelihood(flux); - - // --- 4/4 posterior --- - const p4 = this._posterior4; - const s4 = new Float32Array(4); - s4[0] = p4[3]!; - s4[1] = p4[0]!; - s4[2] = p4[1]!; - s4[3] = p4[2]!; - let sum4 = 0; - - for (let i = 0; i < 4; i++) { - p4[i] = s4[i]! * (likelihood + (i === 0 ? 0.3 : 0)); - sum4 += p4[i]!; - } - - if (sum4 > 0) { - for (let i = 0; i < 4; i++) { - p4[i]! /= sum4; - } - } - - // --- 3/4 posterior --- - const p3 = this._posterior3; - const s3 = new Float32Array(3); - s3[0] = p3[2]!; - s3[1] = p3[0]!; - s3[2] = p3[1]!; - let sum3 = 0; - - for (let i = 0; i < 3; i++) { - p3[i] = s3[i]! * (likelihood + (i === 0 ? 0.3 : 0)); - sum3 += p3[i]!; - } - - if (sum3 > 0) { - for (let i = 0; i < 3; i++) { - p3[i]! /= sum3; - } - } - - // --- Update TS confidences (EMA) --- - let max4 = 0; - - for (let i = 0; i < 4; i++) { - if (p4[i]! > max4) { - max4 = p4[i]!; - } - } - - let max3 = 0; - - for (let i = 0; i < 3; i++) { - if (p3[i]! > max3) { - max3 = p3[i]!; - } - } - - const alpha = 0.1; - this._ts4Confidence = (1 - alpha) * this._ts4Confidence + alpha * max4; - this._ts3Confidence = (1 - alpha) * this._ts3Confidence + alpha * max3; + this._legacyBars.update(this._computeBeatLikelihood(flux)); - // --- Hysteresis switching --- - if (this._enableTsDetection && this._beatsSinceStart > 8) { - const minSwitchMargin = 1.4; - const minSustainBeats = 12; // ~4 bars * 3 beats - const threeFavored = this._ts3Confidence > this._ts4Confidence * minSwitchMargin; - const fourFavored = this._ts4Confidence > this._ts3Confidence * minSwitchMargin; - - if (this._activeTs === '4/4' && threeFavored) { - this._sustainCounter++; - - if (this._sustainCounter >= minSustainBeats) { - this._activeTs = '3/4'; - this._sustainCounter = 0; - } - } else if (this._activeTs === '3/4' && fourFavored) { - this._sustainCounter++; - - if (this._sustainCounter >= minSustainBeats + 4) { - // 16 beats for 4/4 - this._activeTs = '4/4'; - this._sustainCounter = 0; - } - } else { - this._sustainCounter = 0; - } + if (!this._automaticMeter) { + this._barPosition = this._conventionPhase + 1; + this._conventionPhase = (this._conventionPhase + 1) % this._barLength; + } else { + this._barPosition = this._meter.phasePulses !== null ? this._meter.phasePulses + 1 : this._legacyBars.barPosition; } + } - // --- Determine bar position from active TS --- - const barLen = this._activeTs === '3/4' ? 3 : 4; - const posterior = this._activeTs === '3/4' ? p3 : p4; - - if (this._beatsSinceStart >= barLen) { - let maxP = -1; - let maxI = 0; - - for (let i = 0; i < barLen; i++) { - if (posterior[i]! > maxP) { - maxP = posterior[i]!; - maxI = i; - } - } - - const newPos = maxI + 1; // 1-indexed - - if (newPos === 1 && this._barPosition !== 1) { - this._barNumber++; - } - - this._barPosition = newPos; - } else { - // Just advance sequentially - this._barPosition = (this._barPosition % barLen) + 1; + private get _activeTs(): '3/4' | '4/4' { + return this._legacyBars.activeTs; + } - if (this._barPosition === 1) { - this._barNumber++; - } - } + private set _activeTs(value: '3/4' | '4/4') { + this._legacyBars.activeTs = value; } private _updateConfidence(): void { - if (this._candidates.length === 0) { + if (this._candidateCount === 0) { this._confidence = 0; + this._beatConfidence = 0; this._phaseConfidence = 0; return; } // Peak contrast - const top1 = this._candidates[0] ? this._candidates[0].score : 0; - const top2 = this._candidates[1] ? this._candidates[1].score : top1; - const top3 = this._candidates[2] ? this._candidates[2].score : top2; + const top1 = this._candidates[0]!.score; + const top2 = this._candidateCount > 1 ? this._candidates[1]!.score : top1; + const top3 = this._candidateCount > 2 ? this._candidates[2]!.score : top2; const peakContrast = top2 + top3 > 0 ? top1 / ((top2 + top3) / 2) : 1; // Phase consistency from IBI variance @@ -1324,7 +1241,7 @@ class BeatDetectorProcessor extends DisposableProcessor { const phaseConsistency = ibiMean > 0 ? Math.max(0, 1 - ibiVar / (ibiMean * ibiMean)) : 0; // Bar consistency (use the active posterior) - const activePosterior = this._activeTs === '3/4' ? this._posterior3 : this._posterior4; + const activePosterior = this._activeTs === '3/4' ? this._legacyBars.posterior3 : this._legacyBars.posterior4; const activeLen = this._activeTs === '3/4' ? 3 : 4; let maxP = 0; @@ -1336,15 +1253,63 @@ class BeatDetectorProcessor extends DisposableProcessor { const barConsistency = maxP; - const c = Math.sqrt(Math.max(0, peakContrast / 2)) * Math.sqrt(Math.max(0, phaseConsistency)) * (0.5 + 0.5 * barConsistency); + const pulse = Math.sqrt(Math.max(0, peakContrast / 2)) * Math.sqrt(Math.max(0, phaseConsistency)); + this._beatConfidence = Math.max(0, Math.min(1, pulse)); + const c = pulse * (0.5 + 0.5 * barConsistency); this._confidence = Math.max(0, Math.min(1, c)); } + private get _automaticMeter(): boolean { + return this._enableTsDetection && this._manualMeter === null; + } + + private get _meterKnown(): boolean { + if (this._manualMeter) { + return this._manualMeter.phasePulses != null; + } + + return !this._enableTsDetection || (this._meter.cyclePulses > 0 && this._meter.phasePulses !== null); + } + + private _meterView(settled: boolean, analysisTime: number): MeterEstimate { + return createMeterEstimate({ + meter: this._meter, + manualMeter: this._manualMeter, + automatic: this._automaticMeter, + settled, + analysisTime, + meterKnown: this._meterKnown, + cyclePulses: this._barLength, + phasePulses: this._barPosition - 1, + periodSeconds: this._ibiSamples / this._sampleRate, + sampleRate: this._sampleRate, + sample: this._sampleCount, + predictionValidUntilSample: this._predictionValidUntilSample, + hasBeat: this._lastBeatSample >= 0, + }); + } + + private get _barLength(): number { + if (this._manualMeter) { + return this._manualMeter.cyclePulses; + } + + if (this._enableTsDetection && this._meter.cyclePulses !== 0) { + return this._meter.cyclePulses; + } + + return this._activeTs === '3/4' ? 3 : 4; + } + private _updateLookahead(lastBeatTime: number): void { const lookahead: UpcomingBeatInline[] = []; const beatInterval = 60 / this._bestBpm; const barPos = this._barPosition; - const barLen = this._activeTs === '3/4' ? 3 : 4; + const barLen = this._barLength; + this._predictionValidUntilSample = this._lastOnsetSample >= 0 ? this._lastOnsetSample + 2 * this._ibiSamples + this._fftSize : 0; + this._predictionGeneratedAtSample = this._sampleCount; + this._predictionPeriodSamples = beatInterval * this._sampleRate; + this._predictionStatus = this._locked ? 'locked' : 'provisional'; for (let i = 0; i < 8; i++) { const t = lastBeatTime + (i + 1) * beatInterval; @@ -1353,7 +1318,8 @@ class BeatDetectorProcessor extends DisposableProcessor { lookahead.push({ audioTime: t, tempo: this._bestBpm, - isDownbeat: bp === 1, + isDownbeat: this._meterKnown && bp === 1, + meterKnown: this._meterKnown, beatInBar: bp, }); } @@ -1392,7 +1358,8 @@ class BeatDetectorProcessor extends DisposableProcessor { // The state tempo report honours the LOCKED latch (not the old fixed settling window). // Provisional early beats drive the visual layer, but tempo/confidence/tempoCandidates // and the sync-critical fields stay zero until the detector has a trustworthy locked grid. - const settled = this._locked; + const valid = this._predictionValidUntilSample > this._sampleCount; + const settled = this._locked && valid; const beatInterval = this._bestBpm > 0 ? 60 / this._bestBpm : 0; // Beat phase is measured in analysed samples, not on the context clock: a @@ -1406,7 +1373,7 @@ class BeatDetectorProcessor extends DisposableProcessor { const lastBeatTime = this._lastBeatContextTime; const nextBeatTime = lastBeatTime + beatInterval; - let nextDownbeatTime = nextBeatTime; + let nextDownbeatTime = 0; for (let i = 0; i < this._lookahead.length; i++) { if (this._lookahead[i]!.isDownbeat) { @@ -1416,24 +1383,39 @@ class BeatDetectorProcessor extends DisposableProcessor { } const be = this._computeBandEnergy(); + const tempoCandidates = new Array<{ bpm: number; score: number }>(settled ? this._candidateCount : 0); + + if (settled) { + for (let i = 0; i < this._candidateCount; i++) { + tempoCandidates[i] = { bpm: this._candidates[i]!.bpm, score: this._candidates[i]!.score }; + } + } this.port.postMessage({ type: 'state', + epoch: this._epoch, analysisTime, tempo: settled ? this._bestBpm : 0, beatPhase, confidence: settled ? this._confidence : 0, + beatConfidence: valid ? this._beatConfidence : 0, phaseConfidence: settled ? this._phaseConfidence : 0, gridStability: settled ? this._confidence : 0, - tempoCandidates: settled ? this._candidates.map(c => ({ bpm: c.bpm, score: c.score })) : [], + tempoCandidates, rms: this._rms, onsetStrength: this._onsetStrength, bandEnergy: be, barPosition: this._barPosition, - barLength: this._activeTs === '3/4' ? 3 : 4, - timeSignature: this._activeTs === '3/4' ? { numerator: 3, denominator: 4 } : { numerator: 4, denominator: 4 }, - lookahead: this._lookahead, + barLength: this._barLength, + timeSignature: { numerator: this._barLength, denominator: this._manualMeter?.denominator ?? 4 }, + meterEstimate: this._meterView(settled, analysisTime), + lookahead: valid ? this._lookahead : [], lookaheadRevision: this._lookaheadRevision, + predictionValidUntil: valid ? this._contextTimeAt(this._predictionValidUntilSample) : 0, + predictionGeneratedAtAudioTime: this._contextTimeAt(this._predictionGeneratedAtSample), + predictionEvidenceThroughAudioTime: this._contextTimeAt(this._predictionGeneratedAtSample), + predictionPulsePeriodSeconds: this._predictionPeriodSamples / this._sampleRate, + predictionStatus: this._predictionStatus, nextBeatTime: settled ? nextBeatTime : 0, nextDownbeatTime: settled ? nextDownbeatTime : 0, }); diff --git a/packages/exojs-audio-fx/src/worklets/beatEvidence.ts b/packages/exojs-audio-fx/src/worklets/beatEvidence.ts new file mode 100644 index 000000000..33146725a --- /dev/null +++ b/packages/exojs-audio-fx/src/worklets/beatEvidence.ts @@ -0,0 +1,71 @@ +export interface BeatEvidence { + epoch: number; + beatId: number; + gridSample: number; + emittedAtSample: number; + periodSamples: number; + onsetId: number; + onsetSample: number | null; + rawFlux: number | null; + normalizedStrength: number | null; + beatConfidence: number; + kind: 'matched' | 'freeRun'; + phaseResidualSamples: number | null; +} + +export interface OnsetEvidence { + id: number; + sample: number; + rawFlux: number; + strength: number; +} + +export const captureBeatEvidence = ( + evidence: BeatEvidence, + epoch: number, + gridSample: number, + emittedAtSample: number, + periodSamples: number, + onset: OnsetEvidence, +): void => { + evidence.epoch = epoch; + evidence.beatId++; + evidence.gridSample = gridSample; + evidence.emittedAtSample = emittedAtSample; + evidence.periodSamples = periodSamples; + evidence.onsetId = onset.id; + evidence.kind = onset.id > 0 ? 'matched' : 'freeRun'; + evidence.phaseResidualSamples = onset.id > 0 ? onset.sample - gridSample : null; + evidence.onsetSample = onset.id > 0 ? onset.sample : null; + evidence.rawFlux = onset.id > 0 ? onset.rawFlux : null; + evidence.normalizedStrength = onset.id > 0 ? onset.strength : null; +}; + +export const createBeatEvidence = (): BeatEvidence => ({ + epoch: 0, + beatId: 0, + gridSample: 0, + emittedAtSample: 0, + periodSamples: 0, + onsetId: 0, + onsetSample: null, + rawFlux: null, + normalizedStrength: null, + beatConfidence: 0, + kind: 'freeRun', + phaseResidualSamples: null, +}); + +export const resetBeatEvidence = (evidence: BeatEvidence, epoch: number): void => { + evidence.epoch = epoch; + evidence.gridSample = 0; + evidence.emittedAtSample = 0; + evidence.periodSamples = 0; + evidence.onsetId = 0; + evidence.onsetSample = null; + evidence.rawFlux = null; + evidence.normalizedStrength = null; + evidence.beatConfidence = 0; + evidence.kind = 'freeRun'; + evidence.phaseResidualSamples = null; +}; diff --git a/packages/exojs-audio-fx/src/worklets/meterEstimateView.ts b/packages/exojs-audio-fx/src/worklets/meterEstimateView.ts new file mode 100644 index 000000000..a6b3c689b --- /dev/null +++ b/packages/exojs-audio-fx/src/worklets/meterEstimateView.ts @@ -0,0 +1,82 @@ +import type { ManualMeter, MeterCandidate, MeterEstimate, MeterModel } from '../meterTypes'; +import type { MeterEvidence } from './MeterEvidence'; + +interface MeterViewContext { + meter: MeterEvidence; + manualMeter: ManualMeter | null; + automatic: boolean; + settled: boolean; + meterKnown: boolean; + cyclePulses: number; + phasePulses: number; + periodSeconds: number; + sampleRate: number; + sample: number; + analysisTime: number; + predictionValidUntilSample: number; + hasBeat: boolean; +} + +export const createMeterEstimate = (context: MeterViewContext): MeterEstimate => { + const { meter, manualMeter, automatic, settled, analysisTime } = context; + const model = (cyclePulses: number, phasePulses: number | null): MeterModel => ({ + cyclePulses, + phasePulses, + pulsePeriodSeconds: context.periodSeconds, + denominator: manualMeter?.denominator ?? null, + }); + let selected: MeterModel | null = null; + let reason: MeterEstimate['reason'] = 'acquiring'; + let origin: MeterEstimate['origin'] = 'estimated'; + let score = 0; + let deadline = context.predictionValidUntilSample; + + if (automatic) { + deadline = Math.min(deadline, meter.validUntilSample); + + if (settled) { + reason = meter.reason; + score = meter.score; + + if (meter.cyclePulses > 0) { + selected = model(meter.cyclePulses, meter.phasePulses); + } + } + } else { + origin = manualMeter ? 'manual' : 'convention'; + + if (settled) { + selected = model(context.cyclePulses, context.meterKnown ? context.phasePulses : null); + score = 1; + reason = context.meterKnown ? 'available' : 'ambiguousPhase'; + } + } + + if (!settled && context.hasBeat && context.predictionValidUntilSample <= context.sample) { + reason = 'expired'; + } + + const alternatives: MeterCandidate[] = []; + + if (settled && automatic) { + for (let cycle = 3; cycle <= 4; cycle++) { + const candidateScore = meter.candidateScores[cycle - 3]!; + + if (candidateScore > 0) { + alternatives.push({ model: model(cycle, meter.candidatePhase(cycle)), score: candidateScore }); + } + } + + alternatives.sort((a, b) => b.score - a.score); + } + + return { + selected, + alternatives, + score, + origin, + reason, + evidenceThroughAudioTime: analysisTime, + validUntilAudioTime: deadline > context.sample ? analysisTime + (deadline - context.sample) / context.sampleRate : 0, + }; +}; diff --git a/packages/exojs-audio-fx/src/worklets/meterModelParameters.ts b/packages/exojs-audio-fx/src/worklets/meterModelParameters.ts new file mode 100644 index 000000000..cf8479d80 --- /dev/null +++ b/packages/exojs-audio-fx/src/worklets/meterModelParameters.ts @@ -0,0 +1,14 @@ +export interface MeterModelParameters { + readonly channelPriors: readonly [number, number, number, number]; + readonly minimumScore: number; + readonly cycleMargin: number; + readonly phaseMargin: number; +} + +export const meterModelParameters: MeterModelParameters = Object.freeze({ + // PCM attack power is independent of FFT-window alignment; band novelty is not. + channelPriors: Object.freeze([4, 1, 1, 1] as const), + minimumScore: 0.5, + cycleMargin: 0.15, + phaseMargin: 0.35, +}); diff --git a/packages/exojs-audio-fx/test/beat-detector-evidence.test.ts b/packages/exojs-audio-fx/test/beat-detector-evidence.test.ts new file mode 100644 index 000000000..c5197152f --- /dev/null +++ b/packages/exojs-audio-fx/test/beat-detector-evidence.test.ts @@ -0,0 +1,91 @@ +import { getAudioContext } from '@codexo/exojs'; + +import { BeatDetector } from '../src/BeatDetector'; + +describe('beat detector public evidence', () => { + beforeEach(() => { + const context = getAudioContext(); + (context as unknown as { audioWorklet: { addModule: ReturnType } }).audioWorklet.addModule = vi + .fn() + .mockResolvedValue(undefined); + vi.spyOn(URL, 'createObjectURL').mockReturnValue('blob:beat-evidence'); + vi.spyOn(URL, 'revokeObjectURL').mockImplementation(() => undefined); + }); + afterEach(() => vi.restoreAllMocks()); + + it('keeps pulse confidence independent and expires cached rhythm without another message', async () => { + const clock = vi.spyOn(getAudioContext(), 'currentTime', 'get').mockReturnValue(10); + const detector = new BeatDetector(); + + try { + await detector.ready; + const port = (detector as unknown as { _workletNode: AudioWorkletNode })._workletNode.port; + const state = { + type: 'state', + epoch: 0, + tempo: 120, + beatPhase: 0.01, + confidence: 0.2, + beatConfidence: 0.75, + phaseConfidence: 0.8, + gridStability: 0.2, + predictionValidUntil: 11, + tempoCandidates: [{ bpm: 120, score: 0.5 }], + }; + port.onmessage!.call(port, { data: state } as MessageEvent); + expect(detector.beatConfidence).toBe(0.75); + port.onmessage!.call(port, { data: { ...state, confidence: 0.9, gridStability: 0.9 } } as MessageEvent); + expect(detector.beatConfidence).toBe(0.75); + expect(detector.confidence).toBe(0.9); + clock.mockReturnValue(11); + expect(detector.beatConfidence).toBe(0); + expect(detector.tempo).toBe(0); + expect(detector.confidence).toBe(0); + expect(detector.phaseConfidence).toBe(0); + expect(detector.gridStability).toBe(0); + expect(detector.beatPhase).toBe(0); + expect(detector.tempoCandidates).toEqual([]); + expect(detector.pulse).toBe(0); + expect(detector.justBeat).toBe(false); + expect(detector.subdivisionPhase(4)).toBe(0); + detector.reset(); + port.onmessage!.call(port, { data: { ...state, predictionValidUntil: 20 } } as MessageEvent); + expect(detector.beatConfidence).toBe(0); + } finally { + detector.destroy(); + } + }); + + it('delivers matched onset evidence separately from emission energy and preserves missing evidence', async () => { + const detector = new BeatDetector(); + const receive = vi.fn(); + detector.onBeat.add(receive); + + try { + await detector.ready; + const port = (detector as unknown as { _workletNode: AudioWorkletNode })._workletNode.port; + const beat = { + type: 'beat', + epoch: 0, + audioTime: 10, + tempo: 120, + confidence: 0.2, + beatConfidence: 0.8, + beatPhase: 0, + energy: 0, + onsetEnergy: 7, + onsetAudioTime: 9.97, + meterKnown: false, + isDownbeat: false, + beatInBar: 1, + status: 'provisional', + }; + port.onmessage!.call(port, { data: beat } as MessageEvent); + expect(receive.mock.calls[0]![0]).toMatchObject({ energy: 0, onsetEnergy: 7, onsetAudioTime: 9.97, beatConfidence: 0.8 }); + port.onmessage!.call(port, { data: { ...beat, onsetEnergy: null, onsetAudioTime: null } } as MessageEvent); + expect(receive.mock.calls[1]![0]).toMatchObject({ onsetEnergy: null, onsetAudioTime: null }); + } finally { + detector.destroy(); + } + }); +}); diff --git a/packages/exojs-audio-fx/test/beat-detector-lifecycle.test.ts b/packages/exojs-audio-fx/test/beat-detector-lifecycle.test.ts new file mode 100644 index 000000000..db053ddea --- /dev/null +++ b/packages/exojs-audio-fx/test/beat-detector-lifecycle.test.ts @@ -0,0 +1,125 @@ +import { getAudioContext } from '@codexo/exojs'; + +import { BeatDetector } from '../src/BeatDetector'; + +interface TestNode { + port: { onmessage: ((event: { data: Record }) => void) | null; postMessage: ReturnType }; +} + +const nodeOf = (detector: BeatDetector): TestNode => (detector as unknown as { _workletNode: TestNode })._workletNode; +const epochOf = (detector: BeatDetector): number => (detector as unknown as { _epoch: number })._epoch; + +const send = (detector: BeatDetector, data: Record): void => { + nodeOf(detector).port.onmessage?.({ data }); +}; + +describe('beat detector analysis lifetime', () => { + beforeEach(() => { + const context = getAudioContext(); + (context as unknown as { audioWorklet: { addModule: ReturnType } }).audioWorklet.addModule = vi + .fn() + .mockResolvedValue(undefined); + vi.spyOn(URL, 'createObjectURL').mockReturnValue('blob:beat-lifetime'); + vi.spyOn(URL, 'revokeObjectURL').mockImplementation(() => undefined); + }); + + afterEach(() => vi.restoreAllMocks()); + + it('clears cached state on source change and rejects the previous source messages', async () => { + const detector = new BeatDetector(); + await detector.ready; + const beat = vi.fn(); + detector.onBeat.add(beat); + send(detector, { type: 'state', epoch: 0, tempo: 120, confidence: 0.9, predictionValidUntil: getAudioContext().currentTime + 1 }); + expect(detector.tempo).toBe(120); + + detector.source = getAudioContext().createGain(); + expect(detector.tempo).toBe(0); + expect(detector.confidence).toBe(0); + expect(detector.lookahead).toEqual([]); + send(detector, { type: 'state', epoch: 0, tempo: 180 }); + send(detector, { type: 'beat', epoch: 0, audioTime: 0 }); + expect(detector.tempo).toBe(0); + expect(beat).not.toHaveBeenCalled(); + expect(nodeOf(detector).port.postMessage).toHaveBeenCalledWith({ + type: 'reset', + epoch: epochOf(detector), + manualMeter: null, + stopped: false, + }); + detector.destroy(); + }); + + it('treats reassigning the same source as continuous and null as a change', async () => { + const source = getAudioContext().createGain(); + const detector = new BeatDetector({ source }); + await detector.ready; + const epoch = epochOf(detector); + detector.source = source; + expect(epochOf(detector)).toBe(epoch); + detector.source = null; + expect(epochOf(detector)).toBe(epoch + 1); + detector.destroy(); + }); + + it('resets explicitly, discards in-flight messages and stays disposed', async () => { + const detector = new BeatDetector(); + await detector.ready; + const node = nodeOf(detector); + const receive = node.port.onmessage!; + (detector as BeatDetector & { reset(): void }).reset(); + expect(epochOf(detector)).toBe(1); + receive({ data: { type: 'state', epoch: 0, tempo: 120 } }); + expect(detector.tempo).toBe(0); + detector.destroy(); + detector.destroy(); + receive({ data: { type: 'state', epoch: 1, tempo: 120 } }); + expect(detector.tempo).toBe(0); + expect(node.port.onmessage).toBeNull(); + }); + + it('does not dispatch downbeat after an onBeat listener resets the detector', async () => { + const detector = new BeatDetector(); + await detector.ready; + const downbeat = vi.fn(); + detector.onBeat.add(() => (detector as BeatDetector & { reset(): void }).reset()); + detector.onDownbeat.add(downbeat); + send(detector, { type: 'beat', epoch: 0, isDownbeat: true }); + expect(downbeat).not.toHaveBeenCalled(); + detector.destroy(); + }); + + it.each(['eof', 'explicitPause'] as const)('stops current-epoch events after %s until resumed', async reason => { + const detector = new BeatDetector({ manualMeter: { cyclePulses: 3, phasePulses: 0 } }); + await detector.ready; + const beat = vi.fn(); + detector.onBeat.add(beat); + detector.notifyDiscontinuity(reason); + expect(nodeOf(detector).port.postMessage).toHaveBeenLastCalledWith({ + type: 'reset', + epoch: 1, + manualMeter: detector.manualMeter, + stopped: true, + }); + send(detector, { type: 'beat', epoch: 1, isDownbeat: true }); + expect(beat).not.toHaveBeenCalled(); + detector.manualMeter = { cyclePulses: 4, phasePulses: 0 }; + expect(nodeOf(detector).port.postMessage).toHaveBeenLastCalledWith({ + type: 'reset', + epoch: 2, + manualMeter: detector.manualMeter, + stopped: true, + }); + detector.notifyDiscontinuity('resume'); + expect(nodeOf(detector).port.postMessage).toHaveBeenLastCalledWith({ + type: 'reset', + epoch: 3, + manualMeter: detector.manualMeter, + stopped: false, + }); + send(detector, { type: 'beat', epoch: 3, audioTime: 10, tempo: 120 }); + expect(beat).toHaveBeenCalledTimes(1); + expect(detector.manualMeter?.cyclePulses).toBe(4); + detector.destroy(); + }); +}); diff --git a/packages/exojs-audio-fx/test/beat-detector-manual-meter.test.ts b/packages/exojs-audio-fx/test/beat-detector-manual-meter.test.ts new file mode 100644 index 000000000..485e75d08 --- /dev/null +++ b/packages/exojs-audio-fx/test/beat-detector-manual-meter.test.ts @@ -0,0 +1,54 @@ +import { getAudioContext } from '@codexo/exojs'; + +import { BeatDetector } from '../src/BeatDetector'; + +describe('manual meter ownership', () => { + it('snapshots configuration, validates before mutation and resets the epoch on changes', async () => { + const context = getAudioContext(); + (context as unknown as { audioWorklet: { addModule: ReturnType } }).audioWorklet.addModule = vi + .fn() + .mockResolvedValue(undefined); + vi.spyOn(URL, 'createObjectURL').mockReturnValue('blob:manual-meter'); + vi.spyOn(URL, 'revokeObjectURL').mockImplementation(() => undefined); + const value = { cyclePulses: 11, phasePulses: 0, denominator: 8 }; + const detector = new BeatDetector({ manualMeter: value }); + + try { + await detector.ready; + value.cyclePulses = 3; + expect(detector.manualMeter).toEqual({ cyclePulses: 11, phasePulses: 0, denominator: 8 }); + expect(Object.isFrozen(detector.manualMeter)).toBe(true); + expect(detector.meterEstimate).toMatchObject({ selected: null, reason: 'acquiring', origin: 'manual' }); + const port = (detector as unknown as { _workletNode: AudioWorkletNode })._workletNode.port; + const post = vi.spyOn(port, 'postMessage'); + + for (const invalid of [ + { cyclePulses: 0 }, + { cyclePulses: 33 }, + { cyclePulses: NaN }, + { cyclePulses: 3, phasePulses: 3 }, + { cyclePulses: 3, denominator: 3 }, + ]) { + expect(() => { + detector.manualMeter = invalid; + }).toThrow(RangeError); + } + + expect(post).not.toHaveBeenCalled(); + detector.manualMeter = { cyclePulses: 3 }; + expect(post).toHaveBeenLastCalledWith({ + type: 'reset', + epoch: 1, + manualMeter: { cyclePulses: 3, phasePulses: null, denominator: null }, + stopped: false, + }); + detector.reset(); + expect(post).toHaveBeenLastCalledWith({ type: 'reset', epoch: 2, manualMeter: detector.manualMeter, stopped: false }); + detector.manualMeter = null; + expect(post).toHaveBeenLastCalledWith({ type: 'reset', epoch: 3, manualMeter: null, stopped: false }); + } finally { + detector.destroy(); + vi.restoreAllMocks(); + } + }); +}); diff --git a/packages/exojs-audio-fx/test/beat-detector-meter.test.ts b/packages/exojs-audio-fx/test/beat-detector-meter.test.ts new file mode 100644 index 000000000..688e7e058 --- /dev/null +++ b/packages/exojs-audio-fx/test/beat-detector-meter.test.ts @@ -0,0 +1,115 @@ +import { getAudioContext } from '@codexo/exojs'; + +import { BeatDetector } from '../src/BeatDetector'; + +describe('beat detector meter view', () => { + it('exposes a nullable current estimate and clears it on expiry and reset', async () => { + const context = getAudioContext(); + (context as unknown as { audioWorklet: { addModule: ReturnType } }).audioWorklet.addModule = vi + .fn() + .mockResolvedValue(undefined); + vi.spyOn(URL, 'createObjectURL').mockReturnValue('blob:meter-view'); + vi.spyOn(URL, 'revokeObjectURL').mockImplementation(() => undefined); + const clock = vi.spyOn(context, 'currentTime', 'get').mockReturnValue(10); + const detector = new BeatDetector(); + + try { + await detector.ready; + expect(detector.meterEstimate).toMatchObject({ selected: null, reason: 'acquiring', alternatives: [] }); + const port = (detector as unknown as { _workletNode: AudioWorkletNode })._workletNode.port; + port.onmessage!.call(port, { + data: { + type: 'state', + epoch: 0, + predictionValidUntil: 11, + tempo: 120, + nextBeatTime: 10.5, + nextDownbeatTime: 12, + lookaheadRevision: 1, + lookahead: [{ audioTime: 12, tempo: 120, beatInBar: 1, isDownbeat: true, meterKnown: true }], + meterEstimate: { + selected: { cyclePulses: 3, phasePulses: 1, pulsePeriodSeconds: 0.5, denominator: null }, + alternatives: [{ model: { cyclePulses: 4, phasePulses: null, pulsePeriodSeconds: 0.5, denominator: null }, score: 0.2 }], + reason: 'available', + score: 0.9, + origin: 'estimated', + evidenceThroughAudioTime: 10, + validUntilAudioTime: 10.8, + }, + }, + } as MessageEvent); + expect(detector.meterEstimate.selected).toMatchObject({ cyclePulses: 3, phasePulses: 1, denominator: null }); + expect(Object.isFrozen(detector.meterEstimate.selected)).toBe(true); + expect(Object.isFrozen(detector.meterEstimate.alternatives[0]!.model)).toBe(true); + expect(detector.meterKnown).toBe(true); + clock.mockReturnValue(10.9); + expect(detector.meterEstimate).toMatchObject({ selected: null, reason: 'expired', alternatives: [] }); + expect(detector.meterEstimate).toBe(detector.meterEstimate); + expect(detector.meterKnown).toBe(false); + expect(detector.nextBeatTime).toBe(12); + expect(detector.nextDownbeatTime).toBe(0); + expect(detector.barPulse).toBe(0); + expect(detector.lookahead).toEqual([{ audioTime: 12, tempo: 120, beatInBar: 1, isDownbeat: false, meterKnown: false }]); + clock.mockReturnValue(10); + const model = { cyclePulses: 3, phasePulses: null, pulsePeriodSeconds: 0.5, denominator: null }; + const alternatives = [ + { model, score: 0.7 }, + { model: { ...model, cyclePulses: 4 }, score: 0.68 }, + ]; + + for (const selected of [model, null]) { + port.onmessage!.call(port, { + data: { + type: 'state', + epoch: 0, + predictionValidUntil: 11, + nextDownbeatTime: 10.5, + meterEstimate: { + selected, + alternatives, + score: 0.7, + origin: 'estimated', + reason: selected ? 'ambiguousPhase' : 'ambiguousCycle', + evidenceThroughAudioTime: 10, + validUntilAudioTime: 10.8, + }, + }, + } as MessageEvent); + expect(detector.meterEstimate.selected).toEqual(selected); + expect(detector.meterEstimate.alternatives).toHaveLength(2); + expect(detector.meterKnown).toBe(false); + expect(detector.nextDownbeatTime).toBe(0); + } + + clock.mockReturnValue(10.9); + expect(detector.meterEstimate).toMatchObject({ selected: null, alternatives: [], reason: 'expired' }); + clock.mockReturnValue(10); + port.onmessage!.call(port, { + data: { + type: 'state', + epoch: 0, + tempo: 120, + predictionValidUntil: 11, + nextBeatTime: 10.5, + nextDownbeatTime: 0, + meterEstimate: { + selected: { ...model, cyclePulses: 11, phasePulses: 0 }, + alternatives: [], + score: 1, + origin: 'manual', + reason: 'available', + evidenceThroughAudioTime: 10, + validUntilAudioTime: 11, + }, + }, + } as MessageEvent); + expect(detector.nextDownbeatTime).toBe(0); + expect(detector.barPulse).toBe(1); + detector.reset(); + expect(detector.meterEstimate).toMatchObject({ selected: null, reason: 'acquiring' }); + } finally { + detector.destroy(); + vi.restoreAllMocks(); + } + }); +}); diff --git a/packages/exojs-audio-fx/test/beat-detector-schedule-validity.test.ts b/packages/exojs-audio-fx/test/beat-detector-schedule-validity.test.ts new file mode 100644 index 000000000..0aba86150 --- /dev/null +++ b/packages/exojs-audio-fx/test/beat-detector-schedule-validity.test.ts @@ -0,0 +1,91 @@ +import { getAudioContext } from '@codexo/exojs'; + +import { BeatDetector } from '../src/BeatDetector'; + +describe('beat detector cached prediction expiry', () => { + beforeEach(() => { + const context = getAudioContext(); + (context as unknown as { audioWorklet: { addModule: ReturnType } }).audioWorklet.addModule = vi + .fn() + .mockResolvedValue(undefined); + vi.spyOn(URL, 'createObjectURL').mockReturnValue('blob:beat-expiry'); + vi.spyOn(URL, 'revokeObjectURL').mockImplementation(() => undefined); + }); + afterEach(() => vi.restoreAllMocks()); + + it('keeps next-time getters unavailable while the tempo grid is provisional', async () => { + const detector = new BeatDetector(); + await detector.ready; + vi.spyOn(getAudioContext(), 'currentTime', 'get').mockReturnValue(1); + const port = (detector as unknown as { _workletNode: AudioWorkletNode })._workletNode.port; + port.onmessage!.call(port, { + data: { + type: 'state', + epoch: 0, + tempo: 0, + nextBeatTime: 0, + nextDownbeatTime: 0, + predictionValidUntil: 2, + lookahead: [{ audioTime: 1.5, tempo: 120, isDownbeat: true, beatInBar: 1 }], + }, + } as MessageEvent); + expect(detector.nextBeatTime).toBe(0); + expect(detector.nextDownbeatTime).toBe(0); + detector.destroy(); + }); + + it('expires retained schedules without requiring another port message', async () => { + const detector = new BeatDetector(); + await detector.ready; + const context = getAudioContext(); + const clock = vi.spyOn(context, 'currentTime', 'get').mockReturnValue(10); + const port = (detector as unknown as { _workletNode: AudioWorkletNode })._workletNode.port; + const receive = port.onmessage!; + receive.call(port, { + data: { + type: 'state', + epoch: 0, + tempo: 120, + confidence: 0.9, + analysisTime: 10, + predictionValidUntil: 11, + nextBeatTime: 10.5, + nextDownbeatTime: 12, + lookaheadRevision: 1, + lookahead: [{ audioTime: 10.5, tempo: 120, isDownbeat: false, beatInBar: 2 }], + }, + } as MessageEvent); + expect(detector.lookahead).toHaveLength(1); + clock.mockReturnValue(11.1); + expect(detector.lookahead).toEqual([]); + expect(detector.nextBeatTime).toBe(0); + expect(detector.nextDownbeatTime).toBe(0); + expect(detector.barPulse).toBe(0); + detector.destroy(); + }); + + it('ignores already expired schedules and preserves new-revision deduplication', async () => { + const detector = new BeatDetector(); + await detector.ready; + const clock = vi.spyOn(getAudioContext(), 'currentTime', 'get').mockReturnValue(20); + const predict = vi.fn(); + detector.onBeatPredicted.add(predict); + const port = (detector as unknown as { _workletNode: AudioWorkletNode })._workletNode.port; + const receive = port.onmessage!; + const state = { + type: 'state', + epoch: 0, + predictionValidUntil: 19, + lookaheadRevision: 1, + lookahead: [{ audioTime: 18, tempo: 120, isDownbeat: true, beatInBar: 1 }], + }; + receive.call(port, { data: state } as MessageEvent); + expect(predict).not.toHaveBeenCalled(); + clock.mockReturnValue(18); + const fresh = { ...state, lookaheadRevision: 2, predictionValidUntil: 21, lookahead: [{ ...state.lookahead[0], audioTime: 20 }] }; + receive.call(port, { data: fresh } as MessageEvent); + receive.call(port, { data: fresh } as MessageEvent); + expect(predict).toHaveBeenCalledTimes(1); + detector.destroy(); + }); +}); diff --git a/packages/exojs-audio-fx/test/beat-detector-schedule-view.test.ts b/packages/exojs-audio-fx/test/beat-detector-schedule-view.test.ts new file mode 100644 index 000000000..9ee7cb022 --- /dev/null +++ b/packages/exojs-audio-fx/test/beat-detector-schedule-view.test.ts @@ -0,0 +1,77 @@ +import { getAudioContext } from '@codexo/exojs'; + +import { BeatDetector } from '../src/BeatDetector'; + +describe('beat schedule interpretation', () => { + it('retains generation metadata, advances future entries and removes expired meter independently', async () => { + const context = getAudioContext(); + (context as unknown as { audioWorklet: { addModule: ReturnType } }).audioWorklet.addModule = vi + .fn() + .mockResolvedValue(undefined); + vi.spyOn(URL, 'createObjectURL').mockReturnValue('blob:schedule-view'); + vi.spyOn(URL, 'revokeObjectURL').mockImplementation(() => undefined); + const clock = vi.spyOn(context, 'currentTime', 'get').mockReturnValue(10); + const detector = new BeatDetector(); + + try { + await detector.ready; + expect(detector.schedule).toBeNull(); + const port = (detector as unknown as { _workletNode: AudioWorkletNode })._workletNode.port; + port.onmessage!.call(port, { + data: { + type: 'state', + epoch: 0, + predictionValidUntil: 11, + lookaheadRevision: 7, + predictionGeneratedAtAudioTime: 9.9, + predictionEvidenceThroughAudioTime: 9.9, + predictionPulsePeriodSeconds: 0.5, + predictionStatus: 'provisional', + meterEstimate: { + selected: { cyclePulses: 3, phasePulses: 2, pulsePeriodSeconds: 0.5, denominator: null }, + alternatives: [], + score: 0.8, + origin: 'estimated', + reason: 'available', + evidenceThroughAudioTime: 10, + validUntilAudioTime: 10.7, + }, + lookahead: [ + { audioTime: 10.5, tempo: 120, beatInBar: 1, isDownbeat: true, meterKnown: true }, + { audioTime: 11, tempo: 120, beatInBar: 2, isDownbeat: false, meterKnown: true }, + ], + }, + } as MessageEvent); + const original = detector.schedule!; + expect(original).toMatchObject({ + epoch: 0, + revision: 7, + generatedAtAudioTime: 9.9, + evidenceThroughAudioTime: 9.9, + validUntilAudioTime: 10.7, + horizonAudioTime: 11, + pulsePeriodSeconds: 0.5, + status: 'provisional', + nextBeat: { audioTime: 10.5 }, + nextDownbeat: { audioTime: 10.5 }, + }); + expect(detector.schedule).toBe(original); + expect(Object.isFrozen(original)).toBe(true); + expect(Object.isFrozen(original.beats[0])).toBe(true); + clock.mockReturnValue(10.6); + expect(detector.schedule).toMatchObject({ revision: 7, nextBeat: { audioTime: 11 }, nextDownbeat: null }); + clock.mockReturnValue(10.8); + expect(detector.schedule!.validUntilAudioTime).toBe(11); + expect(detector.schedule!.beats.every(beat => !beat.meterKnown && !beat.isDownbeat)).toBe(true); + expect(detector.schedule!.nextDownbeat).toBeNull(); + expect(original.beats[0]!.isDownbeat).toBe(true); + clock.mockReturnValue(11); + expect(detector.schedule).toBeNull(); + detector.reset(); + expect(detector.schedule).toBeNull(); + } finally { + detector.destroy(); + vi.restoreAllMocks(); + } + }); +}); diff --git a/packages/exojs-audio-fx/test/beat-detector.test.ts b/packages/exojs-audio-fx/test/beat-detector.test.ts index b731c8aa0..558c818b3 100644 --- a/packages/exojs-audio-fx/test/beat-detector.test.ts +++ b/packages/exojs-audio-fx/test/beat-detector.test.ts @@ -28,7 +28,28 @@ const simulateMessage = (detector: BeatDetector, data: unknown): void => { const node = getMockWorkletNode(detector); if (node?.port.onmessage) { - node.port.onmessage({ data }); + const message = data as Record; + const now = getAudioContext().currentTime; + node.port.onmessage({ + data: { + predictionValidUntil: now + 10, + meterEstimate: { + selected: { + cyclePulses: message.barLength ?? 4, + phasePulses: ((message.barPosition as number) ?? 1) - 1, + pulsePeriodSeconds: 0.5, + denominator: null, + }, + alternatives: [], + reason: 'available', + score: 1, + origin: 'convention', + evidenceThroughAudioTime: message.analysisTime ?? now, + validUntilAudioTime: now + 10, + }, + ...message, + }, + }); } }; @@ -1311,4 +1332,4 @@ describe('BeatDetector', () => { }); // Import for the type in the test above -import type { UpcomingBeat } from '../src/BeatDetector'; +import type { UpcomingBeat } from '../src/beatScheduleTypes'; diff --git a/packages/exojs-audio-fx/test/browser/beat-detector.audio.test.ts b/packages/exojs-audio-fx/test/browser/beat-detector.audio.test.ts new file mode 100644 index 000000000..427d06acf --- /dev/null +++ b/packages/exojs-audio-fx/test/browser/beat-detector.audio.test.ts @@ -0,0 +1,133 @@ +import beatDetectorWorkletSource from '../../src/worklets/beat-detector.worklet.ts?worklet'; + +interface DetectorMessage { + type: string; + epoch: number; + analysisTime: number; + tempo: number; + audioTime: number; + meterKnown: boolean; + isDownbeat: boolean; + confidence: number; + lookahead: unknown[]; + nextBeatTime: number; + nextDownbeatTime: number; + meterEstimate: { selected: { cyclePulses: number } | null }; +} + +describe('beat detector in real Web Audio', () => { + it.each( + [44_100, 48_000].flatMap(sampleRate => + [120, 50].flatMap(bpm => (bpm === 50 ? [3, 4] : [1, 3, 4]).map(cycle => ({ sampleRate, cycle, bpm }))), + ), + )('tracks $cycle pulses at $bpm BPM, resets and expires at $sampleRate Hz', async ({ sampleRate, cycle, bpm }) => { + const seconds = bpm === 50 ? 44 : 16; + const activeSeconds = bpm === 50 ? 36 : 10; + const scoringStart = bpm === 50 ? 20 : 8; + const context = new OfflineAudioContext(1, sampleRate * seconds, sampleRate); + const url = URL.createObjectURL(new Blob([beatDetectorWorkletSource], { type: 'application/javascript' })); + + try { + await context.audioWorklet.addModule(url); + } finally { + URL.revokeObjectURL(url); + } + + const node = new AudioWorkletNode(context, 'exojs-beat-detector', { + numberOfInputs: 1, + numberOfOutputs: 1, + processorOptions: bpm === 50 ? { minBpm: 45, maxBpm: 65 } : {}, + }); + node.connect(context.destination); + const messages: DetectorMessage[] = []; + let acknowledge: () => void; + let finish: () => void; + const acknowledged = new Promise(resolve => { + acknowledge = resolve; + }); + const finished = new Promise(resolve => { + finish = resolve; + }); + const failed = new Promise((_resolve, reject) => { + node.onprocessorerror = () => reject(new Error('Beat detector worklet failed')); + }); + + node.port.onmessage = event => { + const message = event.data as DetectorMessage; + messages.push(message); + + if (message.type === 'reset' && message.epoch === 1) { + acknowledge(); + } else if (message.type === 'state' && message.analysisTime > seconds - 0.2) { + finish(); + } + }; + + try { + node.port.postMessage({ type: 'reset', epoch: 1 }); + await Promise.race([acknowledged, failed]); + const buffer = context.createBuffer(1, sampleRate * seconds, sampleRate); + const pcm = buffer.getChannelData(0); + + for (let beat = 0; 0.173 + (beat * 60) / bpm < activeSeconds; beat++) { + const time = 0.173 + (beat * 60) / bpm; + const start = Math.round(time * sampleRate); + + for (let j = 0; j < sampleRate * 0.02; j++) { + pcm[start + j] = + (beat % cycle === 0 ? 0.9 : 0.3) * Math.exp(-j / (sampleRate * 0.004)) * Math.cos((j * 2 * Math.PI * 1400) / sampleRate); + } + } + + const source = context.createBufferSource(); + source.buffer = buffer; + source.connect(node); + source.start(); + await Promise.race([Promise.all([context.startRendering(), finished]), failed]); + + expect(messages.every(message => message.epoch === 1)).toBe(true); + expect(messages.some(message => message.type === 'state' && message.tempo > 0)).toBe(true); + const active = messages.filter( + message => message.type === 'state' && message.analysisTime > scoringStart && message.analysisTime < activeSeconds, + ); + expect(active.length).toBeGreaterThan(0); + + if (cycle === 1) { + expect(active.every(message => message.meterEstimate.selected === null)).toBe(true); + } else { + expect(active.every(message => message.meterEstimate.selected?.cyclePulses === cycle)).toBe(true); + } + + if (bpm === 50) { + const beats = messages.filter( + message => message.type === 'beat' && message.audioTime >= scoringStart && message.audioTime < activeSeconds, + ); + expect(beats.length).toBeGreaterThan(10); + expect(beats.every(beat => beat.meterKnown)).toBe(true); + const downbeats = beats.filter(beat => beat.isDownbeat); + expect(downbeats.length).toBeGreaterThan(1); + const bars = messages.filter( + message => message.type === 'barStart' && message.audioTime >= scoringStart && message.audioTime < activeSeconds, + ); + expect(bars.map(bar => bar.audioTime)).toEqual(downbeats.map(beat => beat.audioTime)); + } + + const silent = messages.filter(message => message.type === 'state' && message.analysisTime > activeSeconds + 3); + expect(silent.length).toBeGreaterThan(0); + + for (const message of silent) { + expect(message).toMatchObject({ + confidence: 0, + lookahead: [], + nextBeatTime: 0, + nextDownbeatTime: 0, + meterEstimate: { selected: null }, + }); + } + } finally { + node.port.postMessage({ type: 'destroy' }); + node.disconnect(); + node.port.close(); + } + }); +}); diff --git a/packages/exojs-audio-fx/test/dsp/worklet-parity.test.ts b/packages/exojs-audio-fx/test/dsp/worklet-parity.test.ts index b0f5031ab..eb59c2200 100644 --- a/packages/exojs-audio-fx/test/dsp/worklet-parity.test.ts +++ b/packages/exojs-audio-fx/test/dsp/worklet-parity.test.ts @@ -33,6 +33,7 @@ interface ParityProc { _hopSize: number; _sampleRate: number; _candidates: Array<{ bpm: number; score: number; lag: number }>; + _candidateCount: number; } /** Reconstruct the worklet's flux ring buffer as a linear oldest-first array. */ @@ -82,7 +83,7 @@ const driveAndCompare = (samples: Float32Array): void => { maxBpm: proc._maxBpm, }); - expect(worklet.length).toBe(expected.length); + expect(proc._candidateCount).toBe(expected.length); for (let i = 0; i < expected.length; i++) { expect(worklet[i]!.lag).toBe(expected[i]!.lag); diff --git a/packages/exojs-audio-fx/test/harness/beat-sandbox.ts b/packages/exojs-audio-fx/test/harness/beat-sandbox.ts index 25530e0b5..f1effed68 100644 --- a/packages/exojs-audio-fx/test/harness/beat-sandbox.ts +++ b/packages/exojs-audio-fx/test/harness/beat-sandbox.ts @@ -34,6 +34,7 @@ export interface BeatMessage { beatPhase: number; energy: number; isDownbeat: boolean; + meterKnown?: boolean; beatInBar: number; /** Trust level: early reactive beat vs settled, trustworthy beat. */ status: 'provisional' | 'locked'; @@ -179,10 +180,11 @@ export const runDetector = ( processorOptions?: Record; blockSize?: number; startFrame?: number; + sampleRate?: number; prepare?: (processor: Record) => void; } = {}, ): { messages: WorkletMessage[] } => { - const { blockSize = 128, processorOptions = {}, startFrame = 0, prepare } = options; + const { blockSize = 128, processorOptions = {}, startFrame = 0, sampleRate: rate = SAMPLE_RATE, prepare } = options; // Build (or reuse cached) processor class if (!_processorCtor) { @@ -195,7 +197,7 @@ export const runDetector = ( const g = globalThis as Record; const prevSR = g['sampleRate']; const prevCF = g['currentFrame']; - g['sampleRate'] = SAMPLE_RATE; + g['sampleRate'] = rate; g['currentFrame'] = startFrame; const messages: WorkletMessage[] = []; @@ -214,7 +216,7 @@ export const runDetector = ( const n = samples.length; for (let off = 0; off < n; off += blockSize) { - blockStartSec = off / SAMPLE_RATE; + blockStartSec = off / rate; g['currentFrame'] = startFrame + off; const end = Math.min(off + blockSize, n); proc.process([[samples.subarray(off, end)]], [], {}); diff --git a/packages/exojs-audio-fx/test/worklets/beat-detector-cadence.test.ts b/packages/exojs-audio-fx/test/worklets/beat-detector-cadence.test.ts new file mode 100644 index 000000000..57d61ea65 --- /dev/null +++ b/packages/exojs-audio-fx/test/worklets/beat-detector-cadence.test.ts @@ -0,0 +1,52 @@ +import { clicktrack, SAMPLE_RATE } from '../fixtures/beat-fixtures'; +import { type BeatMessage, runDetector } from '../harness/beat-sandbox'; + +describe('beat detector acquisition cadence', () => { + it('keeps candidate-less ACF retries on the configured cadence', () => { + const attempts: number[] = []; + runDetector(new Float32Array(SAMPLE_RATE * 3), { + prepare: processor => { + const compute = processor['_computeACFAndCandidates'] as () => void; + + processor['_computeACFAndCandidates'] = function (this: Record): void { + attempts.push(this['_sampleCount'] as number); + compute.call(this); + }; + }, + }); + + expect(attempts.length).toBeLessThanOrEqual(Math.ceil((3 * SAMPLE_RATE) / (512 * 15)) + 1); + const shortIntervals = attempts.slice(1).filter((sample, i) => sample - attempts[i]! < 512 * 15); + expect(shortIntervals.length).toBeLessThanOrEqual(1); + }); + + it('respects a long minimum warmup on the full-window acquisition path', () => { + const { messages } = runDetector(clicktrack(120, 6).samples, { processorOptions: { minSettlingMs: 4000 } }); + const beats = messages.filter((message): message is BeatMessage => message.type === 'beat'); + + expect(beats.length).toBeGreaterThan(0); + expect(beats[0]!._audioTimeSec + 128 / SAMPLE_RATE).toBeGreaterThanOrEqual(4); + }); + + it.each([44_100, 48_000])('respects warmup beyond the flux-ring capacity at %i Hz', sampleRate => { + const attempts: number[] = []; + runDetector(new Float32Array(12 * sampleRate), { + sampleRate, + processorOptions: { minSettlingMs: 10_001 }, + prepare: processor => { + const compute = processor['_computeACFAndCandidates'] as () => void; + + processor['_computeACFAndCandidates'] = function (this: Record): void { + attempts.push(this['_sampleCount'] as number); + compute.call(this); + }; + }, + }); + expect(attempts.length).toBeGreaterThan(2); + expect(attempts[0]).toBe(Math.ceil((10.001 * sampleRate) / 512) * 512); + + for (let i = 1; i < attempts.length; i++) { + expect(attempts[i]! - attempts[i - 1]!).toBe(15 * 512); + } + }); +}); diff --git a/packages/exojs-audio-fx/test/worklets/beat-detector-candidate-storage.test.ts b/packages/exojs-audio-fx/test/worklets/beat-detector-candidate-storage.test.ts new file mode 100644 index 000000000..d908ee10a --- /dev/null +++ b/packages/exojs-audio-fx/test/worklets/beat-detector-candidate-storage.test.ts @@ -0,0 +1,157 @@ +import { computeTempoCandidates, type TempoCandidateResult } from '../../src/dsp/tempogram'; +import { TempoCandidateScorer } from '../../src/worklets/TempoCandidateScorer'; +import { runDetector } from '../harness/beat-sandbox'; + +interface CandidateProcessor { + _fluxWindow: Float32Array; + _fluxCount: number; + _fluxWritePos: number; + _minLag: number; + _maxLag: number; + _minBpm: number; + _maxBpm: number; + _hopSize: number; + _sampleRate: number; + _candidates: TempoCandidateResult[]; + _fastCandidates: TempoCandidateResult[]; + _candidateCount?: number; + _computeACFAndCandidates(): void; + _resetAnalysis(epoch: number): void; +} + +const fillNovelty = (processor: CandidateProcessor, seed: number): void => { + let state = seed; + + for (let i = 0; i < processor._fluxWindow.length; i++) { + state = (Math.imul(state, 1_664_525) + 1_013_904_223) >>> 0; + processor._fluxWindow[i] = (state / 0x1_0000_0000) * 0.3 + (i % (17 + (seed % 42)) === 0 ? 1 : 0); + } + + processor._fluxCount = processor._fluxWindow.length; + processor._fluxWritePos = (seed * 29) % processor._fluxWindow.length; +}; + +describe('beat detector candidate storage', () => { + it('reuses the candidate container and slots across ACF updates', () => { + runDetector(new Float32Array(0), { + prepare: value => { + const processor = value as unknown as CandidateProcessor; + fillNovelty(processor, 3); + processor._computeACFAndCandidates(); + const candidates = processor._candidates; + const slots = candidates.slice(); + const fastCandidates = processor._fastCandidates; + const fastSlots = fastCandidates.slice(); + expect(slots).toHaveLength(3); + + for (let seed = 4; seed < 12; seed++) { + fillNovelty(processor, seed); + processor._computeACFAndCandidates(); + expect(processor._candidates).toBe(candidates); + expect(processor._fastCandidates).toBe(fastCandidates); + + for (let i = 0; i < slots.length; i++) { + expect(processor._candidates[i]).toBe(slots[i]); + expect(processor._fastCandidates[i]).toBe(fastSlots[i]); + } + } + }, + }); + }); + + it.each([ + { raw: [0.01, 0.02], order: [1, 5, 3] }, + { raw: [0.02, 0.02], order: [1, 3, 5] }, + ])('retains canonical raw-peak and lag order for zero final-score ties: $order', ({ raw, order }) => { + const scorer = new TempoCandidateScorer(48_000, 512, 1, 10_000); + const candidates = Array.from({ length: 3 }, () => ({ bpm: 0, score: 0, lag: 0 })); + const acf = new Float32Array([1, 0, raw[0]!, 0, raw[1]!, 0]); + expect(scorer.scoreInto(acf, 1, candidates)).toBe(3); + expect(candidates.map(candidate => candidate.lag)).toEqual(order); + expect(candidates.slice(1).map(candidate => candidate.score)).toEqual([0, 0]); + }); + + it('keeps published candidate snapshots unchanged by subsequent pooled updates', () => { + runDetector(new Float32Array(0), { + prepare: value => { + const processor = value as unknown as CandidateProcessor; + fillNovelty(processor, 3); + processor._computeACFAndCandidates(); + value['_locked'] = true; + value['_predictionValidUntilSample'] = 48_000; + let snapshot: Array<{ bpm: number; score: number }> = []; + + (value['port'] as { postMessage: (message: { type: string; tempoCandidates: typeof snapshot }) => void }).postMessage = message => { + if (message.type === 'state') { + snapshot = message.tempoCandidates; + } + }; + + (value['_sendStateMessage'] as () => void).call(value); + const expected = snapshot.map(candidate => ({ ...candidate })); + expect(snapshot).toHaveLength(3); + fillNovelty(processor, 7); + processor._computeACFAndCandidates(); + expect(snapshot).toEqual(expected); + expect(processor._candidates[0]!.bpm).not.toBe(snapshot[0]!.bpm); + }, + }); + }); + + it('retains allocated storage while reset and silence clear active candidates', () => { + runDetector(new Float32Array(0), { + prepare: value => { + const processor = value as unknown as CandidateProcessor; + fillNovelty(processor, 7); + processor._computeACFAndCandidates(); + const candidates = processor._candidates; + const slots = candidates.slice(); + processor._resetAnalysis(1); + expect(processor._candidates).toBe(candidates); + processor._computeACFAndCandidates(); + expect(processor._candidateCount ?? processor._candidates.length).toBe(0); + + for (let i = 0; i < slots.length; i++) { + expect(processor._candidates[i]).toBe(slots[i]); + } + }, + }); + }); + + it('preserves canonical candidates across ring wraps, short histories and empty input', () => { + runDetector(new Float32Array(0), { + prepare: value => { + const processor = value as unknown as CandidateProcessor; + + for (let seed = 1; seed <= 80; seed++) { + fillNovelty(processor, seed); + processor._fluxCount = seed % 5 === 0 ? Math.min(seed, processor._fluxWindow.length) : processor._fluxWindow.length; + + if (seed % 8 === 0) { + processor._fluxWindow.fill(0); + } + + const buffer = processor._fluxWindow; + const linear = Float32Array.from({ length: processor._fluxCount }, (_, i) => { + const index = (processor._fluxWritePos - processor._fluxCount + i + buffer.length) % buffer.length; + + return buffer[index]!; + }); + const expected = computeTempoCandidates(linear, processor._minLag, processor._maxLag, processor._hopSize, processor._sampleRate, { + minBpm: processor._minBpm, + maxBpm: processor._maxBpm, + }); + processor._computeACFAndCandidates(); + const actual = processor._candidates.slice(0, processor._candidateCount ?? processor._candidates.length); + expect(actual).toHaveLength(expected.length); + + for (let i = 0; i < expected.length; i++) { + expect(actual[i]!.lag).toBe(expected[i]!.lag); + expect(actual[i]!.bpm).toBeCloseTo(expected[i]!.bpm, 12); + expect(actual[i]!.score).toBeCloseTo(expected[i]!.score, 14); + } + } + }, + }); + }); +}); diff --git a/packages/exojs-audio-fx/test/worklets/beat-detector-confidence.test.ts b/packages/exojs-audio-fx/test/worklets/beat-detector-confidence.test.ts new file mode 100644 index 000000000..887ae6ebf --- /dev/null +++ b/packages/exojs-audio-fx/test/worklets/beat-detector-confidence.test.ts @@ -0,0 +1,34 @@ +import { runDetector } from '../harness/beat-sandbox'; + +describe('beat detector pulse confidence', () => { + it('does not depend on the meter posterior while preserving legacy confidence', () => { + runDetector(new Float32Array(0), { + prepare: value => { + value['_candidates'] = [ + { bpm: 120, score: 0.5 }, + { bpm: 90, score: 0.25 }, + ]; + value['_candidateCount'] = 2; + (value['_ibiHistory'] as Float32Array).fill(24_000); + const update = value['_updateConfidence'] as () => void; + update.call(value); + const pulse = value['_beatConfidence']; + const legacy = value['_confidence']; + expect(pulse).toBeTypeOf('number'); + (value['_legacyBars'] as { posterior4: Float32Array }).posterior4.set([1, 0, 0, 0]); + update.call(value); + expect(value['_beatConfidence']).toBe(pulse); + expect(value['_confidence']).not.toBe(legacy); + }, + }); + }); + + it('reports no pulse confidence without tempo candidates', () => { + runDetector(new Float32Array(0), { + prepare: value => { + (value['_updateConfidence'] as () => void).call(value); + expect(value['_beatConfidence']).toBe(0); + }, + }); + }); +}); diff --git a/packages/exojs-audio-fx/test/worklets/beat-detector-evidence.test.ts b/packages/exojs-audio-fx/test/worklets/beat-detector-evidence.test.ts new file mode 100644 index 000000000..6c182d18b --- /dev/null +++ b/packages/exojs-audio-fx/test/worklets/beat-detector-evidence.test.ts @@ -0,0 +1,83 @@ +import { runDetector } from '../harness/beat-sandbox'; + +describe('beat detector matched onset evidence', () => { + it('retains the selected onset raw flux and normalized strength after another hop', () => { + let matchedSample = 0; + const { messages } = runDetector(new Float32Array(0), { + prepare: processor => { + const detect = processor['_detectOnset'] as (flux: number) => void; + const nearest = processor['_nearestOnset'] as (sample: number, window: number) => number; + + for (let i = 0; i < 8; i++) { + processor['_sampleCount'] = i * 64; + detect.call(processor, 0); + } + + processor['_sampleCount'] = 1024; + detect.call(processor, 7); + processor['_sampleCount'] = 1536; + detect.call(processor, 0); + + const sample = nearest.call(processor, 800, 512); + matchedSample = sample; + expect(sample).toBeGreaterThan(512); + const evidence = processor['_matchedOnset'] as { sample: number; rawFlux: number; strength: number; id: number }; + expect(evidence).toMatchObject({ sample, rawFlux: 7, id: 1 }); + expect(evidence.strength).toBeGreaterThan(3); + expect(evidence.strength).not.toBe(evidence.rawFlux); + + processor['_sampleCount'] = 24_000; + processor['_bestBpm'] = 120; + processor['_ibiSamples'] = 24_000; + (processor['_emitBeat'] as (sample: number, flux: number) => void).call(processor, sample, 0); + expect(processor['_beatEvidence']).toMatchObject({ + onsetId: 1, + onsetSample: sample, + rawFlux: 7, + kind: 'matched', + phaseResidualSamples: 0, + }); + + nearest.call(processor, 20_000, 100); + expect(evidence.id).toBe(0); + (processor['_emitBeat'] as (sample: number, flux: number) => void).call(processor, 24_000, 0); + expect(processor['_beatEvidence']).toMatchObject({ + onsetId: 0, + rawFlux: null, + onsetSample: null, + kind: 'freeRun', + phaseResidualSamples: null, + }); + }, + }); + const beats = messages.filter(message => message.type === 'beat'); + expect(beats[0]).toMatchObject({ energy: 0, onsetEnergy: 7, onsetAudioTime: matchedSample / 48_000 }); + expect(beats[1]).toMatchObject({ onsetEnergy: null, onsetAudioTime: null }); + }); + + it('preserves bootstrap identity and sample precision across ring wrap', () => { + runDetector(new Float32Array(0), { + prepare: processor => { + const detect = processor['_detectOnset'] as (flux: number) => void; + const bootstrap = processor['_bootstrapOnset'] as (age: number) => number; + const origin = 2 ** 30; + + for (let i = 0; i < 20; i++) { + for (let j = 0; j < 8; j++) { + processor['_sampleCount'] = origin + i * 10_000 + j * 64; + detect.call(processor, 0); + } + + processor['_sampleCount'] = origin + i * 10_000 + 512; + detect.call(processor, i + 1); + } + + const sample = bootstrap.call(processor, 1000); + const evidence = processor['_matchedOnset'] as { sample: number; rawFlux: number; id: number }; + expect(evidence).toMatchObject({ sample, rawFlux: 20, id: 20 }); + expect(sample).toBeGreaterThan(origin + 190_000); + expect(sample).not.toBe(Math.fround(sample)); + }, + }); + }); +}); diff --git a/packages/exojs-audio-fx/test/worklets/beat-detector-feature-cost.test.ts b/packages/exojs-audio-fx/test/worklets/beat-detector-feature-cost.test.ts new file mode 100644 index 000000000..daed2bdaf --- /dev/null +++ b/packages/exojs-audio-fx/test/worklets/beat-detector-feature-cost.test.ts @@ -0,0 +1,36 @@ +import { clicktrack } from '../fixtures/beat-fixtures'; +import { runDetector } from '../harness/beat-sandbox'; + +describe('meter feature demand', () => { + it('avoids new meter observations and fitting for manual and fixed counting', () => { + const audio = clicktrack(120, 6).samples; + const automatic = runDetector(audio) + .messages.filter(message => message.type === 'beat') + .map(beat => beat.audioTime); + + for (const processorOptions of [{ enableTimeSignatureDetection: false }, { manualMeter: { cyclePulses: 3, phasePulses: 0 } }]) { + let observe: ReturnType; + let update: ReturnType; + const { messages } = runDetector(audio, { + processorOptions, + prepare: processor => { + observe = vi.spyOn(processor['_onsets'] as { observePower: () => void }, 'observePower'); + update = vi.spyOn(processor['_meter'] as { update: () => void }, 'update'); + }, + }); + expect(observe!).not.toHaveBeenCalled(); + expect(update!).not.toHaveBeenCalled(); + expect(messages.filter(message => message.type === 'beat').map(beat => beat.audioTime)).toEqual(automatic); + } + }); + + it.each([44_100, 48_000])('keeps regular state transport at or below 20 Hz at %i Hz', sampleRate => { + const { messages } = runDetector(new Float32Array(2 * sampleRate), { sampleRate }); + const times = messages.filter(message => message.type === 'state').map(state => state.analysisTime); + expect(times.length).toBeGreaterThan(20); + + for (let i = 1; i < times.length; i++) { + expect(times[i]! - times[i - 1]!).toBeGreaterThanOrEqual(0.05 - 1e-9); + } + }); +}); diff --git a/packages/exojs-audio-fx/test/worklets/beat-detector-lifecycle.test.ts b/packages/exojs-audio-fx/test/worklets/beat-detector-lifecycle.test.ts new file mode 100644 index 000000000..3f2ab92b3 --- /dev/null +++ b/packages/exojs-audio-fx/test/worklets/beat-detector-lifecycle.test.ts @@ -0,0 +1,106 @@ +import { clicktrack, SAMPLE_RATE } from '../fixtures/beat-fixtures'; +import { runDetector } from '../harness/beat-sandbox'; + +interface ProcessorPort { + onmessage: (event: { data: Record }) => void; +} + +describe('beat processor history lifetime', () => { + it('tags a drift-driven tempo notice with the current epoch', () => { + const { messages } = runDetector(new Float32Array(0), { + processorOptions: { epoch: 7 }, + prepare: value => { + value['_bestBpm'] = 120; + const drift = value['_trackDrift'] as (candidates: Array<{ bpm: number; score: number; lag: number }>) => void; + + for (let i = 0; i < 12; i++) { + drift.call(value, [{ bpm: 128, score: 0.5, lag: 44 }]); + } + }, + }); + expect(messages.filter(message => message.type === 'tempoChange')).toEqual([ + expect.objectContaining({ type: 'tempoChange', epoch: 7, oldTempo: 120, newTempo: 128 }), + ]); + }); + + it('does not consume audio after an explicit stop and restarts on a fresh context mapping', () => { + runDetector(new Float32Array(0), { + prepare: value => { + const port = value['port'] as ProcessorPort; + const process = value['process'] as ( + inputs: Float32Array[][], + outputs: Float32Array[][], + params: Record, + ) => boolean; + const globals = globalThis as Record; + port.onmessage({ data: { type: 'reset', epoch: 1, stopped: true } }); + globals['currentFrame'] = 20 * SAMPLE_RATE; + process.call(value, [[new Float32Array(128)]], [], {}); + expect(value['_sampleCount']).toBe(0); + port.onmessage({ data: { type: 'reset', epoch: 0, stopped: false } }); + process.call(value, [[new Float32Array(128)]], [], {}); + expect(value['_sampleCount']).toBe(0); + port.onmessage({ data: { type: 'reset', epoch: 2, stopped: false } }); + process.call(value, [[new Float32Array(128)]], [], {}); + expect(value['_sampleCount']).toBe(128); + expect((value['_contextTimeAt'] as (sample: number) => number).call(value, 0)).toBe(20); + }, + }); + }); + + it('resets complete analysis history but preserves the lifetime bar count', () => { + let processor: Record; + const { messages } = runDetector(clicktrack(120, 6).samples, { + prepare: value => { + processor = value; + }, + }); + const value = processor!; + expect(value['_bestBpm']).toBeGreaterThan(0); + const bars = value['_barNumber']; + const revision = value['_lookaheadRevision']; + (value['port'] as ProcessorPort).onmessage({ data: { type: 'reset', epoch: 1 } }); + + expect(value['_bestBpm']).toBe(0); + expect(value['_locked']).toBe(false); + expect(value['_sampleCount']).toBe(0); + expect((value['_onsets'] as { count: number }).count).toBe(0); + expect(value['_fluxCount']).toBe(0); + expect(value['_fullWindowAttempted']).toBe(false); + expect(value['_lookahead']).toEqual([]); + expect(value['_barNumber']).toBe(bars); + expect(value['_lookaheadRevision']).toBe(revision); + expect(messages.at(-1)).toMatchObject({ type: 'reset', epoch: 1 }); + (value['port'] as ProcessorPort).onmessage({ data: { type: 'destroy' } }); + expect(value['_destroyed']).toBe(true); + }); + + it('invalidates on disconnected input and reacquires on the context clock', () => { + runDetector(new Float32Array(0), { + prepare: value => { + const process = value['process'] as ( + inputs: Float32Array[][], + outputs: Float32Array[][], + params: Record, + ) => boolean; + const samples = clicktrack(120, 4).samples; + const globals = globalThis as Record; + + for (let offset = 0; offset < samples.length; offset += 128) { + globals['currentFrame'] = offset; + process.call(value, [[samples.subarray(offset, offset + 128)]], [], {}); + } + + expect(value['_bestBpm']).toBeGreaterThan(0); + process.call(value, [], [], {}); + expect(value['_bestBpm']).toBe(0); + expect(value['_lookahead']).toEqual([]); + + globals['currentFrame'] = 20 * SAMPLE_RATE; + process.call(value, [[new Float32Array(512)]], [], {}); + const at = value['_contextTimeAt'] as (sample: number) => number; + expect(at.call(value, 0)).toBe(20); + }, + }); + }); +}); diff --git a/packages/exojs-audio-fx/test/worklets/beat-detector-lookahead.test.ts b/packages/exojs-audio-fx/test/worklets/beat-detector-lookahead.test.ts index 2ae18c5f6..bd73970cc 100644 --- a/packages/exojs-audio-fx/test/worklets/beat-detector-lookahead.test.ts +++ b/packages/exojs-audio-fx/test/worklets/beat-detector-lookahead.test.ts @@ -53,14 +53,17 @@ const statesBetweenBeats = (messages: readonly WorkletMessage[]): Array<{ state: return pairs; }; -// Meter detection does not switch the synthetic fixtures to 3/4 on its own, so -// the 3/4 run pins the processor's meter with detection off. +// A fixed meter isolates scheduling from acquisition and later phase revisions. const pinTripleMeter = (processor: Record): void => { processor['_activeTs'] = '3/4'; }; describe.each([ - { meter: '4/4', barLength: 4, run: () => runDetector(accentedClicktrack(120, 4, 20)) }, + { + meter: '4/4', + barLength: 4, + run: () => runDetector(accentedClicktrack(120, 4, 20), { processorOptions: { enableTimeSignatureDetection: false } }), + }, { meter: '3/4', barLength: 3, diff --git a/packages/exojs-audio-fx/test/worklets/beat-detector-manual-meter.test.ts b/packages/exojs-audio-fx/test/worklets/beat-detector-manual-meter.test.ts new file mode 100644 index 000000000..137bb31b5 --- /dev/null +++ b/packages/exojs-audio-fx/test/worklets/beat-detector-manual-meter.test.ts @@ -0,0 +1,58 @@ +import type { MeterEstimate } from '../../src/meterTypes'; +import { runDetector } from '../harness/beat-sandbox'; + +describe('manual worklet schedules', () => { + it.each([null, 0, 10])('keeps manual phase %s separate from automatic evidence and the eight-beat horizon', phase => { + runDetector(new Float32Array(0), { + processorOptions: { manualMeter: { cyclePulses: 11, phasePulses: phase, denominator: 8 } }, + prepare: processor => { + const meter = processor['_meter'] as { addBeat: () => void; update: () => void }; + const fit = vi.spyOn(meter, 'update'); + const add = vi.spyOn(meter, 'addBeat'); + processor['_bestBpm'] = 120; + processor['_ibiSamples'] = 24_000; + processor['_locked'] = true; + const emit = processor['_emitBeat'] as (sample: number, flux: number) => void; + const state = processor['_sendStateMessage'] as () => void; + const port = processor['port'] as { postMessage: (message: unknown) => void }; + const post = vi.spyOn(port, 'postMessage'); + + for (let index = 0; index < 24; index++) { + const sample = 48_000 + index * 24_000; + processor['_sampleCount'] = sample; + processor['_lastOnsetSample'] = sample; + emit.call(processor, sample, 0); + state.call(processor); + const snapshot = post.mock.calls.at(-1)![0] as { + meterEstimate: MeterEstimate; + lookahead: Array<{ isDownbeat: boolean }>; + nextDownbeatTime: number; + }; + expect(snapshot.meterEstimate.origin).toBe('manual'); + expect(snapshot.meterEstimate.selected).toMatchObject({ + cyclePulses: 11, + denominator: 8, + phasePulses: phase === null ? null : (phase + index) % 11, + }); + const expectedDistance = phase === null ? Infinity : 11 - ((phase + index) % 11); + expect(snapshot.lookahead.filter(beat => beat.isDownbeat)).toHaveLength(expectedDistance <= 8 ? 1 : 0); + + if (expectedDistance > 8) { + expect(snapshot.nextDownbeatTime).toBe(0); + } + } + + expect(fit).not.toHaveBeenCalled(); + expect(add).not.toHaveBeenCalled(); + const receive = (port as unknown as { onmessage: (event: { data: unknown }) => void }).onmessage; + receive({ data: { type: 'reset', epoch: 1 } }); + expect(processor['_conventionPhase']).toBe(phase ?? 0); + expect(processor['_manualMeter']).toMatchObject({ cyclePulses: 11, phasePulses: phase }); + receive({ data: { type: 'reset', epoch: 2, manualMeter: { cyclePulses: 3, phasePulses: 1 } } }); + expect(processor['_conventionPhase']).toBe(1); + receive({ data: { type: 'reset', epoch: 1, manualMeter: null } }); + expect(processor['_manualMeter']).toMatchObject({ cyclePulses: 3, phasePulses: 1 }); + }, + }); + }); +}); diff --git a/packages/exojs-audio-fx/test/worklets/beat-detector-meter.test.ts b/packages/exojs-audio-fx/test/worklets/beat-detector-meter.test.ts new file mode 100644 index 000000000..4155241d8 --- /dev/null +++ b/packages/exojs-audio-fx/test/worklets/beat-detector-meter.test.ts @@ -0,0 +1,144 @@ +import type { MeterEstimate } from '../../src/meterTypes'; +import { type BarStartMessage, type BeatMessage, runDetector, type StateMessage } from '../harness/beat-sandbox'; + +const accents = (bpm: number, cycle: number, offset = 0.173, seconds = 24, gain = 1, carrier = 1400): Float32Array => { + const pcm = new Float32Array(seconds * 48_000); + + for (let beat = 0; offset + (beat * 60) / bpm < seconds; beat++) { + const start = Math.round((offset + (beat * 60) / bpm) * 48_000); + + for (let j = 0; j < 960 && start + j < pcm.length; j++) { + pcm[start + j] += gain * (beat % cycle === 0 ? 0.9 : 0.3) * Math.exp(-j / 180) * Math.cos((j * 2 * Math.PI * carrier) / 48_000); + } + } + + return pcm; +}; + +interface MeterState extends StateMessage { + meterEstimate: MeterEstimate; +} + +describe('automatic meter from full audio', () => { + it.each([50, 59, 60, 61].flatMap(bpm => [3, 4].map(cycle => ({ bpm, cycle }))))( + 'emits known downbeats and bar starts at slow $bpm BPM in a $cycle-pulse cycle', + ({ bpm, cycle }) => { + const { messages } = runDetector(accents(bpm, cycle, 0.173, 40), { + processorOptions: { minBpm: bpm * 0.9, maxBpm: bpm * 1.3 }, + }); + const beats = messages.filter((message): message is BeatMessage => message.type === 'beat' && message.audioTime >= 20); + expect(beats.length).toBeGreaterThan(10); + expect(beats.every(beat => beat.status === 'locked' && beat.meterKnown)).toBe(true); + const downbeats = beats.filter(beat => beat.isDownbeat); + const reference = Array.from({ length: Math.ceil((40 * bpm) / (60 * cycle)) }, (_, i) => 0.173 + (i * 60 * cycle) / bpm).filter( + time => time >= 20 && time < 39, + ); + expect(downbeats.filter(beat => beat.audioTime < 39)).toHaveLength(reference.length); + + for (const time of reference) { + expect(downbeats.some(beat => Math.abs(beat.audioTime - time) < 0.07)).toBe(true); + } + + const bars = messages.filter((message): message is BarStartMessage => message.type === 'barStart' && message.audioTime >= 20); + expect(bars.map(bar => bar.audioTime)).toEqual(downbeats.map(beat => beat.audioTime)); + }, + ); + + it.each([3, 4])('bridges a missing slow pulse in a %i-pulse cycle and expires in silence', cycle => { + const bpm = 50; + const pcm = accents(bpm, cycle, 0.173, 40); + const missing = Math.round((0.173 + 17 * 1.2) * 48_000); + pcm.fill(0, missing, missing + 960); + pcm.fill(0, 32 * 48_000); + const { messages } = runDetector(pcm, { processorOptions: { minBpm: 45, maxBpm: 65 } }); + const reference = [18, 19, 20, 21, 22, 23, 24, 25, 26].filter(index => index % cycle === 0).map(index => 0.173 + index * 1.2); + const downbeats = messages.filter( + (message): message is BeatMessage => + message.type === 'beat' && message.isDownbeat && message.audioTime > 21 && message.audioTime < 32, + ); + expect(downbeats).toHaveLength(reference.length); + + for (const time of reference) { + expect(downbeats.some(beat => Math.abs(beat.audioTime - time) < 0.07 && beat.meterKnown)).toBe(true); + } + + const silent = messages.filter((message): message is MeterState => message.type === 'state' && message.analysisTime > 36); + expect(silent.length).toBeGreaterThan(0); + expect( + silent.every(state => state.meterEstimate.selected === null && state.nextDownbeatTime === 0 && state.lookahead.length === 0), + ).toBe(true); + expect(messages.some(message => message.type === 'barStart' && message.audioTime > 36)).toBe(false); + }); + + it('counts every fourth beat under the explicit fixed convention', () => { + const { messages } = runDetector(accents(120, 1), { processorOptions: { enableTimeSignatureDetection: false } }); + const beats = messages.filter((message): message is BeatMessage => message.type === 'beat'); + expect(beats.length).toBeGreaterThan(20); + + for (let index = 1; index < beats.length; index++) { + expect(beats[index]!.beatInBar).toBe((beats[index - 1]!.beatInBar % 4) + 1); + expect(beats[index]!.isDownbeat).toBe(beats[index]!.beatInBar === 1); + } + }); + + it.each([90, 120, 180].flatMap(bpm => [3, 4].map(cycle => ({ bpm, cycle }))))( + 'finds $cycle pulses at $bpm BPM without a prepared meter', + ({ bpm, cycle }) => { + const { messages } = runDetector(accents(bpm, cycle)); + const states = messages.filter((message): message is MeterState => message.type === 'state'); + expect(states.at(-1)!.meterEstimate).toMatchObject({ selected: { cyclePulses: cycle }, origin: 'estimated' }); + const downbeats = messages.filter( + (message): message is BeatMessage => message.type === 'beat' && message.isDownbeat && message.audioTime > 10, + ); + const reference = Array.from({ length: Math.ceil((24 * bpm) / (60 * cycle)) }, (_, i) => 0.173 + (i * 60 * cycle) / bpm).filter( + time => time > 10 && time < 23.8, + ); + expect(downbeats.filter(beat => beat.audioTime < 23.8)).toHaveLength(reference.length); + + for (const time of reference) { + expect(downbeats.some(beat => Math.abs(beat.audioTime - time) < 0.07)).toBe(true); + } + + for (const beat of downbeats) { + const nearestBar = Math.round(((beat.audioTime - 0.173) * bpm) / (60 * cycle)); + expect(Math.abs(beat.audioTime - (0.173 + (nearestBar * 60 * cycle) / bpm))).toBeLessThan(0.07); + } + }, + ); + + it('keeps equal-accent meter unknown and emits beats without invented downbeats', () => { + const { messages } = runDetector(accents(120, 1)); + const beats = messages.filter((message): message is BeatMessage => message.type === 'beat'); + expect(beats.length).toBeGreaterThan(20); + expect(beats.every(beat => !beat.isDownbeat)).toBe(true); + const states = messages.filter((message): message is MeterState => message.type === 'state'); + expect(states.at(-1)!.meterEstimate.selected).toBeNull(); + expect(states.at(-1)!.nextDownbeatTime).toBe(0); + expect(messages.some(message => message.type === 'barStart')).toBe(false); + }); + + it.each([3, 4])('recognizes a quiet %i-pulse control with another attack spectrum', cycle => { + const { messages } = runDetector(accents(120, cycle, 0, 24, 0.02, 440)); + const states = messages.filter((message): message is MeterState => message.type === 'state'); + expect(states.at(-1)!.meterEstimate).toMatchObject({ selected: { cyclePulses: cycle } }); + }); + + it.each([ + [4, 3], + [3, 4], + ])('reacquires after an actual %i-to-%i audio transition', (before, afterCycle) => { + const pcm = new Float32Array(60 * 48_000); + pcm.set(accents(120, before, 0.173, 24)); + pcm.set(accents(120, afterCycle, 0.173, 36), 24 * 48_000); + const { messages } = runDetector(pcm); + const states = messages.filter((message): message is MeterState => message.type === 'state'); + expect( + states + .filter(state => state.analysisTime > 18 && state.analysisTime < 22) + .every(state => state.meterEstimate.selected?.cyclePulses === before), + ).toBe(true); + const after = states.filter(state => state.analysisTime > 24 + 32 * 0.5 + 0.2); + expect(after.length).toBeGreaterThan(0); + expect(after.every(state => state.meterEstimate.selected?.cyclePulses === afterCycle)).toBe(true); + }); +}); diff --git a/packages/exojs-audio-fx/test/worklets/beat-detector-schedule-validity.test.ts b/packages/exojs-audio-fx/test/worklets/beat-detector-schedule-validity.test.ts new file mode 100644 index 000000000..c78ca6d8c --- /dev/null +++ b/packages/exojs-audio-fx/test/worklets/beat-detector-schedule-validity.test.ts @@ -0,0 +1,61 @@ +import { clicktrack, SAMPLE_RATE } from '../fixtures/beat-fixtures'; +import { runDetector } from '../harness/beat-sandbox'; + +describe('beat processor prediction validity', () => { + it('timestamps evidence and trust at forecast generation rather than state delivery', () => { + const { messages } = runDetector(new Float32Array(0), { + processorOptions: { enableTimeSignatureDetection: false }, + prepare: value => { + value['_bestBpm'] = 120; + value['_ibiSamples'] = 24_010; + value['_lastBeatSample'] = 48_000; + value['_lastBeatContextTime'] = 1; + value['_lastOnsetSample'] = 48_000; + value['_sampleCount'] = 48_000; + (value['_updateLookahead'] as (time: number) => void).call(value, 1); + value['_sampleCount'] = 72_000; + value['_locked'] = true; + (value['_sendStateMessage'] as () => void).call(value); + }, + }); + expect(messages[0]).toMatchObject({ + analysisTime: 1.5, + predictionGeneratedAtAudioTime: 1, + predictionEvidenceThroughAudioTime: 1, + predictionPulsePeriodSeconds: 0.5, + predictionStatus: 'provisional', + lookaheadRevision: 1, + }); + }); + + it('expires predictions and current confidence during a long recorded silence', () => { + const audio = new Float32Array(12 * SAMPLE_RATE); + audio.set(clicktrack(120, 6).samples); + const { messages } = runDetector(audio); + const before = messages.filter(message => message.type === 'state' && message.analysisTime > 4 && message.analysisTime < 5); + expect(before.some(message => message.type === 'state' && message.lookahead.length > 0)).toBe(true); + const after = messages.filter(message => message.type === 'state' && message.analysisTime > 9); + expect(after.length).toBeGreaterThan(0); + + for (const message of after) { + expect(message).toMatchObject({ lookahead: [], nextBeatTime: 0, nextDownbeatTime: 0, confidence: 0 }); + } + }); + + it('does not invent a downbeat when a schedule has no downbeat entry', () => { + const { messages } = runDetector(new Float32Array(0), { + prepare: value => { + value['_locked'] = true; + value['_bestBpm'] = 120; + value['_lastBeatSample'] = 48_000; + value['_lastBeatContextTime'] = 1; + value['_lastOnsetSample'] = 48_000; + value['_sampleCount'] = 48_000; + value['_predictionValidUntilSample'] = 96_000; + value['_lookahead'] = [{ audioTime: 1.5, tempo: 120, beatInBar: 2, isDownbeat: false }]; + (value['_sendStateMessage'] as () => void).call(value); + }, + }); + expect(messages[0]).toMatchObject({ nextBeatTime: 1.5, nextDownbeatTime: 0 }); + }); +}); diff --git a/packages/exojs-audio-fx/test/worklets/meter-evidence.test.ts b/packages/exojs-audio-fx/test/worklets/meter-evidence.test.ts new file mode 100644 index 000000000..b6cabdfb2 --- /dev/null +++ b/packages/exojs-audio-fx/test/worklets/meter-evidence.test.ts @@ -0,0 +1,233 @@ +import { MeterEvidence } from '../../src/worklets/MeterEvidence'; +import { OnsetEvidenceRing } from '../../src/worklets/OnsetEvidenceRing'; + +describe('finalized onset accent', () => { + it('uses complete attack power without changing raw crossing novelty', () => { + const ring = new OnsetEvidenceRing(); + ring.push(100, 0.2, 8); + ring.observePower(100, 0.01, 2048); + ring.observePower(612, 0.09, 2048); + expect(ring.accent(1, 1000, 2048)).toBeNull(); + ring.observePower(2200, 0, 2048); + expect(ring.accent(1, 2200, 2048)).toBe(0.09); + ring.nearest(100, 1); + expect(ring.selected).toMatchObject({ rawFlux: 0.2, strength: 8 }); + ring.reset(); + expect(ring.accent(1, 3000, 2048)).toBeNull(); + }); +}); + +describe('compact meter evidence', () => { + const period = 24_000; + const window = 2048; + + const feed = (meter: MeterEvidence, ring: OnsetEvidenceRing, index: number, power: number | null): void => { + const sample = index * period; + let id = 0; + + if (power !== null) { + ring.push(sample, 1, 8); + ring.nearest(sample, 1); + id = ring.selected.id; + ring.observePower(sample + 512, power, window); + } + + meter.addBeat(sample, period, id); + meter.update(sample + window, ring, window); + }; + + it.each([3, 4] as const)('recovers every starting phase of a %i-pulse cycle without a gain bias', cycle => { + for (let offset = 0; offset < cycle; offset++) { + for (const gain of [0.0001, 1, 100]) { + const meter = new MeterEvidence(); + const ring = new OnsetEvidenceRing(); + let firstCorrect = -1; + let stableRun = 0; + + for (let i = 0; i < 48; i++) { + feed(meter, ring, i, gain * ((i + offset) % cycle === 0 ? 9 : 1)); + const correct = meter.cyclePulses === cycle && meter.phasePulses === (i + offset) % cycle; + + if (correct && firstCorrect < 0) { + firstCorrect = i; + } + + stableRun = correct ? stableRun + 1 : 0; + } + + expect(meter.cyclePulses).toBe(cycle); + expect(meter.phasePulses).toBe((47 + offset) % cycle); + expect(firstCorrect).toBeGreaterThanOrEqual(0); + expect(firstCorrect).toBeLessThan(4 * cycle); + expect(stableRun).toBeGreaterThanOrEqual(2 * cycle); + } + } + }); + + it('abstains on equal accents, missing observations and a two-pulse alias', () => { + for (const pattern of [[1], [9, 1]]) { + const meter = new MeterEvidence(); + const ring = new OnsetEvidenceRing(); + + for (let i = 0; i < 64; i++) { + feed(meter, ring, i, i % 11 === 0 ? null : pattern[i % pattern.length]!); + } + + expect(meter.cyclePulses).toBe(0); + } + }); + + it('does not infer phase from an unfinished attack and expires without evidence', () => { + const meter = new MeterEvidence(); + const ring = new OnsetEvidenceRing(); + + for (let i = 0; i < 24; i++) { + feed(meter, ring, i, i % 3 === 0 ? 9 : 1); + } + + expect(meter.cyclePulses).toBe(3); + meter.update(30 * period, ring, window); + expect(meter.cyclePulses).toBe(0); + expect(meter.validUntilSample).toBe(0); + meter.reset(); + ring.reset(); + ring.push(40 * period, 1, 8); + ring.nearest(40 * period, 1); + meter.addBeat(40 * period, period, ring.selected.id); + meter.update(40 * period + 512, ring, window); + expect(meter.cyclePulses).toBe(0); + }); + + it('reacquires after a four-to-three switch without retaining a stale phase', () => { + const meter = new MeterEvidence(); + const ring = new OnsetEvidenceRing(); + + for (let i = 0; i < 48; i++) { + feed(meter, ring, i, i % 4 === 0 ? 9 : 1); + } + + expect(meter.cyclePulses).toBe(4); + + for (let i = 48; i < 96; i++) { + feed(meter, ring, i, i % 3 === 0 ? 9 : 1); + } + + expect(meter.cyclePulses).toBe(3); + expect(meter.phasePulses).toBe(2); + }); + + it('retains cycle alignment through missing onsets and skipped grid pulses', () => { + const meter = new MeterEvidence(); + const ring = new OnsetEvidenceRing(); + + for (let i = 0; i < 64; i++) { + if (i % 13 !== 7) { + feed(meter, ring, i, i % 11 === 5 ? null : i % 4 === 0 ? 9 : 1); + } + } + + expect(meter.cyclePulses).toBe(4); + expect(meter.phasePulses).toBe(3); + }); + + it.each([3, 4])('handles structured missing positions without invalid scores for a %i-pulse input', cycle => { + const meter = new MeterEvidence(); + const ring = new OnsetEvidenceRing(); + + for (let i = 0; i < 64; i++) { + feed(meter, ring, i, i % 2 === 0 ? (i % cycle === 0 ? 9 : 1) : null); + } + + expect([...meter.candidateScores].every(Number.isFinite)).toBe(true); + expect(meter.candidateScores[1]).toBe(0); + // All three positions have support; the four-position model has two unseen positions. + expect(meter.cyclePulses).toBe(cycle === 3 ? 3 : 0); + + if (cycle === 3) { + expect(meter.phasePulses).toBe(0); + } + }); + + it('keeps slow-cycle phase through the next pulse and expires after the finalization deadline', () => { + const meter = new MeterEvidence(); + const ring = new OnsetEvidenceRing(); + const slowPeriod = 57_600; + + for (let i = 0; i < 24; i++) { + const sample = i * slowPeriod; + meter.update(sample - 512, ring, window); + + if (i > 15) { + expect(meter.cyclePulses).toBe(3); + } + + ring.push(sample, 1, 8); + ring.nearest(sample, 1); + ring.observePower(sample + 512, i % 3 === 0 ? 9 : 1, window); + meter.addBeat(sample, slowPeriod, ring.selected.id); + meter.update(sample + window, ring, window); + } + + expect(meter.cyclePulses).toBe(3); + expect(meter.phasePulses).toBe(2); + meter.update(25 * slowPeriod + window - 1, ring, window); + expect(meter.cyclePulses).toBe(3); + meter.update(25 * slowPeriod + window, ring, window); + expect(meter.cyclePulses).toBe(0); + }); + + it('does not turn random accent noise into a confident cycle', () => { + const meter = new MeterEvidence(); + const ring = new OnsetEvidenceRing(); + let seed = 0x7351; + let known = 0; + + for (let i = 0; i < 200; i++) { + seed = (Math.imul(seed, 1664525) + 1013904223) >>> 0; + feed(meter, ring, i, 0.5 + seed / 2 ** 32); + known += meter.cyclePulses > 0 ? 1 : 0; + } + + expect(known / 200).toBeLessThanOrEqual(0.05); + }); + + it('recognizes a repeating cycle without inventing an anchor between equally strong positions', () => { + const meter = new MeterEvidence(); + const ring = new OnsetEvidenceRing(); + + for (let i = 0; i < 48; i++) { + feed(meter, ring, i, i % 3 === 2 ? 1 : 9); + } + + expect(meter.cyclePulses).toBe(3); + expect(meter.phasePulses).toBeNull(); + expect(meter.reason).toBe('ambiguousPhase'); + expect(meter.candidateScores[0]).toBeGreaterThan(0.5); + }); + + it('uses aligned band attacks when total power is uniform and never substitutes a later onset', () => { + const meter = new MeterEvidence(); + const ring = new OnsetEvidenceRing(); + const bands = new Float64Array(3); + const observed = new Float64Array(4); + + for (let i = 0; i < 48; i++) { + const sample = i * period; + bands.set(i % 4 === 0 ? [9, 1, 1] : [1, 1, 1]); + ring.push(sample, 1, 8); + ring.nearest(sample, 1); + const id = ring.selected.id; + ring.observePower(sample + 512, 1, window, bands); + expect(ring.accentFeatures(id, sample + 512, window, observed)).toBe(false); + meter.addBeat(sample, period, id); + meter.update(sample + window, ring, window); + expect(ring.accentFeatures(id, sample + window, window, observed)).toBe(true); + expect([...observed]).toEqual([1, ...bands]); + } + + expect(meter.cyclePulses).toBe(4); + expect(meter.phasePulses).toBe(3); + ring.reset(); + expect(ring.accentFeatures(48, 50 * period, window, observed)).toBe(false); + }); +}); diff --git a/site/src/content/api/beat-analysis-discontinuity.json b/site/src/content/api/beat-analysis-discontinuity.json new file mode 100644 index 000000000..c96fa7900 --- /dev/null +++ b/site/src/content/api/beat-analysis-discontinuity.json @@ -0,0 +1,117 @@ +{ + "title": "BeatAnalysisDiscontinuity", + "description": "", + "symbol": "BeatAnalysisDiscontinuity", + "kind": "type", + "subsystem": "audio", + "importPath": "@codexo/exojs-audio-fx", + "tier": "stable", + "memberCount": 0, + "counts": { + "constructors": 0, + "methods": 0, + "properties": 0, + "events": 0 + }, + "sections": [ + { + "id": "import", + "title": "Import", + "members": [], + "paragraphs": [], + "importLine": "import { BeatAnalysisDiscontinuity } from '@codexo/exojs-audio-fx'", + "sourceLink": null + }, + { + "id": "definition", + "title": "Definition", + "members": [ + { + "name": "BeatAnalysisDiscontinuity", + "signature": "\"eof\" | \"explicitPause\" | \"loop\" | \"reset\" | \"resume\" | \"seek\" | \"sourceChange\" | \"starvation\"", + "signatureTokens": [ + { + "text": "\"eof\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"explicitPause\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"loop\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"reset\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"resume\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"seek\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"sourceChange\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"starvation\"", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "" + } + ], + "paragraphs": [], + "importLine": null, + "sourceLink": null + }, + { + "id": "source", + "title": "Source", + "members": [], + "paragraphs": [], + "importLine": null, + "sourceLink": { + "label": "packages/exojs-audio-fx/src/BeatAnalysisDiscontinuity.ts", + "href": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/BeatAnalysisDiscontinuity.ts" + } + } + ], + "sourcePath": "packages/exojs-audio-fx/src/BeatAnalysisDiscontinuity.ts", + "sourceUrl": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/BeatAnalysisDiscontinuity.ts" +} diff --git a/site/src/content/api/beat-detector-options.json b/site/src/content/api/beat-detector-options.json index 84a2b7573..8c8732783 100644 --- a/site/src/content/api/beat-detector-options.json +++ b/site/src/content/api/beat-detector-options.json @@ -6,11 +6,11 @@ "subsystem": "audio", "importPath": "@codexo/exojs-audio-fx", "tier": "stable", - "memberCount": 15, + "memberCount": 16, "counts": { "constructors": 0, "methods": 0, - "properties": 15, + "properties": 16, "events": 0 }, "sections": [ @@ -124,7 +124,7 @@ ], "params": [], "returnType": null, - "description": "When true (default), the worklet runs parallel 3/4 and 4/4 posteriors and switches active time signature via hysteresis. Set false to lock to 4/4." + "description": "When true (default), estimates three- or four-pulse accent cycles and leaves ambiguous meter unknown. Set false for a fixed four-beat counting convention." }, { "name": "fastTempoWindowSec", @@ -226,6 +226,39 @@ "returnType": null, "description": "Time window in seconds for BeatDetector.justBeat. Default 0.03 - true for the visual frame(s) within 30ms of a beat onset." }, + { + "name": "manualMeter", + "signature": "manualMeter?: ManualMeter | null", + "signatureTokens": [ + { + "text": "manualMeter", + "kind": "name" + }, + { + "text": "?", + "kind": "punctuation" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "ManualMeter", + "kind": "type" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "null", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Overrides automatic meter with explicit counting context; changing it resets analysis." + }, { "name": "maxBpm", "signature": "maxBpm?: number", diff --git a/site/src/content/api/beat-detector.json b/site/src/content/api/beat-detector.json index 989008b98..4f004d1db 100644 --- a/site/src/content/api/beat-detector.json +++ b/site/src/content/api/beat-detector.json @@ -1,16 +1,16 @@ { "title": "BeatDetector", - "description": "Real-time tempo + beat tracker. Splits work between the audio-rendering thread (an AudioWorklet that runs onset detection, tempogram analysis, and parallel 3/4 and 4/4 posterior estimation) and the main thread (this class - receives beats, fires Signals, handles configuration and source routing). Accepts a wide range of BeatDetectorSources - a bus, an individual Voice, a raw MediaStream, or any AudioNode - and exposes a Signal for each notable event: - BeatDetector.onBeat - every detected beat - BeatDetector.onDownbeat - first beat of each bar - BeatDetector.onBarStart - bar boundary - BeatDetector.onTempoChange - when the tracked BPM changes - BeatDetector.onBeatPredicted - look-ahead schedule notice Early beats arrive with low latency tagged `status: 'provisional'` (after a short `minSettlingMs` warm-up, default 400 ms) for snappy visual reactivity, then promote to `status: 'locked'` once the tempo grid is trustworthy - a single BeatDetector.onBeat signal carries both via BeatInfo.status. Set `emitProvisionalBeats: false` to receive only locked beats. Time-signature detection (3/4 vs 4/4) is on by default; lock to 4/4 by setting `enableTimeSignatureDetection: false`.", + "description": "Real-time tempo + beat tracker. Splits work between the audio-rendering thread (an AudioWorklet that runs onset detection, tempogram analysis, and three-/four-pulse accent-cycle estimation) and the main thread (this class - receives beats, fires Signals, handles configuration and source routing). Accepts a wide range of BeatDetectorSources - a bus, an individual Voice, a raw MediaStream, or any AudioNode - and exposes a Signal for each notable event: - BeatDetector.onBeat - every detected beat - BeatDetector.onDownbeat - first beat of each bar - BeatDetector.onBarStart - bar boundary - BeatDetector.onTempoChange - when the tracked BPM changes - BeatDetector.onBeatPredicted - look-ahead schedule notice Early beats arrive with low latency tagged `status: 'provisional'` (after a short `minSettlingMs` warm-up, default 400 ms) for snappy visual reactivity, then promote to `status: 'locked'` once the tempo grid is trustworthy - a single BeatDetector.onBeat signal carries both via BeatInfo.status. Set `emitProvisionalBeats: false` to receive only locked beats. Time-signature detection is on by default; ambiguous meter leaves beats available without downbeats. Use a fixed four-beat convention with `enableTimeSignatureDetection: false`.", "symbol": "BeatDetector", "kind": "class", "subsystem": "audio", "importPath": "@codexo/exojs-audio-fx", "tier": "stable", - "memberCount": 34, + "memberCount": 42, "counts": { "constructors": 1, - "methods": 2, - "properties": 26, + "methods": 4, + "properties": 32, "events": 5 }, "sections": [ @@ -19,9 +19,9 @@ "title": "Import", "members": [], "paragraphs": [ - "Real-time tempo + beat tracker. Splits work between the audio-rendering thread (an AudioWorklet that runs onset detection, tempogram analysis, and parallel 3/4 and 4/4 posterior estimation) and the main thread (this class - receives beats, fires Signals, handles configuration and source routing).", + "Real-time tempo + beat tracker. Splits work between the audio-rendering thread (an AudioWorklet that runs onset detection, tempogram analysis, and three-/four-pulse accent-cycle estimation) and the main thread (this class - receives beats, fires Signals, handles configuration and source routing).", "Accepts a wide range of BeatDetectorSources - a bus, an individual Voice, a raw MediaStream, or any AudioNode - and exposes a Signal for each notable event: - BeatDetector.onBeat - every detected beat - BeatDetector.onDownbeat - first beat of each bar - BeatDetector.onBarStart - bar boundary - BeatDetector.onTempoChange - when the tracked BPM changes - BeatDetector.onBeatPredicted - look-ahead schedule notice", - "Early beats arrive with low latency tagged `status: 'provisional'` (after a short `minSettlingMs` warm-up, default 400 ms) for snappy visual reactivity, then promote to `status: 'locked'` once the tempo grid is trustworthy - a single BeatDetector.onBeat signal carries both via BeatInfo.status. Set `emitProvisionalBeats: false` to receive only locked beats. Time-signature detection (3/4 vs 4/4) is on by default; lock to 4/4 by setting `enableTimeSignatureDetection: false`." + "Early beats arrive with low latency tagged `status: 'provisional'` (after a short `minSettlingMs` warm-up, default 400 ms) for snappy visual reactivity, then promote to `status: 'locked'` once the tempo grid is trustworthy - a single BeatDetector.onBeat signal carries both via BeatInfo.status. Set `emitProvisionalBeats: false` to receive only locked beats. Time-signature detection is on by default; ambiguous meter leaves beats available without downbeats. Use a fixed four-beat convention with `enableTimeSignatureDetection: false`." ], "importLine": "import { BeatDetector } from '@codexo/exojs-audio-fx'", "sourceLink": null @@ -119,6 +119,82 @@ "returnType": "void", "description": "" }, + { + "name": "notifyDiscontinuity", + "signature": "notifyDiscontinuity(reason: BeatAnalysisDiscontinuity): void", + "signatureTokens": [ + { + "text": "notifyDiscontinuity", + "kind": "name" + }, + { + "text": "(", + "kind": "punctuation" + }, + { + "text": "reason", + "kind": "param" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "BeatAnalysisDiscontinuity", + "kind": "type" + }, + { + "text": ")", + "kind": "punctuation" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "void", + "kind": "keyword" + } + ], + "params": [ + { + "name": "reason", + "type": "BeatAnalysisDiscontinuity", + "optional": false + } + ], + "returnType": "void", + "description": "Discards current rhythm evidence immediately. EOF and explicit pause stop analysis until resume, reset or a source identity change. Loops reset phase. Manual configuration persists; the next epoch establishes a new anchor." + }, + { + "name": "reset", + "signature": "reset(): void", + "signatureTokens": [ + { + "text": "reset", + "kind": "name" + }, + { + "text": "(", + "kind": "punctuation" + }, + { + "text": ")", + "kind": "punctuation" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "void", + "kind": "keyword" + } + ], + "params": [], + "returnType": "void", + "description": "Discards rhythm history and pending predictions. Call after a seek, loop discontinuity or explicit playback interruption on the same source. Assigning a different source resets automatically." + }, { "name": "subdivisionPhase", "signature": "subdivisionPhase(division: number): number", @@ -320,7 +396,7 @@ ], "params": [], "returnType": null, - "description": "" + "description": "Display cycle length; use meterEstimate for an available estimate." }, { "name": "barPosition", @@ -341,7 +417,7 @@ ], "params": [], "returnType": null, - "description": "" + "description": "Display counting only while meterKnown is false." }, { "name": "barPulse", @@ -364,6 +440,27 @@ "returnType": null, "description": "Like pulse but resets on downbeats and decays per barPulseHalfLife. Useful for emphasizing the first beat of each bar (e.g. brighter flash on \"1\" vs \"2,3,4\")." }, + { + "name": "beatConfidence", + "signature": "beatConfidence: number", + "signatureTokens": [ + { + "text": "beatConfidence", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Pulse-grid confidence without the legacy meter factor, 0..1, or 0 without current support. Available during provisional tracking and uncalibrated; it does not change the legacy lock threshold or establish meter accuracy." + }, { "name": "beatPhase", "signature": "beatPhase: number", @@ -425,7 +522,7 @@ ], "params": [], "returnType": null, - "description": "" + "description": "Legacy alias of confidence, not an independent measure of phase stability." }, { "name": "justBeat", @@ -479,7 +576,78 @@ ], "params": [], "returnType": null, - "description": "The next eight beats, extrapolated from the last tracked beat at the current tempo and numbered from the bar position that follows it. Predictions, not analysed audio: a tempo change or break is not anticipated." + "description": "The next eight beats, extrapolated from the last tracked beat at the current tempo and numbered from the bar position that follows it. Entries may have passed since delivery. Predictions are conditional on the rhythm continuing; a tempo change or break is not anticipated. Empty after prediction expiry. Unknown or expired meter clears downbeat flags while retaining beat times." + }, + { + "name": "manualMeter", + "signature": "manualMeter: ManualMeter | null", + "signatureTokens": [ + { + "text": "manualMeter", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "ManualMeter", + "kind": "type" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "null", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "" + }, + { + "name": "meterEstimate", + "signature": "meterEstimate: MeterEstimate", + "signatureTokens": [ + { + "text": "meterEstimate", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "MeterEstimate", + "kind": "type" + } + ], + "params": [], + "returnType": null, + "description": "Immutable interpretation view, including reasons and competing models while abstaining. A null selected model means no cycle; a null selected phase means no downbeat anchor. The notated denominator is never inferred." + }, + { + "name": "meterKnown", + "signature": "meterKnown: boolean", + "signatureTokens": [ + { + "text": "meterKnown", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "boolean", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "" }, { "name": "nextBeatTime", @@ -500,7 +668,7 @@ ], "params": [], "returnType": null, - "description": "" + "description": "Next predicted future beat on the audio-context clock, or 0 without a valid prediction." }, { "name": "nextDownbeatTime", @@ -521,7 +689,7 @@ ], "params": [], "returnType": null, - "description": "" + "description": "Next predicted future bar start on the audio-context clock, or 0 when unavailable." }, { "name": "onsetStrength", @@ -563,7 +731,28 @@ ], "params": [], "returnType": null, - "description": "How well recent onsets support the beat grid's phase, 0 to 1, or 0 before the grid locks. Distinct from confidence, which reports how sure the detector is of the tempo. A metronomic loop at a wrong-by-an-octave tempo reads high confidence and high phase confidence; a rubato passage at a known tempo reads high confidence and low phase confidence. Gate anything that must land exactly on the beat - a scored hit, a quantised trigger - on this rather than on confidence." + "description": "Current onset support for the tracked phase, 0..1 and uncalibrated." + }, + { + "name": "predictionValidUntil", + "signature": "predictionValidUntil: number", + "signatureTokens": [ + { + "text": "predictionValidUntil", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Audio-context time when the current prediction schedule must be discarded. Returns 0 without a current schedule. Later extrapolated beat times remain conditional on the rhythm continuing; this is not a delivery deadline." }, { "name": "pulse", @@ -640,6 +829,35 @@ "returnType": null, "description": "" }, + { + "name": "schedule", + "signature": "schedule: BeatSchedule | null", + "signatureTokens": [ + { + "text": "schedule", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "BeatSchedule", + "kind": "type" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "null", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Current immutable conditional forecast, or null after expiry/reset. Future entries are checked at access time." + }, { "name": "secondsSinceLastBeat", "signature": "secondsSinceLastBeat: number", @@ -755,7 +973,7 @@ ], "params": [], "returnType": null, - "description": "" + "description": "Legacy display counting; the denominator is manual or defaults to the convention 4." } ], "paragraphs": [], @@ -887,7 +1105,7 @@ ], "params": [], "returnType": null, - "description": "Dispatched with the first entry of each new lookahead schedule. The detector computes a schedule once per tracked beat, so a listener hears about every upcoming beat once, not on every state update. A newer schedule supersedes the times an earlier one predicted for the same beats, and one replaced before the main thread received it is skipped. Nothing is dispatched while the detector coasts through silence." + "description": "Dispatched with the first future entry of each new lookahead schedule. Each revision is announced at most once; queued revisions are not coalesced. An expired schedule or one without a future entry produces no notice. A newer revision supersedes earlier predictions for the same beats." }, { "name": "onDownbeat", diff --git a/site/src/content/api/beat-info.json b/site/src/content/api/beat-info.json index 2ccea79fa..0d612eb17 100644 --- a/site/src/content/api/beat-info.json +++ b/site/src/content/api/beat-info.json @@ -6,11 +6,11 @@ "subsystem": "audio", "importPath": "@codexo/exojs-audio-fx", "tier": "stable", - "memberCount": 8, + "memberCount": 12, "counts": { "constructors": 0, "methods": 0, - "properties": 8, + "properties": 12, "events": 0 }, "sections": [ @@ -47,6 +47,31 @@ "returnType": null, "description": "audioContext.currentTime when the beat occurred." }, + { + "name": "beatConfidence", + "signature": "beatConfidence?: number", + "signatureTokens": [ + { + "text": "beatConfidence", + "kind": "name" + }, + { + "text": "?", + "kind": "punctuation" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Pulse confidence without the meter factor, 0..1 and uncalibrated." + }, { "name": "beatInBar", "signature": "beatInBar: number", @@ -108,7 +133,7 @@ ], "params": [], "returnType": null, - "description": "Confidence 0..1." + "description": "Legacy combined confidence, 0..1; not a calibrated probability." }, { "name": "energy", @@ -129,7 +154,7 @@ ], "params": [], "returnType": null, - "description": "Novelty/onset strength at the beat." + "description": "Raw spectral novelty of the emission hop, which may follow the matched onset." }, { "name": "isDownbeat", @@ -152,6 +177,97 @@ "returnType": null, "description": "Is this beat the first in a bar?" }, + { + "name": "meterKnown", + "signature": "meterKnown?: boolean", + "signatureTokens": [ + { + "text": "meterKnown", + "kind": "name" + }, + { + "text": "?", + "kind": "punctuation" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "boolean", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "True when the cycle and its phase are available; false leaves beatInBar as display counting only." + }, + { + "name": "onsetAudioTime", + "signature": "onsetAudioTime?: null | number", + "signatureTokens": [ + { + "text": "onsetAudioTime", + "kind": "name" + }, + { + "text": "?", + "kind": "punctuation" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "null", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Matched onset time on the audio-context clock, or null without a match." + }, + { + "name": "onsetEnergy", + "signature": "onsetEnergy?: null | number", + "signatureTokens": [ + { + "text": "onsetEnergy", + "kind": "name" + }, + { + "text": "?", + "kind": "punctuation" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "null", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Matched onset novelty, or null when this beat has no associated onset." + }, { "name": "status", "signature": "status: \"locked\" | \"provisional\"", @@ -179,7 +295,7 @@ ], "params": [], "returnType": null, - "description": "Detection trust level: - 'provisional' - an early, low-latency beat emitted before the tempo grid is fully settled. Good for visual reactivity (a \"blink\"); may be revised. - 'locked' - the tempo grid is settled and trustworthy. Safe for sync-critical use. Each stable segment yields exactly one provisional→locked transition." + "description": "Detection trust level: - 'provisional' - an early, low-latency beat emitted before the tempo grid is fully settled. Good for visual reactivity (a \"blink\"); may be revised. - 'locked' - the tempo grid passed the detector's settling heuristic. This does not establish meter accuracy or calibrated confidence." }, { "name": "tempo", diff --git a/site/src/content/api/beat-schedule.json b/site/src/content/api/beat-schedule.json new file mode 100644 index 000000000..fae5031ed --- /dev/null +++ b/site/src/content/api/beat-schedule.json @@ -0,0 +1,351 @@ +{ + "title": "BeatSchedule", + "description": "", + "symbol": "BeatSchedule", + "kind": "interface", + "subsystem": "audio", + "importPath": "@codexo/exojs-audio-fx", + "tier": "stable", + "memberCount": 11, + "counts": { + "constructors": 0, + "methods": 0, + "properties": 11, + "events": 0 + }, + "sections": [ + { + "id": "import", + "title": "Import", + "members": [], + "paragraphs": [], + "importLine": "import { BeatSchedule } from '@codexo/exojs-audio-fx'", + "sourceLink": null + }, + { + "id": "properties", + "title": "Properties", + "members": [ + { + "name": "beats", + "signature": "beats: readonly Readonly[]", + "signatureTokens": [ + { + "text": "beats", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "readonly", + "kind": "keyword" + }, + { + "text": " ", + "kind": "punctuation" + }, + { + "text": "Readonly", + "kind": "type" + }, + { + "text": "<", + "kind": "punctuation" + }, + { + "text": "UpcomingBeat", + "kind": "type" + }, + { + "text": ">", + "kind": "punctuation" + }, + { + "text": "[]", + "kind": "punctuation" + } + ], + "params": [], + "returnType": null, + "description": "Retained revision entries can include times that have since passed." + }, + { + "name": "epoch", + "signature": "epoch: number", + "signatureTokens": [ + { + "text": "epoch", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "" + }, + { + "name": "evidenceThroughAudioTime", + "signature": "evidenceThroughAudioTime: number", + "signatureTokens": [ + { + "text": "evidenceThroughAudioTime", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "End of the audio consumed by this forecast revision." + }, + { + "name": "generatedAtAudioTime", + "signature": "generatedAtAudioTime: number", + "signatureTokens": [ + { + "text": "generatedAtAudioTime", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Time this forecast revision was generated, not when it reached the caller." + }, + { + "name": "horizonAudioTime", + "signature": "horizonAudioTime: number", + "signatureTokens": [ + { + "text": "horizonAudioTime", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Last conditional target; this does not extend validity." + }, + { + "name": "nextBeat", + "signature": "nextBeat: Readonly | null", + "signatureTokens": [ + { + "text": "nextBeat", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "Readonly", + "kind": "type" + }, + { + "text": "<", + "kind": "punctuation" + }, + { + "text": "UpcomingBeat", + "kind": "type" + }, + { + "text": ">", + "kind": "punctuation" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "null", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "" + }, + { + "name": "nextDownbeat", + "signature": "nextDownbeat: Readonly | null", + "signatureTokens": [ + { + "text": "nextDownbeat", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "Readonly", + "kind": "type" + }, + { + "text": "<", + "kind": "punctuation" + }, + { + "text": "UpcomingBeat", + "kind": "type" + }, + { + "text": ">", + "kind": "punctuation" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "null", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Null without valid anchored meter or a future boundary within the horizon." + }, + { + "name": "pulsePeriodSeconds", + "signature": "pulsePeriodSeconds: number", + "signatureTokens": [ + { + "text": "pulsePeriodSeconds", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Period used to extrapolate this forecast, not a notated note value." + }, + { + "name": "revision", + "signature": "revision: number", + "signatureTokens": [ + { + "text": "revision", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "" + }, + { + "name": "status", + "signature": "status: \"locked\" | \"provisional\"", + "signatureTokens": [ + { + "text": "status", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "\"locked\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"provisional\"", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "" + }, + { + "name": "validUntilAudioTime", + "signature": "validUntilAudioTime: number", + "signatureTokens": [ + { + "text": "validUntilAudioTime", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Discard the forecast at this context time without renewed evidence." + } + ], + "paragraphs": [], + "importLine": null, + "sourceLink": null + }, + { + "id": "source", + "title": "Source", + "members": [], + "paragraphs": [], + "importLine": null, + "sourceLink": { + "label": "packages/exojs-audio-fx/src/beatScheduleTypes.ts", + "href": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/beatScheduleTypes.ts" + } + } + ], + "sourcePath": "packages/exojs-audio-fx/src/beatScheduleTypes.ts", + "sourceUrl": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/beatScheduleTypes.ts" +} diff --git a/site/src/content/api/manual-meter.json b/site/src/content/api/manual-meter.json new file mode 100644 index 000000000..5cd9b0913 --- /dev/null +++ b/site/src/content/api/manual-meter.json @@ -0,0 +1,137 @@ +{ + "title": "ManualMeter", + "description": "Manual counting context; phase anchors at the first tracked beat after reset.", + "symbol": "ManualMeter", + "kind": "interface", + "subsystem": "audio", + "importPath": "@codexo/exojs-audio-fx", + "tier": "stable", + "memberCount": 3, + "counts": { + "constructors": 0, + "methods": 0, + "properties": 3, + "events": 0 + }, + "sections": [ + { + "id": "import", + "title": "Import", + "members": [], + "paragraphs": [ + "Manual counting context; phase anchors at the first tracked beat after reset." + ], + "importLine": "import { ManualMeter } from '@codexo/exojs-audio-fx'", + "sourceLink": null + }, + { + "id": "properties", + "title": "Properties", + "members": [ + { + "name": "cyclePulses", + "signature": "cyclePulses: number", + "signatureTokens": [ + { + "text": "cyclePulses", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Integer from 1 through 32." + }, + { + "name": "denominator", + "signature": "denominator?: null | number", + "signatureTokens": [ + { + "text": "denominator", + "kind": "name" + }, + { + "text": "?", + "kind": "punctuation" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "null", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "A power of two from 1 through 128, or null when unspecified." + }, + { + "name": "phasePulses", + "signature": "phasePulses?: null | number", + "signatureTokens": [ + { + "text": "phasePulses", + "kind": "name" + }, + { + "text": "?", + "kind": "punctuation" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "null", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Null or omitted leaves phase unknown." + } + ], + "paragraphs": [], + "importLine": null, + "sourceLink": null + }, + { + "id": "source", + "title": "Source", + "members": [], + "paragraphs": [], + "importLine": null, + "sourceLink": { + "label": "packages/exojs-audio-fx/src/meterTypes.ts", + "href": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/meterTypes.ts" + } + } + ], + "sourcePath": "packages/exojs-audio-fx/src/meterTypes.ts", + "sourceUrl": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/meterTypes.ts" +} diff --git a/site/src/content/api/meter-candidate.json b/site/src/content/api/meter-candidate.json new file mode 100644 index 000000000..a05629f6a --- /dev/null +++ b/site/src/content/api/meter-candidate.json @@ -0,0 +1,90 @@ +{ + "title": "MeterCandidate", + "description": "", + "symbol": "MeterCandidate", + "kind": "interface", + "subsystem": "audio", + "importPath": "@codexo/exojs-audio-fx", + "tier": "stable", + "memberCount": 2, + "counts": { + "constructors": 0, + "methods": 0, + "properties": 2, + "events": 0 + }, + "sections": [ + { + "id": "import", + "title": "Import", + "members": [], + "paragraphs": [], + "importLine": "import { MeterCandidate } from '@codexo/exojs-audio-fx'", + "sourceLink": null + }, + { + "id": "properties", + "title": "Properties", + "members": [ + { + "name": "model", + "signature": "model: MeterModel", + "signatureTokens": [ + { + "text": "model", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "MeterModel", + "kind": "type" + } + ], + "params": [], + "returnType": null, + "description": "" + }, + { + "name": "score", + "signature": "score: number", + "signatureTokens": [ + { + "text": "score", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Uncalibrated model strength, not a probability." + } + ], + "paragraphs": [], + "importLine": null, + "sourceLink": null + }, + { + "id": "source", + "title": "Source", + "members": [], + "paragraphs": [], + "importLine": null, + "sourceLink": { + "label": "packages/exojs-audio-fx/src/meterTypes.ts", + "href": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/meterTypes.ts" + } + } + ], + "sourcePath": "packages/exojs-audio-fx/src/meterTypes.ts", + "sourceUrl": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/meterTypes.ts" +} diff --git a/site/src/content/api/meter-estimate-reason.json b/site/src/content/api/meter-estimate-reason.json new file mode 100644 index 000000000..7e79c8200 --- /dev/null +++ b/site/src/content/api/meter-estimate-reason.json @@ -0,0 +1,109 @@ +{ + "title": "MeterEstimateReason", + "description": "", + "symbol": "MeterEstimateReason", + "kind": "type", + "subsystem": "audio", + "importPath": "@codexo/exojs-audio-fx", + "tier": "stable", + "memberCount": 0, + "counts": { + "constructors": 0, + "methods": 0, + "properties": 0, + "events": 0 + }, + "sections": [ + { + "id": "import", + "title": "Import", + "members": [], + "paragraphs": [], + "importLine": "import { MeterEstimateReason } from '@codexo/exojs-audio-fx'", + "sourceLink": null + }, + { + "id": "definition", + "title": "Definition", + "members": [ + { + "name": "MeterEstimateReason", + "signature": "\"acquiring\" | \"ambiguousCycle\" | \"ambiguousPhase\" | \"available\" | \"expired\" | \"insufficientEvidence\" | \"lowContrast\"", + "signatureTokens": [ + { + "text": "\"acquiring\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"ambiguousCycle\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"ambiguousPhase\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"available\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"expired\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"insufficientEvidence\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"lowContrast\"", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "" + } + ], + "paragraphs": [], + "importLine": null, + "sourceLink": null + }, + { + "id": "source", + "title": "Source", + "members": [], + "paragraphs": [], + "importLine": null, + "sourceLink": { + "label": "packages/exojs-audio-fx/src/meterTypes.ts", + "href": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/meterTypes.ts" + } + } + ], + "sourcePath": "packages/exojs-audio-fx/src/meterTypes.ts", + "sourceUrl": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/meterTypes.ts" +} diff --git a/site/src/content/api/meter-estimate.json b/site/src/content/api/meter-estimate.json new file mode 100644 index 000000000..2bc946ab5 --- /dev/null +++ b/site/src/content/api/meter-estimate.json @@ -0,0 +1,231 @@ +{ + "title": "MeterEstimate", + "description": "", + "symbol": "MeterEstimate", + "kind": "interface", + "subsystem": "audio", + "importPath": "@codexo/exojs-audio-fx", + "tier": "stable", + "memberCount": 7, + "counts": { + "constructors": 0, + "methods": 0, + "properties": 7, + "events": 0 + }, + "sections": [ + { + "id": "import", + "title": "Import", + "members": [], + "paragraphs": [], + "importLine": "import { MeterEstimate } from '@codexo/exojs-audio-fx'", + "sourceLink": null + }, + { + "id": "properties", + "title": "Properties", + "members": [ + { + "name": "alternatives", + "signature": "alternatives: readonly MeterCandidate[]", + "signatureTokens": [ + { + "text": "alternatives", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "readonly", + "kind": "keyword" + }, + { + "text": " ", + "kind": "punctuation" + }, + { + "text": "MeterCandidate", + "kind": "type" + }, + { + "text": "[]", + "kind": "punctuation" + } + ], + "params": [], + "returnType": null, + "description": "At most two distinct cycle models, ordered by score." + }, + { + "name": "evidenceThroughAudioTime", + "signature": "evidenceThroughAudioTime: number", + "signatureTokens": [ + { + "text": "evidenceThroughAudioTime", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Audio-context time through which observations were processed." + }, + { + "name": "origin", + "signature": "origin: \"convention\" | \"estimated\" | \"manual\"", + "signatureTokens": [ + { + "text": "origin", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "\"convention\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"estimated\"", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "\"manual\"", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "" + }, + { + "name": "reason", + "signature": "reason: MeterEstimateReason", + "signatureTokens": [ + { + "text": "reason", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "MeterEstimateReason", + "kind": "type" + } + ], + "params": [], + "returnType": null, + "description": "" + }, + { + "name": "score", + "signature": "score: number", + "signatureTokens": [ + { + "text": "score", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Uncalibrated strength of the leading model, not a probability." + }, + { + "name": "selected", + "signature": "selected: MeterModel | null", + "signatureTokens": [ + { + "text": "selected", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "MeterModel", + "kind": "type" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "null", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "" + }, + { + "name": "validUntilAudioTime", + "signature": "validUntilAudioTime: number", + "signatureTokens": [ + { + "text": "validUntilAudioTime", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Audio-context time at which this view loses support without new evidence." + } + ], + "paragraphs": [], + "importLine": null, + "sourceLink": null + }, + { + "id": "source", + "title": "Source", + "members": [], + "paragraphs": [], + "importLine": null, + "sourceLink": { + "label": "packages/exojs-audio-fx/src/meterTypes.ts", + "href": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/meterTypes.ts" + } + } + ], + "sourcePath": "packages/exojs-audio-fx/src/meterTypes.ts", + "sourceUrl": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/meterTypes.ts" +} diff --git a/site/src/content/api/meter-model.json b/site/src/content/api/meter-model.json new file mode 100644 index 000000000..361a429d5 --- /dev/null +++ b/site/src/content/api/meter-model.json @@ -0,0 +1,148 @@ +{ + "title": "MeterModel", + "description": "", + "symbol": "MeterModel", + "kind": "interface", + "subsystem": "audio", + "importPath": "@codexo/exojs-audio-fx", + "tier": "stable", + "memberCount": 4, + "counts": { + "constructors": 0, + "methods": 0, + "properties": 4, + "events": 0 + }, + "sections": [ + { + "id": "import", + "title": "Import", + "members": [], + "paragraphs": [], + "importLine": "import { MeterModel } from '@codexo/exojs-audio-fx'", + "sourceLink": null + }, + { + "id": "properties", + "title": "Properties", + "members": [ + { + "name": "cyclePulses", + "signature": "cyclePulses: number", + "signatureTokens": [ + { + "text": "cyclePulses", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Pulses per cycle of the tracked grid, not necessarily a notated numerator." + }, + { + "name": "denominator", + "signature": "denominator: null | number", + "signatureTokens": [ + { + "text": "denominator", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "null", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Not inferred from audio; null unless explicitly supplied." + }, + { + "name": "phasePulses", + "signature": "phasePulses: null | number", + "signatureTokens": [ + { + "text": "phasePulses", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "null", + "kind": "keyword" + }, + { + "text": " | ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "Zero-based position of the last tracked beat, or null when unanchored." + }, + { + "name": "pulsePeriodSeconds", + "signature": "pulsePeriodSeconds: number", + "signatureTokens": [ + { + "text": "pulsePeriodSeconds", + "kind": "name" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "number", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "" + } + ], + "paragraphs": [], + "importLine": null, + "sourceLink": null + }, + { + "id": "source", + "title": "Source", + "members": [], + "paragraphs": [], + "importLine": null, + "sourceLink": { + "label": "packages/exojs-audio-fx/src/meterTypes.ts", + "href": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/meterTypes.ts" + } + } + ], + "sourcePath": "packages/exojs-audio-fx/src/meterTypes.ts", + "sourceUrl": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/meterTypes.ts" +} diff --git a/site/src/content/api/upcoming-beat.json b/site/src/content/api/upcoming-beat.json index 98d2e5f76..0f3c98dca 100644 --- a/site/src/content/api/upcoming-beat.json +++ b/site/src/content/api/upcoming-beat.json @@ -6,11 +6,11 @@ "subsystem": "audio", "importPath": "@codexo/exojs-audio-fx", "tier": "stable", - "memberCount": 4, + "memberCount": 5, "counts": { "constructors": 0, "methods": 0, - "properties": 4, + "properties": 5, "events": 0 }, "sections": [ @@ -89,6 +89,31 @@ "returnType": null, "description": "" }, + { + "name": "meterKnown", + "signature": "meterKnown?: boolean", + "signatureTokens": [ + { + "text": "meterKnown", + "kind": "name" + }, + { + "text": "?", + "kind": "punctuation" + }, + { + "text": ": ", + "kind": "punctuation" + }, + { + "text": "boolean", + "kind": "keyword" + } + ], + "params": [], + "returnType": null, + "description": "" + }, { "name": "tempo", "signature": "tempo: number", @@ -122,11 +147,11 @@ "paragraphs": [], "importLine": null, "sourceLink": { - "label": "packages/exojs-audio-fx/src/BeatDetector.ts", - "href": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/BeatDetector.ts" + "label": "packages/exojs-audio-fx/src/beatScheduleTypes.ts", + "href": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/beatScheduleTypes.ts" } } ], - "sourcePath": "packages/exojs-audio-fx/src/BeatDetector.ts", - "sourceUrl": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/BeatDetector.ts" + "sourcePath": "packages/exojs-audio-fx/src/beatScheduleTypes.ts", + "sourceUrl": "https://github.com/Exoridus/ExoJS/blob/main/packages/exojs-audio-fx/src/beatScheduleTypes.ts" } diff --git a/site/src/content/guide/audio/beat-detection.mdx b/site/src/content/guide/audio/beat-detection.mdx index 1fa405c6d..d917a87f3 100644 --- a/site/src/content/guide/audio/beat-detection.mdx +++ b/site/src/content/guide/audio/beat-detection.mdx @@ -45,6 +45,8 @@ const detector = new BeatDetector({ source: voice }); The detector runs its own FFT inside the worklet processor. No `AudioAnalyser` is required — `BeatDetector` handles frequency analysis internally. The `fftSize` option (default 2048) and `hopSize` (default 512) control the worklet's internal analysis window. +Assigning a different source, including `null`, clears rhythm history and pending predictions immediately. Assigning the same source again preserves continuity. Call `detector.reset()` after a seek or another playback discontinuity on the same source. The detector cannot infer a track seek, voice pause or EOF from an arbitrary bus. Missing input or a gap in the context-frame stream resets analysis; recorded silence instead remains valid audio and lets predictions expire. `barNumber` remains a lifetime counter across resets. For explicit playback ownership, call `notifyDiscontinuity(reason)` with `seek`, `loop`, `starvation`, `explicitPause`, `resume`, `eof` or `reset`. EOF and explicit pause stop analysis until resume, reset or another source identity. Seek/loop/starvation and manual-context changes preserve an existing stop; recorded silence alone does not end the source. + ## Tempo and confidence Two read-only getters give you the detector's current tempo estimate: @@ -76,7 +78,7 @@ Four signals fire at discrete moments in the music: | Signal | Fires when | Payload | |--------|------------|---------| -| `onBeat` | A beat occurs | `BeatInfo` — audio time, tempo, confidence, energy, isDownbeat, beatInBar, status (`'provisional'` \| `'locked'`) | +| `onBeat` | A beat occurs | `BeatInfo` — audio time, tempo, confidence, energy, isDownbeat, meterKnown, beatInBar, status (`'provisional'` \| `'locked'`) | | `onDownbeat` | The first beat of a bar | Same `BeatInfo` | | `onBarStart` | A new bar begins | `BarInfo` — audio time, tempo, confidence, barNumber | | `onTempoChange` | Tempo estimate changes | `(newBpm, oldBpm)` | @@ -123,7 +125,7 @@ declare const detector: BeatDetector; detector.onBeat.add(info => { if (info.isDownbeat) { // big effect on beat 1 - } else if (info.beatInBar === 3) { + } else if (info.meterKnown && info.beatInBar === 3) { // secondary emphasis on beat 3 } }); @@ -141,9 +143,11 @@ declare const detector: BeatDetector; detector.tempo // BPM, or 0 before lock detector.confidence // 0..1 locking quality detector.beatPhase // 0..1 phase within current beat -detector.barPosition // 1..N beat within current bar +detector.barPosition // display counting; check meterKnown for bar phase detector.barLength // beats per bar (default 4) -detector.timeSignature // { numerator, denominator } +detector.timeSignature // legacy display convention, denominator 4 +detector.meterEstimate // interpretation view; selected can be null +detector.meterKnown // current cycle and phase available detector.rms // 0..1 overall RMS energy detector.onsetStrength // 0..1 onset novelty at current frame detector.bandEnergy // { low, mid, high } band energies @@ -153,13 +157,23 @@ detector.lookahead // readonly array of upcoming beats These are pure derivations from the detector's most recent worklet state update — no allocation, no event dispatch. +Predictions additionally check the current audio-context time. `predictionValidUntil` reports when a retained schedule must be discarded, or 0 when unavailable. Without renewed onset support, this is two tracked beat periods plus one FFT window. After expiry, `lookahead` returns an empty array and `nextBeatTime` / `nextDownbeatTime` return 0. The next-time getters skip past entries; a retained `lookahead` array can still contain them. Later extrapolated times are conditional on the rhythm continuing. `onBeatPredicted` announces each received revision at most once, skips expired schedules, and does not coalesce distinct revisions queued for delivery. + +`minSettlingMs` bounds the first beat's posting time, rounded up to an analysis hop. A beat posted after warm-up can carry an earlier onset timestamp. `BeatInfo.energy` is raw novelty from the hop that emitted the beat, rather than a normalized score or necessarily the matched onset's energy. `onsetEnergy` and `onsetAudioTime` separately identify the matched onset, or are null without a match. `beatConfidence` omits the legacy meter factor; `confidence` retains it for compatibility with lock behavior. These are uncalibrated scores, and `gridStability` remains a legacy alias of `confidence`. Cached rhythm getters return unavailable values once their prediction validity expires, including when the worklet sends no new state. + Beats arrive `'provisional'` first — low-latency and ideal for snappy visual reactivity — then promote to `'locked'` once the grid settles, and `tempo`/`confidence` read 0 until that point. Gate anything that needs a dependable BPM — a sequencer, a UI readout — on `info.status === 'locked'`. ## Bar and time-signature tracking -The detector attempts 4/4 vs. 3/4 time-signature detection when `enableTimeSignatureDetection` is `true` (the default). Set it `false` in the constructor to lock to 4/4: +With `enableTimeSignatureDetection: true` (the default), the detector compares three- and four-pulse accent cycles on the same matched onset observations, using finalized attack power. Meter stays unknown until enough observations favor a cycle and phase over the alternatives. Beats continue during acquisition and ambiguity, with `meterKnown: false` and `isDownbeat: false`; no automatic `onDownbeat` or `onBarStart` events fire then. + +`meterEstimate` is an immutable view with `selected`, up to two distinct `alternatives`, `score`, `reason`, `origin`, `evidenceThroughAudioTime` and `validUntilAudioTime`. Its selected model contains `cyclePulses`, `pulsePeriodSeconds`, zero-based `phasePulses` and `denominator`. A null selection means the cycle is unknown; a null phase means that cycle has no identified bar anchor. `meterKnown` requires both. Reasons distinguish acquisition, insufficient observations, low contrast, cycle ambiguity, phase ambiguity, availability and expiry. Scores are not probabilities, and the automatic denominator remains null. + +The times use the audio-context clock. A current interpretation can expire before the beat schedule: `nextDownbeatTime` and `barPulse` then return 0 and `lookahead` keeps beat times while clearing downbeat flags. Finalized evidence may revise a schedule without a new beat; handle every new `onBeatPredicted` revision. The separate `schedule` view records epoch, revision, generation/evidence times, validity, conditional horizon, tracked pulse period and trust status at generation. It returns null after invalidation. Its `nextBeat` and `nextDownbeat` are future entries or null. A retained schedule with downbeat flags expires with the meter; fetching a new view can retain beat-only predictions for the remaining pulse deadline. + +Set detection `false` for an explicit four-beat counting convention, anchored to the first tracked beat rather than an inferred musical bar. Its estimate has `origin: 'convention'` and score 1, which describes the chosen rule rather than certainty about the music: ```ts import { BeatDetector } from '@codexo/exojs-audio-fx'; @@ -169,7 +183,26 @@ const detector = new BeatDetector({ }); ``` -`barPosition` (1-based) tells you which beat you're on within the current bar. `barLength` reports the detected bar length. Together they let you build sequencers, rhythm-game logic, and procedural music controllers that follow the music's structure. +`barPosition` (1-based), `barLength` and `timeSignature` remain legacy display values even while `meterEstimate.selected` is null; check `meterKnown` before treating them as bar information. `cyclePulses` counts the detector's tempo-grid pulses, not notated beats, and the default legacy denominator 4 is a display convention. Automatic discrimination and phase are not independently qualified on a representative music corpus. A locked beat grid does not establish a correct downbeat. Use an independently authored timeline when gameplay or a sequencer requires authoritative bars. + +When migrating existing callers, allow missing automatic bar events during acquisition or ambiguity. Use `onBeat` for continuous beat-level visuals and gate bar-dependent behavior on `meterKnown`; callers intentionally counting every four beats can select the fixed convention above. + +Provide explicit context when the intended cycle is already known: + +```ts +import { BeatDetector } from '@codexo/exojs-audio-fx'; + +declare const detector: BeatDetector; + +detector.manualMeter = { cyclePulses: 3, phasePulses: 0, denominator: 4 }; +// The next epoch anchors phase 0 to its first tracked beat. +detector.manualMeter = { cyclePulses: 3 }; // known cycle, unknown phase +// Return to automatic interpretation. +detector.manualMeter = null; +``` + +The origin is `manual`, so this does not claim an acoustic downbeat. Integers from 1 through 32 are accepted as cycle lengths; phase must be within that cycle, and an optional denominator must be a power of two from 1 through 128. Invalid values throw before changing analysis. The setter snapshots its input; changing context resets history and predictions. Source changes and `reset()` preserve configuration but establish a new phase anchor. Reapplying identical context preserves continuity. With a long cycle, the next downbeat may lie outside the eight-entry forecast and return 0 while a valid `barPulse` can still describe the last boundary. + ## Worklet processor @@ -198,7 +231,7 @@ const detector = new BeatDetector({ minSettlingMs: 400, // warm-up before the first (provisional) beat emitProvisionalBeats: true, // emit early provisional beats for visual reactivity melBands: 24, // mel filterbank bands - enableTimeSignatureDetection: true, // auto-detect 3/4 vs 4/4 + enableTimeSignatureDetection: true, // estimate a three-/four-pulse cycle, otherwise unknown source: application.audio.music, // initial source: bus or Voice (optional) }); ```