Skip to content

Add real-time multitaper spectrum diagnostics - #3

Merged
anjaldoshi merged 87 commits into
mainfrom
real-time-spectrum-refactor
Nov 10, 2026
Merged

anjaldoshi merged 87 commits into
mainfrom
real-time-spectrum-refactor

Conversation

@galenlynch

Copy link
Copy Markdown
Collaborator

not ready to be merged: need to test with the GUI

Summary

  • Move sample collection out of overlapping FFT buffers into a bounded planar
    FIFO and worker-owned circular history. Callbacks now copy each selected
    channel once and publish or drop the complete multichannel block.
  • Replace the uncalibrated Hamming estimate with mean-detrended, one-sided
    multitaper PSD estimation using runtime-generated DPSS tapers and persistent
    float FFTW plans.
  • Display the complete spectrum through Nyquist by default, with correct
    PSD/ASD units, linear or logarithmic frequency mapping, cursor values, and a
    worker-side mean-plus-peak pixel reduction.
  • Add asynchronous profile preparation, observable readiness and loss states,
    stable fixed/automatic dB ranges, low-variance Fine capture, and session
    reference overlay/delta comparison.
  • Harden shutdown and stream/channel validation without blocking the audio or
    message thread. Remove the unused legacy spectrum pipeline.

Why

The released implementation can discard callback tails, join samples across
gaps, and publish channels independently. It also plots an unnormalized squared
FFT magnitude after several undocumented smoothing stages. The result depends
on callback partitioning, window length, sample rate, and display range, which
makes comparisons unreliable during rig debugging.

This change gives every accepted block and spectrum an explicit sample range,
configuration generation, channel map, estimator description, and physical
unit. Expensive allocation, DPSS generation, and FFT planning occur on a
dedicated configuration thread while the previous valid runtime remains live.

Validation

  • Both standalone and Open Ephys processor CTest targets pass locally.
  • A double-precision direct-DFT oracle covers calibration, endpoints,
    detrending, leakage, noise density, channel isolation, and odd/even lengths.
  • Processor tests cover callback partitioning, gaps and overlaps, queue
    overflow, backlog shedding, asynchronous replacement, failed preparation,
    stop/restart races, capture, references, and display metadata.
  • On an i9-12900K, the eight-channel worker-to-display path measured about
    0.45 ms Fast, 1.04 ms Balanced, and 4.67 ms Fine at the median. Software
    repaint medians were 2.75–5.54 ms. These are local measurements, not target-rig
    acceptance results.

Dependencies and remaining validation

  • Requires the additive float/batched real-to-complex API proposed for
    OpenEphysFFTW and corresponding cross-platform fftw3f binaries.
  • The embedded selected-tridiagonal eigensolver uses a pinned, static, private
    OpenBLAS backend; hosted Windows and universal-macOS builds remain required.
  • Graphical testing with recorded rig data and controlled target-rig tail
    latency measurements remain before requesting final review.
  • Line detection/removal and post-subtraction adaptive weighting are follow-up
    scientific work; this PR keeps the raw calibrated spectrum visible.

Attribution

Defensive stream/range checks and the MSVC exception setting were adapted from
Anjal Doshi's refactor-optimization branch. The relevant commits preserve
co-authorship.

Track active acquisition callbacks before they access FIFO storage, reject starts that would replace a live worker, and retain processing state when a cooperative stop times out. Join the configuration builder cooperatively so JUCE never force-kills it while it owns planning state.
The canvas redraws at 60 Hz, and every redraw allocated:

- FrequencyPlot::plot() takes its vectors by value, because
  InteractivePlot::plot() does, so each series cost two vector copies.
  With up to four series on each of eight channels that is 64 allocations
  a frame.
- transformFrequencies() built a new frequency axis every frame, and the
  delta view a new zero line, though both change only with the bins or
  the scale.
- beginSpectrumFrame() built an Array of the frame's channels only to
  compare it with the legend's.
- updatePowerSpectrum() rebuilt the displayed traces of all eight channels
  on each channel's update, copying every trace eight times a frame.
- setFrequencyAxis() rebuilt the log tick labels and the axis title every
  frame, when usually only the amplitude range had moved.

Add FrequencyPlot::plotLine(), which takes const references and copies into
series storage kept from the previous frame, and draw through it. Cache the
transformed axis and the zero line until the bins or scale change, compare
the legend in place, rebuild only the channel that was updated, and rebuild
the tick labels only when the frequency axis changes. plot() stays for
callers of the base-class interface. What is drawn does not change.

DEVELOPMENT.md gains the invariant: no per-frame allocation in the canvas
render path.
The spectrogram mode stays as it was; these are its outright defects.

- Each frame wrote its column with setPixelAt(), which constructs an
  Image::BitmapData per call, and evaluated a ColourGradient rebuilt that
  frame for every pixel. The column is now written through one BitmapData,
  from a colour map sampled once at construction.
- The image was 1000x1000 whatever the canvas size or display scale, so
  the frequency axis was resampled to 1000 rows. Its height now follows
  the drawn area in physical pixels, and in this mode the worker is asked
  for one display column per row rather than the plot's width. The width
  stays at 1000 frames stretched across the area, so the scroll speed is
  unchanged. Resizing rescales the history instead of clearing it.
- The last row of each column was never written, so the lowest frequency
  stayed black.
- Only the first selected channel is drawn, and nothing said so. The
  spectrogram now names that channel above the image, and the Display
  control's tooltip says which controls apply to it: background removal
  does; the background fit, peak envelope and reference comparison are
  drawn on the power spectrum only. README.md and DEVELOPMENT.md say the
  same.
AperiodicSpectrumBaseline::estimate() had two undocumented behaviours:

- Its interpolation walks the knots forward only, so it interpolated
  between the wrong knots whenever outputFrequenciesHz was not ascending.
  It now restarts the walk when an output frequency goes backwards, so
  order affects speed only.
- A channel with too little positive power to fit made it return false
  from inside the channel loop, after writing the channels before it. The
  caller then dropped the background for every channel, so one silent
  channel - a disconnected one, say - removed the fit from all of them.
  That channel now gets NaN, the value already meaning "no fit here", and
  the others are unaffected. estimate() returns false only for invalid
  arguments, before writing anything.

SingleTaperPeriodogram::compute() lacked the check MultitaperPeriodogram
has for power beyond float range: finite input can still square past it,
and the narrowing cast published infinity. It now rejects that, and like
the multitaper estimator it writes into a working buffer and swaps, so a
rejected input leaves the previous output unchanged.

Both headers now state these contracts. Three new tests, each of which
fails with its fix reverted; 118/118 tests pass.
CanvasPlot::resized() sized the plot as the canvas width less the legend
and margins, which goes negative below 190 px. The power spectrum enforces
an 800 px minimum, but the spectrogram fills whatever width the canvas has,
so a narrow canvas handed the plot a negative size. Clamp it at zero.

Also remove a LOGC that SpectrumCanvas::resized() wrote on every resize
event, left over from debugging the layout.
Remove code that nothing reaches:

- MS_FROM_START, a timing macro nothing expands.
- SpectrumViewer::displayType, a public member nothing reads; the canvas
  keeps its own display type.
- Commented-out declarations left from an earlier sample-buffering design.
- CanvasPlot::setBinWidth, defined and never called.
- The knot pre-fill loop in AperiodicSpectrumBaseline::estimate. Every
  knot it wrote is overwritten per channel before it is read, so it only
  cost time.
The estimators and transport buffers each carried their own copy of the
same few helpers: overflow-checked size products (ten variants), the
size-to-int check, the FIFO sizing rule, the detrend-mode check (five
copies), the fitted-trend struct (three) and the rule for which one-sided
bins are doubled. They live once in SpectrumSupport.h now.

Only the arithmetic is shared. Each call site keeps its own policy -
whether an empty dimension is an error, which exception it throws and
with what message - so no constructor accepts or rejects anything it did
not before, and the PSD arithmetic is unchanged.

ReferencePeriodogram keeps its own copy of the doubled-bins rule on
purpose. It is the oracle the estimators are tested against, and sharing
the code it checks would let a bug in that rule pass both. DEVELOPMENT.md
records that as an invariant and lists the new header in the source map.

The +/-25 dB default range and the 20 dB minimum span were likewise
defined in both SpectrumAmplitudeRange and SpectrumDisplaySettings. They
are now defined once, in SpectrumAmplitudeRange.h, which enforces them,
and the settings refer to them.

SpectrumSupportTests covers the helpers directly: overflow in any factor,
zero factors, the int and FIFO bounds, and the DC/Nyquist rule for odd and
even lengths. 123/123 tests pass.
SpectrumViewer carried about thirty loose atomics, and five places reset
overlapping subsets of them by hand: startAcquisition,
clearAcquisitionState, the route teardown, runtime adoption and capture
start. Nothing checked that a field added to one list reached the others.

Group them: TransportDiagnostics for the counts of lost, rejected and
skipped input, and CaptureRuntimeState for the capture in progress or on
screen, with its worker-only fields kept apart from the atomics it
publishes. Each has reset methods, and the five sites call those instead.
A full capture reset now also clears the worker-only fields, and withdraws
retainedId before releasing the capture it names. It runs only on the
worker or while the worker is stopped, so neither difference is visible.

run() was over 200 lines, nested four deep. It is now a short loop over
named steps - republish a frozen capture, process queued input, wait -
and processing a block is split the same way. The order of operations is
unchanged, including one that is easy to lose: the shedding decision is
taken before a block is checked for rejection, because the policy has to
see every dequeued block.
The configuration thread lives as long as the plugin and woke every 10 ms
whether or not it had anything to do. It now sleeps until request(),
retire() or the destructor notifies it, and polls only while a retired
runtime is still held elsewhere, since nothing announces when that holder
lets go. A notify that arrives before the wait leaves the event signalled,
so none is lost; 200 repeats of the configuration tests pass.

The analysis worker cannot be notified - the audio callback is its only
producer, and Thread::notify() takes a mutex - so it polls, and did so
every 2 ms throughout acquisition. Input arrives one block per callback,
so polling more than twice a block only finds the queue empty. It now
waits half the last block's duration, clamped to 2-8 ms: never later than
before for short blocks, and at most half a 60 Hz frame of added display
latency. With no runtime installed it waits 20 ms, since nothing can
arrive until a build finishes. For typical 20-35 ms blocks that is about
125 wakeups a second instead of 500, estimated from block sizes rather
than measured.

The worker change is in SpectrumViewer.cpp, which the processor tests
cover and which could not be built here. DEVELOPMENT.md describes both
changes.
A reference captured in one session could not be kept for the next, so a
baseline recorded under ideal conditions had to be captured again before
every experiment that needed it.

Export now writes one channel of the reference to a JSON file, asking
which one when the reference has several, and Import loads such a file
as a baseline. A baseline is a different kind of reference from one
captured this session. It is compared with every selected channel of
whichever stream is selected, and only the sample rate and the Fine
estimator have to match: stream ids are assigned per session, and a
baseline describes what one channel should look like, not which channel
it came from. A session reference is still held to its stream, channels
and units, and findMismatch() reports the first difference so the status
can say what to change. Both share one slot, and either survives stopping
and starting acquisition until it is cleared.

The worker broadcasts a baseline into each channel's comparison data, so
the dB difference and the readouts work unchanged. A frame flag tells the
plot to draw the overlay once, in a neutral colour, rather than once per
channel. Export is disabled while a baseline is loaded, since it is
already a file.

The file stores the spectra as base64 little-endian float32, readable with
numpy, and stores each double that compatibility compares exactly as its
IEEE-754 bit pattern, because a decimal round trip can lose the last bit.
Its size is checked before anything is allocated. Files from a newer
version are rejected. A saved signal chain remembers an imported
baseline's path, never the spectra or a reference captured in the
session. An import that fails keeps the current reference and says why.

Set Ref moves to the capture group, because it only ever acts on a frozen
capture. The capture status reads "Ref set; Go Live" once the frozen
capture is the reference, and the Go Live tooltip says whether going live
keeps or discards it. The reference status names the kind and when it was
captured: the time for a session reference, the date and time for a
baseline.

SpectrumReferenceFileTests covers exact round trips of the chosen channel,
the little-endian encoding, the looser matching of a baseline and files
that must be rejected. 130/130 tests pass. The processor and canvas
changes are covered by the processor tests, which could not be built here.
README.md and DEVELOPMENT.md describe the workflow, the file format and
the two kinds of reference.
@galenlynch

Copy link
Copy Markdown
Collaborator Author

@anjaldoshi just tested the latest commit on a rig and it worked beautifully.

anjaldoshi and others added 6 commits October 21, 2026 12:47
After a capture freezes, draw an arrow from the Set Ref button to the Reference group in the options drawer. Disable Set Ref and hide the arrow once the capture has been set as the reference.
@anjaldoshi
anjaldoshi force-pushed the real-time-spectrum-refactor branch from ff900fd to 296df90 Compare October 26, 2026 23:23
@anjaldoshi
anjaldoshi requested a review from jsiegle October 27, 2026 18:57
@anjaldoshi
anjaldoshi merged commit 2f2ccb9 into main Nov 10, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants