The Big One 🌊

A Great Leap Forward! The old call style and all the deprecation warnings and guards and tests that went with it have been fully removed, and I've had the excuse to make a few other backwards-compatibility-breaking changes.

• (breaking) interface improvements:
  • bandlimit in Hz replaces unitless tvgamma
  • metric parameter dropped from optimize because error correlation is optimized by degenerately not smoothing at all
  • spectraldiff now has short-named cutoff_freq to match butterdiff, pad_to_zero_dxdt renamed pad_to_flat, and even_extension renamed extension now taking a string
  • renamed mean_kernel to uniform_kernel
  • renamed step_size to stride for sliding window methods
  • splinediff drops num_iterations, because it became inert after sensitivity of its other parameter s to data scale was fixed last version
  • solver dropped from convex solve methods, now depending on either OSQP or CLARABEL intentionally, so the user doesn't have to think about it
• lineardiff is now mathematically leaner, faster, and easier to understand; it also elegantly handles missing and irregularly-spaced data
• slide_function now takes samples from a window in t units, rather than some number of samples, which was risking shrunken window width in dense samples, allowing methods to fit noise
• *_kernel functions are now optionally sampleable at arbitrary locations in addition to supporting fixed grids
• waveletdiff no longer commits the "wavelet crime" (now performing both denoising and differentiation in the basis), adds cycle-spinning for more reliable answers, can do adaptive decomposition depth, chooses hard thresholding over soft for less bias, and gets a more complete optimization search space
• spectraldiff can now perform odd extension, as well as detrending alone
• new utilities: robust_data_scale and robust_noise_scale
• speed: gradients now given to estimate_integration_constant's huber-branch SLSQP minimization; waveletdiff uses local FIR filters instead of sparse matrices or a global FFT
• guards: time vectors must be strictly increasing; waveletdiff rejects families that won't work
• bugfixes: integer type can't accidentally truncate answers anymore; polyfit fed mathematically-correct √kernel; robustdiff better handling for stiff cases, fewer warnings; rescaled friedrichs kernel to not go incredibly close to zero at edges, which could cause warnings

JOSS 2.0 in-review

A round of fixes and improvements prompted by JOSS review

• Methods no longer lose relative accuracy when data is rescaled; a test now enforces f(a*x) == a*f(x)
• lineardiff now supports axis, defaults to CLARABEL, and is fast enough to join suggest_method and the optimization demo notebook
• robustdiff's search space is leaner and faster, now that log_r has a natural set point at 0
• optimize no longer swallows warnings, which surfaced a batch of numerical bugs since fixed, including in splinediff, estimate_integration_constant, and robust_rme; also now uses starmap for speed, to make CPU usage less bursty
• Windows are constrained and rounded to odd sizes in optimize, because evenness silently degraded results; windows also now forced to touch or overlap so samples cannot be left uncovered between windows nor at the array tail
• Missing values and variable timesteps now raise explanatory errors instead of often silently returning NaNs
• Dependency list corrected, with a few min versions added
• General hardening and bugfixes: spectraldiff wavenumber indexing was off-by-one, rtsdiff no longer aliasies on circular domains; savgoldiff now measures length along correct axis; butterdiff now uses second-order sections; polydiff bounds window size against degree; waveletdiff no longer NaNs out at threshold 0; optimize now has a test to ensure cache hash collisions in the case of duplicated queries; and methods that do not take variable step size or missing data now ValueError if given incompatible inputs, with messages about why

Joss 2.0 pre-review

• Added wavelet-based differentiation/smoothing method
• Added circular domain support for rtsdiff via custom innovation function to Kalman filter

Revamped++

Still further improvements to complete the overhaul:

• Multidimensional support: all non-deprecated methods (except lineardiff) now have an axis parameter.
• Several methods have been updated to handle missing values.
• robustdiff has been updated to handle variable dt between data points while still running fast.

Revamped+

Further enhancements to the overhaul, including:

• collection of most methods in the smooth_finite_difference module as kerneldiff
• convex optimization improvements to make robustdiff run in linear time
• caching to avoid duplicate calls during optimization
• extension of utilities and loss function to better optimize all methods in the presence of outliers
• extension of TVR to better handle outliers
• improved test coverage and control of coveralls
• linted code
• added support for multidimensional data to several methods

Revamped

It's been Pavelized, everything combed through and rewritten. The biggest user-facing change is the support of (and preference for) keyword arguments, but methods have also been corrected, expanded, reorganized, extended to handle variable step sizes where possible, given better tests, integrated with improved optimization code, documented thoroughly, and been put through a head-to-head performance analysis. All while removing about 2000 lines of code to make it more readable, manageable, and understandable. Enjoy.

Final v0.1 checkpoint

Final tag and release before major overhaul.

PyNumDiff 0.1.2.4 JOSS

Final JOSS release with corrected author list.

PyNumDiff 0.1.2

Notable changes:

• Requires python >= 3.5, largely due to the deprecated numpy.matrix syntax
• New pi_cruise_control function
• No longer requires cvxpy and pychebfun for installation, though these are optionally required for certain functions
• Skip tests that require cvxpy and pychebfun if these are not installed

PyNumDiff 0.0.3

Final logging fixes.