From 4e85cadf03880685edf5467db6f44768af3aab09 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 02:55:06 +0000 Subject: [PATCH 1/6] test: score the prediction on 30 hand shapes and 9 devices MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The prediction was tuned against four shapes on four devices, which left most of what a hand does unwatched and made every change a question for the rig and an afternoon. This replaces that with a replay rig the fit can be iterated against on its own. `harness.ts` holds it: a hand on a known path, a modelled device reporting it in whole counts at its own rate and clock wander, a frame loop reading the lead back, and seventeen numbers scoring the run. Devices now span an 8KHz mouse to one reporting less than once a frame, including a low-DPI one whose counts are four pixels wide; displays span 30Hz to 144Hz; delivery lag is an axis of its own. Shapes: steady drags from a crawl to a whip, ramps, flicks stopping in anywhere from 40ms to 400ms, abrupt starts and stops, stop-and-go, point to point, sharp and rounded corners, circles both ways, a stir, a figure eight, a spiral, a serpentine, a zigzag, shakes, target acquisition, overshoot-and-correct, a double flick and tremor at rest and on a drag. Scored per run, and never adding overshoot to lateness: how far past the hand the guess sat and how far behind, how much the picture moved against the hand or moved at all while the hand was still, how much the size and heading of the guess jumped in a frame, and the error against the error of not guessing. `bun run bench` prints the same runs as a table, with --save/--diff so a tuning change is read as a change rather than found later on a desk. Every constant in predictor.ts was broken in turn against this and every one now fails something; two lines no modelled hand reaches are named in predictor.test.ts rather than tested for the sake of it. Two findings are pinned at their current values with the diagnosis next to them: a device whose counts are wider than a pixel gets floors written in units of one, so it spins the lead 28° a frame on a dead straight path and leads a crawl it should not; and an eased stop on a slow radio hands back up to 14px in a single frame. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HctBLnHDXwx1XURcvnEB9T --- README.md | 33 + library/src/__tests__/harness.ts | 867 ++++++++++++++++++ .../src/__tests__/predictor-devices.test.ts | 284 ++++++ .../__tests__/predictor-invariants.test.ts | 312 +++++++ .../src/__tests__/predictor-shapes.test.ts | 186 ++++ library/src/__tests__/predictor-stops.test.ts | 296 ++++++ library/src/__tests__/predictor.test.ts | 208 +---- library/src/__tests__/scoreboard.ts | 117 +++ library/tsconfig.json | 3 + package.json | 1 + 10 files changed, 2133 insertions(+), 174 deletions(-) create mode 100644 library/src/__tests__/harness.ts create mode 100644 library/src/__tests__/predictor-devices.test.ts create mode 100644 library/src/__tests__/predictor-invariants.test.ts create mode 100644 library/src/__tests__/predictor-shapes.test.ts create mode 100644 library/src/__tests__/predictor-stops.test.ts create mode 100644 library/src/__tests__/scoreboard.ts diff --git a/README.md b/README.md index 5514e3d..9d3c420 100644 --- a/README.md +++ b/README.md @@ -266,6 +266,39 @@ The red square is the reported position, the green circle is the prediction, and (the compositor draws it, so it is never late). `?compare=1` shows two leads at once; `?nocursor=1`, `?noreported=1` and `?nopredicted=1` strip the frame down to what you want to read. +## Changing the prediction + +The rig above is the only way to see presentation latency, and it needs a hand, a camera and an afternoon. +Everything else is measured off a replay instead, so the fit can be changed and judged without one: + +```bash +bun test # 30 shapes × 9 devices, against floors that were measured +bun run bench # the same runs, printed as numbers +bun run bench circle --all # one family of shapes, on every device +bun run bench --save before.json +bun run bench --diff before.json # what your change actually moved, marked ↓ better ↑ worse +``` + +A hand moves along a known path, a modelled device reports it in whole counts at its own rate and with its +own clock wander, a frame loop reads the lead back out, and the run is scored against where the hand really +is when each frame reaches the screen. Both ends are modelled because both ends matter: a wired mouse and one +on a radio are different problems, and so are 60Hz and 144Hz. + +The shapes are in `library/src/__tests__/harness.ts` — steady drags from a crawl to a whip, flicks stopping +in anywhere from 40ms to 400ms, circles, figure eights, spirals, corners taken sharp and round, shakes, +zigzags, target acquisition, tremor, and stops and starts between all of them. Add one there and the +scoreboard picks it up; the shapes suite will then fail until it is given a floor, which is the point. + +What is scored, per run: how far past the hand the guess ever sat and how far behind, how much the picture +moved against the hand or moved at all while the hand was still, how far the guess jumped in a frame, and the +error against the error of not guessing at all. Overshooting and falling short are never added together — +lead that was never needed has to be handed back, and handing it back walks the view backwards under the +hand, where lead that was never taken is only the lateness the library exists to remove. + +Every figure in a test was read off that rig and given about a quarter more room. **Improve one and tighten +it in the same change**, or the floor stops being a floor. Most changes trade one column for another; the +`--diff` output is there so the trade is visible rather than discovered later on somebody's desk. + ## License MIT diff --git a/library/src/__tests__/harness.ts b/library/src/__tests__/harness.ts new file mode 100644 index 0000000..6409da9 --- /dev/null +++ b/library/src/__tests__/harness.ts @@ -0,0 +1,867 @@ +// The rig the guess is judged on, written down once so every test is judging the same thing: a hand moving +// along a known path, a device reporting that motion the way real hardware does, a frame loop reading the +// lead back out, and the handful of numbers that say whether the guess was worth having. +// +// The point of keeping it here rather than in any one test file is that the prediction is tuned from these +// numbers. A change to the fit moves all of them at once, and the only way to tell an improvement from a +// trade is to see the whole board — so the same replay feeds the assertions and `bun run bench`, and a +// scenario added here is scored by both. +// +// Everything is deterministic: the clock wander comes from a seeded generator, so a figure measured today is +// the figure measured on the next machine. Any number in a test file was read off this rig and given room. + +import { createPointerPredictor, DEFAULT_DECAY_MS, DEFAULT_LEAD_FRAMES, DEFAULT_MAX_LEAD_PX } from '../predictor.js' + +export const FRAME = 16.7 +/** What the newest sample's age measures at at 60Hz, per the example rig. */ +export const AGE = 13 +export const LEAD_FRAMES = DEFAULT_LEAD_FRAMES + +/** Refresh intervals worth replaying at. The lead is a count of refreshes, so each of these is a different + * horizon in milliseconds off the same hand. */ +export const HZ_30 = 1000 / 30 +export const HZ_60 = 1000 / 60 +export const HZ_120 = 1000 / 120 +export const HZ_144 = 1000 / 144 + +export type Vec = { x: number; y: number } +/** Where the hand is at a given moment, in pointer pixels. Continuous — the quantisation is the device's + * job, and modelling it here would hide the thing most likely to wreck a second derivative. */ +export type Path = (t: number) => Vec + +/** How a device reports: how often, how many at a time, how much its clock wanders, and how coarse a step + * one of its counts is worth. */ +export type Device = { + name: string + periodMs: number + burst: number + jitterMs: number + /** Pointer pixels per count. A high-DPI mouse at default sensitivity moves about a pixel per count; a low + * one moves several, and a fit over four of those is looking at a staircase rather than a path. */ + countPx: number +} + +const device = (name: string, periodMs: number, burst: number, jitterMs: number, countPx = 1): Device => ({ + name, + periodMs, + burst, + jitterMs, + countPx, +}) + +export const WIRED = device('wired', 1, 1, 0.1) +export const TRACKPAD = device('trackpad', 11, 1, 0.4) +/** A mouse on a radio: reports in clumps, and says nothing at all for a third of frames while moving. */ +export const BLUETOOTH = device('bluetooth', 8, 3, 2.5) +/** Slower still, and the one whose gaps most resemble a hand that has stopped. */ +export const SLOW_RADIO = device('slow radio', 16, 2, 3) +/** The default of every mouse that is not a gaming one: one report per 8ms, which is two per frame. */ +export const USB_125 = device('125Hz mouse', 8, 1, 0.5) +export const USB_500 = device('500Hz mouse', 2, 1, 0.2) +/** Eight reports per millisecond, so the window is full and every sample carries almost no motion — the + * case where a second derivative is being asked of nothing but quantisation. */ +export const USB_8K = device('8KHz mouse', 0.125, 1, 0.05) +/** Coarse counts: four pixels a step, which is what a low-DPI mouse reports. The path arrives as a + * staircase, and every noise floor in the fit is written in units of one count. */ +export const COARSE = device('low-DPI mouse', 8, 1, 0.5, 4) +/** The sparsest thing that still calls itself a pointer: less than one report a frame, so most frames see + * nothing and the window holds two or three samples. */ +export const SPARSE = device('sparse', 25, 1, 4) + +/** The four the shipped figures are quoted against. */ +export const DEVICES: readonly Device[] = [WIRED, TRACKPAD, BLUETOOTH, SLOW_RADIO] +/** Every device, including the awkward ones that exist to be survived rather than to be fast. */ +export const ALL_DEVICES: readonly Device[] = [ + WIRED, + USB_8K, + USB_500, + USB_125, + TRACKPAD, + BLUETOOTH, + SLOW_RADIO, + COARSE, + SPARSE, +] + +export type ReplayOptions = { + path: Path + device: Device + durationMs: number + /** Stalls one frame in this many to three times its length, as a real one does. 0 for none. */ + hitchEvery?: number + frameMs?: number + leadFrames?: number + decayMs?: number + maxLeadPx?: number + /** When scoring starts, so the window filling up is not read as steady-state behaviour. */ + scoreAfterMs?: number + /** How long before the frame the browser stops handing events over. A couple of milliseconds is an + * ordinary machine; a busy main thread, a compositor under load or a device driver having a bad day make + * it tens, and every one of them goes into the horizon as time the guess has to cover. */ + deliveryLagMs?: number + /** How many refreshes after a frame is drawn it reaches the screen — what the guess is scored against. + * Defaults to `leadFrames`, which is the assumption the lead is set from. */ + presentFrames?: number + seed?: number + /** Every frame as it happens, for the properties that are about each frame rather than about the run: a + * lead that is finite, inside the cap, and pointing somewhere the samples support. Scored frames only are + * not enough for those — a guess that goes wrong while the window is filling has still gone wrong. */ + onFrame?: (frame: { + /** Frame index from 1, and the clock reading it was drawn at. */ + index: number + frameAt: number + lead: { x: number; y: number; live: number } + /** Where the hand truly is, where the device has said it is, and what the screen would show. */ + hand: Vec + reported: Vec + shown: Vec + /** What the hand did since the previous frame. Zero while it is still. */ + handStep: Vec + }) => void +} + +export type Metrics = { + /** Frames scored, so a figure averaged over them can be read for what it is. */ + frames: number + /** Frame-to-frame move of the lead. On a hand doing nothing new every bit of it is jitter, and jitter on + * a camera is nauseating, so the worst one matters more than the average. */ + worstStep: number + meanStep: number + /** A step that is itself a change of step: the lead changing speed, which is what the eye reads as + * shaking rather than as motion. */ + worstJerk: number + /** How much the size of the guess jumps in one frame, and how much its heading jumps against the hand's. + * On a curving path most of `worstStep` is the lead legitimately swinging round with the hand — it is + * 10px a frame on an ordinary circle and none of it is jitter — so these two are what steadiness means + * anywhere except a straight line. Degrees for the second. */ + worstSizeStep: number + worstTurnStep: number + /** RMS distance from where the hand really is when this frame reaches the screen. */ + error: number + /** The same, for a caller that does not guess at all. The only thing `error` means anything against. */ + errorWithout: number + /** error / errorWithout. Under 1 is worth having; over 1 is worse than nothing. */ + gain: number + /** Mean signed distance along the way the hand is going: positive is systematically ahead of the hand, + * negative is systematically behind it. Which side of zero the errors sit on, rather than how big. */ + bias: number + /** Furthest past the hand the guess ever sat, along the way the hand is going. Lead that was never needed + * has to be handed back, and handing it back walks the view backwards under the hand. */ + worstOvershoot: number + /** Furthest behind the hand it ever sat: the lateness the library exists to remove. Scored apart from + * overshoot because the two are not the same mistake. */ + worstLate: number + /** Total on-screen travel against the way the hand is moving at that moment, over the whole run. Measured + * against what the hand did between these two frames rather than against a fixed direction, so a hand + * that genuinely turns round is not scored as the picture fighting it. Overshoot being handed back and + * jitter both land here, and this is the one artefact a camera cannot hide. */ + backwards: number + /** The worst of that in a single frame — one visible kick rather than a slow drift back. */ + worstBackwards: number + /** On-screen travel over frames where the hand is not moving at all: motion nobody asked for, in either + * direction. A guess still holding lead when the hand stops has to give every pixel of it back, and this + * is what that costs. */ + phantom: number + worstLead: number + meanLead: number + /** How far the guessed position sits from the origin, averaged. Only means something on a circle, where + * it is the radius the guess draws against the radius the hand drew. */ + meanRadius: number + /** What a caller applying only the growth accumulates. Jitter is rectified into rotation there: every + * upward wobble is kept and none of the downward ones give it back. */ + kept: number + /** Frames the predictor called dead while the hand was in fact moving. A stop seen late is overshoot; a + * stop seen where there is none is the lead dropping out under a moving hand. */ + falseDeadFrames: number +} + +/** What a caller applying only the growth of the lead would keep from this frame: an axis moving further + * out in the direction it already pointed, and nothing given back when it comes in again. */ +const growth = (value: number, applied: number) => + value * applied >= 0 && Math.abs(value) > Math.abs(applied) ? value - applied : 0 + +/** An angle difference brought back into a half turn either side, so a swing past pi is read as the small + * turn it is rather than as a nearly whole one. */ +const wrapped = (angle: number) => { + let value = angle + while (value > Math.PI) value -= 2 * Math.PI + while (value < -Math.PI) value += 2 * Math.PI + return value +} + +/** + * One run of a hand along `path`, reported by `device`, read by a frame loop, scored against where the hand + * really is when each frame reaches the screen. + */ +export const replay = (options: ReplayOptions): Metrics => { + const { + path, + device: dev, + durationMs, + hitchEvery = 0, + frameMs = FRAME, + leadFrames = LEAD_FRAMES, + decayMs = DEFAULT_DECAY_MS, + maxLeadPx = DEFAULT_MAX_LEAD_PX, + scoreAfterMs = 600, + presentFrames = leadFrames, + deliveryLagMs = 2, + seed = 20260808, + onFrame, + } = options + + const predictor = createPointerPredictor() + let rndState = seed + const rnd = () => (rndState = (rndState * 1103515245 + 12345) & 0x7fffffff) / 0x7fffffff + + // Whole counts, and a count is not always a pixel. + const countPx = dev.countPx + const quantise = (value: number) => Math.trunc(value / countPx) * countPx + + let reportedX = 0 + let reportedY = 0 + let integratedX = 0 + let integratedY = 0 + let frameAt = 0 + let prevFrameAt = 0 + let nextReport = 0 + let previous: Vec | null = null + let previousStep: Vec | null = null + let previousShown: Vec | null = null + let keptX = 0 + let keptY = 0 + let appliedX = 0 + let appliedY = 0 + let worstStep = 0 + let stepSum = 0 + let worstJerk = 0 + let worstSizeStep = 0 + let worstTurnStep = 0 + let previousRelativeAngle: number | null = null + let previousHandAngle: number | null = null + let phantom = 0 + let worstLead = 0 + let leadSum = 0 + let worstOvershoot = 0 + let worstLate = 0 + let backwards = 0 + let worstBackwards = 0 + let biasSum = 0 + let radialSum = 0 + let withSum = 0 + let withoutSum = 0 + let falseDeadFrames = 0 + let scored = 0 + + let frameIndex = 0 + while (frameAt <= durationMs) { + frameIndex++ + frameAt += frameMs * (hitchEvery > 0 && frameIndex % hitchEvery === 0 ? 3 : 1) + // Events are held until shortly before the callback, and arrive a burst at a time. + while (nextReport <= frameAt - deliveryLagMs) { + for (let b = 0; b < dev.burst; b++) { + const t = nextReport - (dev.burst - 1 - b) * dev.periodMs + (rnd() - 0.5) * dev.jitterMs + if (t <= 0) continue + const at = path(t) + const dx = quantise(at.x) - reportedX + const dy = quantise(at.y) - reportedY + if (dx !== 0 || dy !== 0) { + predictor.push(t, dx, dy) + reportedX += dx + reportedY += dy + integratedX += dx + integratedY += dy + } + } + nextReport += dev.periodMs * dev.burst + } + + // What the hand itself did over this frame, which is what everything on screen is judged against. Read + // before the frame clock moves on, since that is what it is a difference against. + const from = path(prevFrameAt) + const to = path(frameAt) + const handX = to.x - from.x + const handY = to.y - from.y + const handStep = Math.hypot(handX, handY) + const moving = handStep > 0.02 * (frameAt - prevFrameAt) + + const lead = predictor.update({ + nowMs: frameAt, + deltaMs: frameAt - prevFrameAt, + leadFrames, + decayMs, + maxLeadPx, + }) + prevFrameAt = frameAt + + keptX += growth(lead.x, appliedX) + keptY += growth(lead.y, appliedY) + appliedX = lead.x + appliedY = lead.y + + // Once the window is full of the motion rather than of starting up. + if (frameAt > scoreAfterMs) { + const step = previous ? Math.hypot(lead.x - previous.x, lead.y - previous.y) : 0 + if (previous) { + worstStep = Math.max(worstStep, step) + stepSum += step + const stepVec = { x: lead.x - previous.x, y: lead.y - previous.y } + if (previousStep) { + worstJerk = Math.max(worstJerk, Math.hypot(stepVec.x - previousStep.x, stepVec.y - previousStep.y)) + } + previousStep = stepVec + worstSizeStep = Math.max( + worstSizeStep, + Math.abs(Math.hypot(lead.x, lead.y) - Math.hypot(previous.x, previous.y)), + ) + } + // Where the guess points against where the hand is going. A lead swinging round with a turning hand + // holds this still; a lead being shaken about by noise does not. Below a pixel there is no heading to + // read and the size step has it covered, and a hand that has itself turned by a quarter turn between + // two frames is asking a question about the reversal rather than about steadiness. + const handAngle = Math.atan2(handY, handX) + const handTurned = previousHandAngle === null ? 0 : Math.abs(wrapped(handAngle - previousHandAngle)) + previousHandAngle = moving ? handAngle : null + if (moving && handTurned < Math.PI / 4 && Math.hypot(lead.x, lead.y) > 1) { + const relative = Math.atan2(lead.y, lead.x) - handAngle + if (previousRelativeAngle !== null) { + const turned = wrapped(relative - previousRelativeAngle) + worstTurnStep = Math.max(worstTurnStep, Math.abs((turned * 180) / Math.PI)) + } + previousRelativeAngle = relative + } else { + previousRelativeAngle = null + } + const truth = path(frameAt + presentFrames * frameMs) + + // How far past the hand the guess sits, along the way the hand is going. Overshooting and falling + // short are not the same error and are not worth scoring as one: lead that was never there has to + // be handed back, and handing it back is a reversal on screen, where falling short is only the + // lateness this whole thing exists to remove. + const ahead = path(frameAt + presentFrames * frameMs + 1) + const behind = path(frameAt + presentFrames * frameMs - 1) + const towardsX = ahead.x - behind.x + const towardsY = ahead.y - behind.y + const towards = Math.hypot(towardsX, towardsY) + const shownX = integratedX + lead.x + const shownY = integratedY + lead.y + if (towards > 1e-9) { + const past = ((shownX - truth.x) * towardsX + (shownY - truth.y) * towardsY) / towards + worstOvershoot = Math.max(worstOvershoot, past) + worstLate = Math.max(worstLate, -past) + biasSum += past + // The hand is demonstrably moving here, so a dead reading is the predictor losing a moving hand + // rather than noticing a stopped one. + if (moving && lead.live === 0) falseDeadFrames++ + } + // What the screen does against what the hand does over the same frame. A picture moving the other way + // is the guess handing something back, whether it was overshoot or noise; a picture moving at all + // while the hand is still is motion nobody asked for. + if (previousShown) { + const movedX = shownX - previousShown.x + const movedY = shownY - previousShown.y + if (moving) { + const along = (movedX * handX + movedY * handY) / handStep + if (along < 0) { + backwards -= along + worstBackwards = Math.max(worstBackwards, -along) + } + } else { + phantom += Math.hypot(movedX, movedY) + } + } + withSum += (shownX - truth.x) ** 2 + (shownY - truth.y) ** 2 + withoutSum += (integratedX - truth.x) ** 2 + (integratedY - truth.y) ** 2 + radialSum += Math.hypot(shownX, shownY) + const magnitude = Math.hypot(lead.x, lead.y) + worstLead = Math.max(worstLead, magnitude) + leadSum += magnitude + previousShown = { x: shownX, y: shownY } + scored++ + } + previous = { x: lead.x, y: lead.y } + onFrame?.({ + index: frameIndex, + frameAt, + lead: { x: lead.x, y: lead.y, live: lead.live }, + hand: to, + reported: { x: reportedX, y: reportedY }, + shown: { x: integratedX + lead.x, y: integratedY + lead.y }, + handStep: { x: handX, y: handY }, + }) + } + + const frames = Math.max(1, scored) + const error = Math.sqrt(withSum / frames) + const errorWithout = Math.sqrt(withoutSum / frames) + return { + frames: scored, + worstStep, + meanStep: stepSum / frames, + worstJerk, + worstSizeStep, + worstTurnStep, + error, + errorWithout, + gain: errorWithout > 1e-9 ? error / errorWithout : error > 1e-9 ? Infinity : 0, + bias: biasSum / frames, + worstOvershoot, + worstLate, + backwards, + worstBackwards, + phantom, + worstLead, + meanLead: leadSum / frames, + meanRadius: radialSum / frames, + kept: Math.hypot(keptX, keptY), + falseDeadFrames, + } +} + +/** The worst reading of every number across a set of devices, which is what a floor has to hold at. */ +export const worstAcross = (devices: readonly Device[], options: Omit): Metrics => { + let worst: Metrics | null = null + for (const dev of devices) { + const run = replay({ ...options, device: dev }) + if (!worst) { + worst = run + continue + } + worst = { + frames: Math.min(worst.frames, run.frames), + worstStep: Math.max(worst.worstStep, run.worstStep), + meanStep: Math.max(worst.meanStep, run.meanStep), + worstJerk: Math.max(worst.worstJerk, run.worstJerk), + worstSizeStep: Math.max(worst.worstSizeStep, run.worstSizeStep), + worstTurnStep: Math.max(worst.worstTurnStep, run.worstTurnStep), + error: Math.max(worst.error, run.error), + errorWithout: Math.max(worst.errorWithout, run.errorWithout), + gain: Math.max(worst.gain, run.gain), + bias: Math.abs(run.bias) > Math.abs(worst.bias) ? run.bias : worst.bias, + worstOvershoot: Math.max(worst.worstOvershoot, run.worstOvershoot), + worstLate: Math.max(worst.worstLate, run.worstLate), + backwards: Math.max(worst.backwards, run.backwards), + worstBackwards: Math.max(worst.worstBackwards, run.worstBackwards), + phantom: Math.max(worst.phantom, run.phantom), + worstLead: Math.max(worst.worstLead, run.worstLead), + meanLead: Math.min(worst.meanLead, run.meanLead), + meanRadius: Math.max(worst.meanRadius, run.meanRadius), + kept: Math.max(worst.kept, run.kept), + falseDeadFrames: Math.max(worst.falseDeadFrames, run.falseDeadFrames), + } + } + if (!worst) throw new Error('worstAcross needs at least one device') + return worst +} + +// --------------------------------------------------------------------------------------------------------- +// Paths. A hand does a small number of things, and each of them breaks a different part of the guess: steady +// motion is where jitter shows, a stop is where overshoot shows, a turn is where a per-axis fit falls apart, +// and a reversal is where extrapolating at all stops being safe. +// --------------------------------------------------------------------------------------------------------- + +const radians = (degrees: number) => (degrees * Math.PI) / 180 + +/** A leg of a path: where it goes over its own lifetime, starting from wherever the previous one left off. + * `at(0)` must be the origin, or the path teleports at the seam. */ +export type Segment = { durationMs: number; at: (u: number) => Vec } + +/** Segments laid end to end, each starting where the last one finished, holding the final position for ever + * after — which is what a hand that has stopped does, and what the score has to be able to ask about. */ +export const piecewise = (segments: readonly Segment[]): Path => { + const starts: number[] = [] + const offsets: Vec[] = [] + let startAt = 0 + let ox = 0 + let oy = 0 + for (const segment of segments) { + starts.push(startAt) + offsets.push({ x: ox, y: oy }) + const end = segment.at(segment.durationMs) + ox += end.x + oy += end.y + startAt += segment.durationMs + } + return (t: number) => { + if (t <= 0 || segments.length === 0) return { x: 0, y: 0 } + let index = segments.length - 1 + while (index > 0 && t < starts[index]!) index-- + const segment = segments[index]! + const local = segment.at(Math.min(t - starts[index]!, segment.durationMs)) + return { x: offsets[index]!.x + local.x, y: offsets[index]!.y + local.y } + } +} + +/** A hand sitting still. */ +export const hold = (durationMs: number): Segment => ({ durationMs, at: () => ({ x: 0, y: 0 }) }) + +/** Constant speed in a straight line. */ +export const glide = (speed: number, angleDeg: number, durationMs: number): Segment => ({ + durationMs, + at: (u: number) => ({ x: speed * Math.cos(radians(angleDeg)) * u, y: speed * Math.sin(radians(angleDeg)) * u }), +}) + +/** Rest to full speed over the segment, on a cosine — a hand starting a movement rather than a step + * function. */ +export const easeUp = (speed: number, angleDeg: number, durationMs: number): Segment => ({ + durationMs, + at: (u: number) => { + const travelled = speed * (u / 2 - (durationMs / (2 * Math.PI)) * Math.sin((Math.PI * u) / durationMs)) + return { x: Math.cos(radians(angleDeg)) * travelled, y: Math.sin(radians(angleDeg)) * travelled } + }, +}) + +/** Full speed to a dead stop over the segment, on a cosine — a hand letting go. */ +export const easeDown = (speed: number, angleDeg: number, durationMs: number): Segment => ({ + durationMs, + at: (u: number) => { + const travelled = speed * (u / 2 + (durationMs / (2 * Math.PI)) * Math.sin((Math.PI * u) / durationMs)) + return { x: Math.cos(radians(angleDeg)) * travelled, y: Math.sin(radians(angleDeg)) * travelled } + }, +}) + +/** A point-to-point move that starts and ends at rest: smoothstep, which is what a hand moving something + * somewhere deliberately looks like. */ +export const smoothMove = (distancePx: number, angleDeg: number, durationMs: number): Segment => ({ + durationMs, + at: (u: number) => { + const s = Math.min(1, Math.max(0, u / durationMs)) + const travelled = distancePx * s * s * (3 - 2 * s) + return { x: Math.cos(radians(angleDeg)) * travelled, y: Math.sin(radians(angleDeg)) * travelled } + }, +}) + +/** Constant speed around a bend of a given radius: the turn a hand actually makes at a corner, as opposed + * to the instantaneous heading change of `glide` into `glide`. */ +export const arc = (speed: number, radiusPx: number, sweepDeg: number, headingDeg: number): Segment => { + const sweep = radians(sweepDeg) + const sign = Math.sign(sweep) || 1 + const heading = radians(headingDeg) + // The centre sits one radius off to the side the hand is turning towards. + const cx = -Math.sin(heading) * radiusPx * sign + const cy = Math.cos(heading) * radiusPx * sign + return { + durationMs: (Math.abs(sweep) * radiusPx) / speed, + at: (u: number) => { + const turned = ((speed * u) / radiusPx) * sign + const cos = Math.cos(turned) + const sin = Math.sin(turned) + // Where the starting point ends up after being swung about that centre. + return { x: cx - cx * cos + cy * sin, y: cy - cx * sin - cy * cos } + }, + } +} + +/** Constant acceleration from rest — the half of a movement a velocity estimator is systematically late on, + * and the half a curvature is supposed to catch. */ +export const accelerate = (accelPxPerMs2: number, angleDeg: number, durationMs: number): Segment => ({ + durationMs, + at: (u: number) => { + const travelled = 0.5 * accelPxPerMs2 * u * u + return { x: Math.cos(radians(angleDeg)) * travelled, y: Math.sin(radians(angleDeg)) * travelled } + }, +}) + +export const steady = + (speed: number, angleDeg: number): Path => + (t: number) => ({ x: speed * Math.cos(radians(angleDeg)) * t, y: speed * Math.sin(radians(angleDeg)) * t }) + +export const shake = + (amplitude: number, hz: number): Path => + (t: number) => ({ x: amplitude * Math.sin((2 * Math.PI * hz * t) / 1000), y: 0 }) + +export const circle = + (r: number, periodMs: number, direction: 1 | -1 = 1): Path => + (t: number) => ({ + x: r * Math.cos((2 * Math.PI * t) / periodMs), + y: direction * r * Math.sin((2 * Math.PI * t) / periodMs), + }) + +/** A lemniscate: the curvature reverses sign smoothly twice a lap, which is the one shape that asks the arc + * extrapolation to change which way it bends without the hand ever slowing or turning back. */ +export const figureEight = + (r: number, periodMs: number): Path => + (t: number) => { + const w = (2 * Math.PI * t) / periodMs + return { x: r * Math.sin(w), y: r * Math.sin(w) * Math.cos(w) } + } + +/** Round and outwards: the radius and the speed both grow, so the turn rate the fit reads has to keep + * falling while the speed keeps rising. */ +export const spiral = + (r0: number, growthPxPerMs: number, periodMs: number): Path => + (t: number) => { + const r = r0 + growthPxPerMs * t + const w = (2 * Math.PI * t) / periodMs + return { x: r * Math.cos(w), y: r * Math.sin(w) } + } + +/** Advancing while weaving: constant forward speed with a sinusoidal wander across it, which is a hand + * tracking something that will not hold still. */ +export const serpentine = + (speed: number, angleDeg: number, amplitude: number, periodMs: number): Path => + (t: number) => { + const along = speed * t + const across = amplitude * Math.sin((2 * Math.PI * t) / periodMs) + const heading = radians(angleDeg) + return { + x: along * Math.cos(heading) - across * Math.sin(heading), + y: along * Math.sin(heading) + across * Math.cos(heading), + } + } + +/** A triangle wave: constant speed with instantaneous reversals, so there is no curvature anywhere to see + * the turn coming and the whole warning is that it happened. */ +export const zigzag = + (speed: number, angleDeg: number, legMs: number): Path => + (t: number) => { + const leg = Math.floor(t / legMs) + const u = t - leg * legMs + const along = leg % 2 === 0 ? speed * u : speed * (legMs - u) + const heading = radians(angleDeg) + return { x: along * Math.cos(heading), y: along * Math.sin(heading) } + } + +/** A hand that throws the view somewhere and lets go: a cruise, then a cosine ease down to a dead stop over + * fallMs. The stop is the whole point — it is the one moment the guess is asked to give back everything it + * is holding, and the moment it is furthest from being able to see it coming. */ +export const flick = (speed: number, angleDeg: number, cruiseMs: number, fallMs: number): Path => { + const ux = Math.cos(radians(angleDeg)) + const uy = Math.sin(radians(angleDeg)) + const travelled = (t: number) => { + if (t <= cruiseMs) return speed * t + const u = Math.min(t - cruiseMs, fallMs) + return speed * (cruiseMs + u / 2 + (fallMs / (2 * Math.PI)) * Math.sin((Math.PI * u) / fallMs)) + } + return (t: number) => ({ x: ux * travelled(t), y: uy * travelled(t) }) +} + +/** Anything, with a hand's tremor on top of it. A hand holding a mouse still is not still, and the wobble is + * the same size as the counts the device reports — which is exactly where a curvature reads it as motion. */ +export const withTremor = + (path: Path, amplitude: number, hz: number): Path => + (t: number) => { + const base = path(t) + const wobble = amplitude * Math.sin((2 * Math.PI * hz * t) / 1000) + // Across both axes at different rates, so it is not a motion the fit can lock onto. + return { x: base.x + wobble, y: base.y + amplitude * 0.7 * Math.sin((2 * Math.PI * hz * 1.31 * t) / 1000 + 1) } + } + +// --------------------------------------------------------------------------------------------------------- +// The scenarios. Every one of them is something a hand does, and each is here because it asks the guess a +// question the others do not. The assertions live in the test files; this list is what both they and the +// scoreboard iterate. +// --------------------------------------------------------------------------------------------------------- + +export type Scenario = { + name: string + path: Path + durationMs: number + scoreAfterMs?: number + /** What this one is asking, in a line, so a failure names the case rather than a number. */ + note: string +} + +const scenario = (name: string, note: string, path: Path, durationMs: number, scoreAfterMs?: number): Scenario => ({ + name, + note, + path, + durationMs, + scoreAfterMs, +}) + +export const SCENARIOS: readonly Scenario[] = [ + scenario('steady fast', 'a hand crossing the pad at speed, off-axis so the counts zig-zag', steady(0.8, 30), 2500), + scenario('steady slow', 'slow enough that the fit is mostly reading quantisation', steady(0.15, 30), 2500), + scenario('crawl', 'below the speed gate, where the only honest answer is no lead at all', steady(0.06, 30), 4000), + scenario( + 'steady on-axis', + 'the same drag with one axis idle, which is where a 2D fit can go wrong', + steady(0.8, 0), + 2500, + ), + scenario('whip', 'far past what the cap allows, so what is being judged is the ceiling', steady(4, 15), 2000), + scenario( + 'ramp up', + 'constant acceleration from rest: the half of a movement a velocity estimator is late on', + piecewise([hold(300), accelerate(0.0025, 25, 1200), glide(3, 25, 600)]), + 2100, + 400, + ), + scenario( + 'flick, snapped stop', + 'thrown and let go in 80ms — the least warning a stop ever gives', + flick(2.6, 20, 800, 80), + 1180, + ), + scenario('flick, eased stop', 'letting go of a long drag over 250ms', flick(2.6, 20, 800, 250), 1350), + scenario( + 'abrupt stop', + 'full speed to nothing between two samples, which no fit can see coming', + piecewise([glide(2, 40, 900), hold(500)]), + 1400, + ), + scenario( + 'abrupt start', + 'nothing to full speed between two samples: the guess has to arrive without inventing motion first', + piecewise([hold(500), glide(2, 40, 900)]), + 1400, + 520, + ), + scenario( + 'stop and go', + 'repeated bursts with pauses between them, so every stop is followed by a start', + piecewise([ + hold(200), + ...Array.from({ length: 6 }, () => [ + easeUp(1.6, 15, 60), + glide(1.6, 15, 140), + easeDown(1.6, 15, 60), + hold(150), + ]).flat(), + ]), + 2700, + 300, + ), + scenario( + 'point to point', + 'a deliberate move that starts and ends at rest, twice over', + piecewise([ + hold(200), + smoothMove(400, 10, 500), + hold(300), + smoothMove(250, 190, 400), + hold(300), + smoothMove(600, 70, 700), + hold(300), + ]), + 2700, + 300, + ), + scenario( + 'sharp corner', + 'a right angle at full speed with no slowing: the heading inverts on one axis and holds on the other', + piecewise([glide(1.2, 0, 900), glide(1.2, 90, 900)]), + 1800, + ), + scenario( + 'rounded corner', + 'the same corner taken round a radius, which is what a hand actually does', + piecewise([glide(1.2, 0, 900), arc(1.2, 140, 90, 0), glide(1.2, 90, 700)]), + 1900, + ), + scenario('circle', 'a hand going round, where a per-axis fit reads a stop twice a lap', circle(120, 400), 3000), + scenario( + 'circle, tight and fast', + 'the same, wound up until the turn rate is most of the answer', + circle(60, 260), + 2500, + ), + scenario( + 'stir', + 'a small circle fast enough that a whole horizon of turning is past a right angle', + circle(25, 110), + 2500, + ), + scenario('circle, reversed', 'and the other way round, so a turn sign error cannot hide', circle(120, 400, -1), 3000), + scenario( + 'figure eight', + 'curvature that reverses smoothly, with the hand neither slowing nor turning back', + figureEight(150, 900), + 3200, + ), + scenario( + 'spiral', + 'turning and speeding up at once, which are the two things read off the same window', + spiral(30, 0.09, 700), + 3000, + ), + scenario( + 'serpentine', + 'advancing while weaving, the way a hand tracks something moving', + serpentine(0.7, 20, 60, 500), + 3000, + ), + scenario( + 'zigzag', + 'constant speed with instantaneous reversals: no curvature warns of the turn', + zigzag(0.9, 35, 260), + 3000, + ), + scenario('shake, 8Hz', 'a hand settling on a target — it turns round before the horizon is out', shake(20, 8), 2500), + scenario( + 'shake, fast and small', + 'the same, quicker and tighter, which is the worst case for it', + shake(10, 12), + 2500, + ), + scenario( + 'overshoot and correct', + 'past the target at speed, then back onto it slowly: a real reversal after a real flick', + piecewise([ + hold(200), + easeUp(1.8, 0, 120), + glide(1.8, 0, 300), + easeDown(1.8, 0, 160), + hold(120), + smoothMove(70, 180, 400), + hold(400), + ]), + 1900, + 300, + ), + scenario( + 'target acquire', + 'the two phases of aiming: most of the distance fast, the last of it in micro-adjustments', + piecewise([ + hold(200), + easeUp(2.2, 12, 100), + glide(2.2, 12, 260), + easeDown(2.2, 12, 140), + hold(80), + smoothMove(24, 12, 220), + hold(60), + smoothMove(9, 192, 180), + hold(60), + smoothMove(4, 12, 160), + hold(500), + ]), + 2100, + 300, + ), + scenario( + 'wide slow arc', + 'a turn gentle enough to sit near the floor the turn rate is believed against', + piecewise([glide(0.35, 0, 400), arc(0.35, 900, 120, 0), glide(0.35, 120, 400)]), + 3300, + ), + scenario( + 'double flick', + 'a second throw before the first has finished stopping, which is where a brake read early has to let go', + piecewise([ + hold(200), + easeUp(2.4, 30, 90), + glide(2.4, 30, 220), + easeDown(2.4, 30, 110), + hold(70), + easeUp(2.4, 45, 90), + glide(2.4, 45, 260), + easeDown(2.4, 45, 140), + hold(400), + ]), + 1900, + 300, + ), + scenario( + 'tremor on a drag', + 'a steady hand is not a still one, and the wobble is the size of the counts underneath it', + withTremor(steady(0.35, 25), 1.2, 9), + 2500, + ), + scenario( + 'tremor at rest', + 'a hand resting on the mouse, which must not be led anywhere at all', + withTremor(steady(0.004, 25), 0.9, 7), + 2500, + ), +] diff --git a/library/src/__tests__/predictor-devices.test.ts b/library/src/__tests__/predictor-devices.test.ts new file mode 100644 index 0000000..58c39c6 --- /dev/null +++ b/library/src/__tests__/predictor-devices.test.ts @@ -0,0 +1,284 @@ +import { describe, expect, test } from 'bun:test' + +import { + ALL_DEVICES, + BLUETOOTH, + COARSE, + flick, + HZ_120, + HZ_144, + HZ_30, + HZ_60, + replay, + SPARSE, + steady, + TRACKPAD, + USB_125, + WIRED, +} from './harness.js' + +// The two axes that are not the hand: what the device does, and what the display does. +// +// The same hand reads completely differently through a wired mouse and through one on a radio, and the whole +// difficulty of the prediction is that there is no telling them apart in advance — the guess has to be +// steady on a device reporting a thousand times a second and on one reporting twenty, without being told +// which it is on. The display axis is the same question about the other end: the lead is a count of +// refreshes because the delay it covers is a count of refreshes, so the same setting has to mean half as +// long on a 120Hz screen and the figures have to hold there too. + +describe('every device', () => { + // How much the size of the guess may jump in one frame, on a hand that is doing nothing new. Measured, + // then given room. The ordering is the interesting part: it tracks how often the device reports, and the + // one device that breaks the ordering breaks it for a reason worth knowing (see below). + const STEADY: Record = { + wired: { fast: 0.05, slow: 0.09, gain: 0.06 }, + '8KHz mouse': { fast: 0.04, slow: 0.07, gain: 0.05 }, + '500Hz mouse': { fast: 0.05, slow: 0.09, gain: 0.07 }, + '125Hz mouse': { fast: 0.2, slow: 0.18, gain: 0.14 }, + trackpad: { fast: 0.23, slow: 0.18, gain: 0.17 }, + bluetooth: { fast: 0.73, slow: 0.25, gain: 0.28 }, + 'slow radio': { fast: 0.64, slow: 0.29, gain: 0.36 }, + 'low-DPI mouse': { fast: 5.9, slow: 3, gain: 0.4 }, + sparse: { fast: 0.95, slow: 0.29, gain: 0.29 }, + } + + for (const device of ALL_DEVICES) { + test(`${device.name}: a steady drag gives a steady lead`, () => { + const limits = STEADY[device.name]! + const fast = replay({ path: steady(0.8, 30), device, durationMs: 2500 }) + expect(fast.worstSizeStep).toBeLessThan(limits.fast) + expect(fast.gain).toBeLessThan(limits.gain) + // Slow motion is where a shaky guess shows most: the lead is a pixel and a half, so a tenth of a pixel + // of wobble is a tenth of the whole thing. + const slow = replay({ path: steady(0.15, 30), device, durationMs: 2500 }) + expect(slow.worstSizeStep).toBeLessThan(limits.slow) + }) + } + + test('a crawl is led by nothing at all', () => { + // Below the gate there is nothing in the samples a fit can tell from quantisation, and a caller applying + // only the growth keeps every upward wobble of it for as long as the drag lasts. Exactly nothing, not + // nearly nothing — the low-DPI mouse is the exception and it is a bug rather than a tolerance, so it is + // held on its own below. + for (const device of ALL_DEVICES) { + if (device === COARSE) continue + const { worstLead, kept } = replay({ path: steady(0.06, 30), device, durationMs: 4000 }) + expect(worstLead).toBe(0) + expect(kept).toBe(0) + } + }) + + test('a stalling frame clock does not throw the lead about', () => { + // A three-frame stall at 60Hz is 50ms, which is close enough to a plausible refresh interval to be taken + // for one. Believing it moves the horizon by two frames and hands over pixels of lead on the frame after + // every stall. + const ALLOWED: Record = { + wired: 0.45, + '8KHz mouse': 0.35, + '500Hz mouse': 0.4, + '125Hz mouse': 0.25, + trackpad: 0.55, + bluetooth: 1.2, + 'slow radio': 1.9, + 'low-DPI mouse': 12, + sparse: 1.7, + } + for (const device of ALL_DEVICES) { + const { worstSizeStep } = replay({ path: steady(0.8, 30), device, durationMs: 3000, hitchEvery: 20 }) + expect(worstSizeStep).toBeLessThan(ALLOWED[device.name]!) + } + }) + + test('events handed over late are covered, up to a point', () => { + // The horizon starts at the newest sample rather than at the frame, so a machine that hands its events + // over late has to be led further — and that is a real setting the library reads off the clock, not an + // assumption. What it may not do is follow it anywhere: past about two frames of age the sample is not + // evidence about where the hand is now, it is history, and extrapolating a whole extra frame from it + // is how a stall turns into a lurch. + const leads = [2, 15, 30, 60].map(deliveryLagMs => ({ + deliveryLagMs, + run: replay({ path: steady(0.8, 30), device: WIRED, durationMs: 2500, deliveryLagMs }), + })) + // It follows the lag while the lag is plausible... + expect(leads[1]!.run.meanLead).toBeGreaterThan(leads[0]!.run.meanLead * 1.4) + expect(leads[2]!.run.meanLead).toBeGreaterThan(leads[1]!.run.meanLead * 1.3) + // ...and then stops, because a sample two frames old is not news. + expect(leads[3]!.run.meanLead).toBeLessThan(42) + for (const { run } of leads) { + expect(run.worstSizeStep).toBeLessThan(0.3) + expect(run.gain).toBeLessThan(0.45) + } + + // And a stop is still a stop when everything about it arrives late. + for (const device of [WIRED, TRACKPAD]) { + const stop = replay({ path: flick(2.6, 20, 800, 150), device, durationMs: 1350, deliveryLagMs: 30 }) + expect(stop.worstOvershoot).toBeLessThan(40) + expect(stop.phantom).toBeLessThan(30) + } + }) + + test('a device nobody would call fast still gets a guess worth having', () => { + // One report every 25ms with 4ms of wander: most frames see nothing at all and the window holds two or + // three samples. It cannot be as good as a wired mouse and it must not be useless. + for (const path of [steady(0.8, 30), flick(2.6, 20, 800, 150)]) { + const metrics = replay({ path, device: SPARSE, durationMs: 2500 }) + expect(metrics.gain).toBeLessThan(0.45) + } + }) +}) + +describe('a device whose counts are wider than a pixel', () => { + // **A known weak spot, written down as numbers so that fixing it shows up as numbers.** + // + // Every noise floor in the fit is written in units of one count and one count is assumed to be about one + // pixel — `QUANTISATION_PX`. That holds for a mouse at default sensitivity, and it does not hold for a + // low-DPI one, a page at a device pixel ratio other than 1, or an OS scaling the counts on the way + // through: this device reports four pixels a step, so the bend, the turn and the trend all sit four times + // higher above floors that have not moved, and noise gets believed as motion. + // + // What that costs, against the same hand on a 125Hz mouse that reports just as often: the lead's size + // jumps thirty times as far in a frame, it swings through 28° of heading on a path with no turn in it at + // all, and a caller keeping the growth accumulates nine times as much. None of that is a device problem — + // it is one constant that does not know what a count is worth. + // + // These are ceilings on a known-bad case, not a floor anyone is proud of. Scaling the floors by the count + // size — from the deltas the device actually reports, since nothing else knows — should collapse all four. + test('is the case the noise floors do not fit', () => { + const fast = replay({ path: steady(0.8, 30), device: COARSE, durationMs: 2500 }) + expect(fast.worstSizeStep).toBeLessThan(5.9) + expect(fast.worstTurnStep).toBeLessThan(35) + expect(fast.kept).toBeLessThan(250) + + const slow = replay({ path: steady(0.15, 30), device: COARSE, durationMs: 2500 }) + expect(slow.worstTurnStep).toBeLessThan(175) + + // And the same constant is what lets a crawl through the speed gate: at 0.06px/ms the hand covers less + // than a count a frame, so two counts landing close together read as a hand going six times faster than + // it is. Every other device is led by exactly nothing here. + const crawl = replay({ path: steady(0.06, 30), device: COARSE, durationMs: 4000 }) + expect(crawl.worstLead).toBeLessThan(8) + expect(crawl.kept).toBeLessThan(50) + }) + + test('a mouse reporting just as often but in single pixels is steady', () => { + // The control for the case above: same period, same burst, same wander, one-pixel counts. + const fine = replay({ path: steady(0.8, 30), device: USB_125, durationMs: 2500 }) + expect(fine.worstSizeStep).toBeLessThan(0.2) + expect(fine.worstTurnStep).toBeLessThan(1) + expect(fine.kept).toBeLessThan(25) + }) +}) + +describe('every refresh rate', () => { + const RATES = [ + { name: '30Hz', frameMs: HZ_30 }, + { name: '60Hz', frameMs: HZ_60 }, + { name: '120Hz', frameMs: HZ_120 }, + { name: '144Hz', frameMs: HZ_144 }, + ] as const + + test('the lead is a count of refreshes, so a faster display leads for less time', () => { + // Not a duration: the delay being covered is a count of refreshes, so the same setting has to mean half + // as long on a screen running twice as fast. What is left at the short end is the age of the newest + // sample, which is a property of the machine rather than of the display and does not shrink with it. + const leads = RATES.map( + rate => replay({ path: steady(0.8, 30), device: WIRED, durationMs: 2500, frameMs: rate.frameMs }).meanLead, + ) + for (let i = 1; i < leads.length; i++) expect(leads[i]!).toBeLessThan(leads[i - 1]!) + // A refresh at 30Hz is worth about 13px of this hand, and at 144Hz about 5.5. + expect(leads[0]!).toBeGreaterThan(24) + expect(leads[3]!).toBeLessThan(10) + }) + + for (const rate of RATES) { + test(`${rate.name}: steady motion is steady and stopping is not overshot`, () => { + // Both figures are held at every rate rather than at 60Hz alone, because everything the ramp and the + // horizon do is written in milliseconds and a shorter frame is a different number of them. + const LIMITS: Record = { + '30Hz': { size: 1, gain: 0.19, over: 33, still: 11 }, + '60Hz': { size: 0.75, gain: 0.28, over: 38, still: 9 }, + '120Hz': { size: 0.5, gain: 0.43, over: 30, still: 9 }, + '144Hz': { size: 0.62, gain: 0.45, over: 29, still: 9 }, + } + const limits = LIMITS[rate.name]! + for (const device of [WIRED, TRACKPAD, BLUETOOTH]) { + const drag = replay({ path: steady(0.8, 30), device, durationMs: 2500, frameMs: rate.frameMs }) + expect(drag.worstSizeStep).toBeLessThan(limits.size) + expect(drag.gain).toBeLessThan(limits.gain) + + const stop = replay({ path: flick(2.6, 20, 800, 150), device, durationMs: 1350, frameMs: rate.frameMs }) + expect(stop.worstOvershoot).toBeLessThan(limits.over) + expect(stop.phantom).toBeLessThan(limits.still) + } + }) + } +}) + +describe('the caller settings', () => { + test('leadFrames scales the guess, and zero means almost none of it', () => { + const leads = [0, 1, 2, 3].map( + leadFrames => replay({ path: steady(0.8, 30), device: WIRED, durationMs: 2500, leadFrames }).meanLead, + ) + // Zero still covers the time the samples spent getting here, which is a couple of milliseconds on this + // rig — a few pixels, not a frame's worth. + expect(leads[0]!).toBeLessThan(4) + // And each further refresh is worth about another frame of this hand's travel, near enough. + for (let i = 1; i < leads.length; i++) { + const added = leads[i]! - leads[i - 1]! + expect(added).toBeGreaterThan(8) + expect(added).toBeLessThan(16) + } + }) + + test('a longer ramp buys steadiness and pays for it in overshoot', () => { + // The ramp is the caller's feel setting, and the trade it makes has a direction: smoothing the guess + // harder means arriving later at a stop, which is overshoot and picture moving under a still hand. If a + // change ever makes both ends better at once, something else has moved and this test should be the one + // to say so. + const smooth = (decayMs: number) => replay({ path: steady(0.8, 30), device: TRACKPAD, durationMs: 2500, decayMs }) + const stop = (decayMs: number) => + replay({ path: flick(2.6, 20, 800, 150), device: TRACKPAD, durationMs: 1350, decayMs }) + + expect(smooth(5).worstSizeStep).toBeGreaterThan(smooth(30).worstSizeStep * 2) + expect(stop(5).worstOvershoot).toBeLessThan(stop(50).worstOvershoot) + expect(stop(5).phantom).toBeLessThan(stop(50).phantom) + }) + + test('maxLeadPx is exact, and applies to the vector rather than to each axis', () => { + // A whip on a diagonal: uncapped this asks for hundreds of pixels. Capping per axis would let a diagonal + // through at √2 times the figure the caller asked for. + for (const cap of [5, 20, 60]) { + let worst = 0 + let worstAxis = 0 + replay({ + path: steady(4, 15), + device: WIRED, + durationMs: 1500, + maxLeadPx: cap, + onFrame: frame => { + worst = Math.max(worst, Math.hypot(frame.lead.x, frame.lead.y)) + worstAxis = Math.max(worstAxis, Math.abs(frame.lead.x), Math.abs(frame.lead.y)) + }, + }) + expect(worst).toBeLessThanOrEqual(cap + 1e-9) + expect(worst).toBeGreaterThan(cap * 0.99) + expect(worstAxis).toBeLessThanOrEqual(cap + 1e-9) + } + }) + + test('a ceiling nobody reaches changes nothing', () => { + // The cap is a feel setting, not a safety rail. Raising it past what the samples ask for has to leave + // the guess exactly as it was, or it is doing something other than capping. + // 1.5px/ms asks for about 29px, so none of these ceilings is ever in the way and all three runs have to + // be the same run. Bit for bit: a cap that changes an answer it was never reached by is a cap being + // applied to something other than the answer. + const runs = [60, 100, 1e6].map(maxLeadPx => + replay({ path: steady(1.5, 15), device: WIRED, durationMs: 1500, maxLeadPx }), + ) + for (const run of runs) { + expect(run.worstLead).toBe(runs[0]!.worstLead) + expect(run.error).toBe(runs[0]!.error) + } + expect(runs[0]!.worstLead).toBeLessThan(60) + }) +}) diff --git a/library/src/__tests__/predictor-invariants.test.ts b/library/src/__tests__/predictor-invariants.test.ts new file mode 100644 index 0000000..e1c0721 --- /dev/null +++ b/library/src/__tests__/predictor-invariants.test.ts @@ -0,0 +1,312 @@ +import { describe, expect, test } from 'bun:test' + +import { createPointerBuffer } from '../pointer.js' +import { createPointerPredictor, DEFAULT_DECAY_MS, DEFAULT_LEAD_FRAMES, DEFAULT_MAX_LEAD_PX } from '../predictor.js' +import { ALL_DEVICES, circle, FRAME, flick, replay, SCENARIOS, steady, WIRED, type Path } from './harness.js' + +// The things that have to be true of every answer, whatever the hand did and whatever reported it, and the +// inputs that are not a hand at all. +// +// The measured floors in the other files say how good the guess is. These say what it may never do — point +// somewhere impossible, return something that is not a number, depend on which way round the screen is, or +// blow up on a device having a bad moment. A floor can be argued about; none of these can. + +const uncapped = { leadFrames: DEFAULT_LEAD_FRAMES, decayMs: DEFAULT_DECAY_MS, maxLeadPx: Number.MAX_SAFE_INTEGER } +const shipped = { leadFrames: DEFAULT_LEAD_FRAMES, decayMs: DEFAULT_DECAY_MS, maxLeadPx: DEFAULT_MAX_LEAD_PX } + +/** The same hand, reflected in the X axis. */ +const mirror = + (path: Path): Path => + (t: number) => { + const at = path(t) + return { x: at.x, y: -at.y } + } + +const push = (predictor: ReturnType, samples: [number, number, number][]) => { + for (const [t, dx, dy] of samples) predictor.push(t, dx, dy) +} + +describe('what every answer has to be', () => { + test('a number, inside the cap, on every shape and every device', () => { + // The whole matrix, every frame — including the frames while the window is still filling, which the + // scored figures elsewhere deliberately skip. + for (const scenario of SCENARIOS) { + for (const device of ALL_DEVICES) { + let frames = 0 + replay({ + path: scenario.path, + device, + durationMs: scenario.durationMs, + maxLeadPx: 40, + onFrame: frame => { + frames++ + const { x, y, live } = frame.lead + if (!Number.isFinite(x) || !Number.isFinite(y) || !Number.isFinite(live)) { + throw new Error(`${scenario.name} on ${device.name}: lead ${x},${y} live ${live}`) + } + if (Math.hypot(x, y) > 40 + 1e-9) { + throw new Error(`${scenario.name} on ${device.name}: ${Math.hypot(x, y)}px past a 40px cap`) + } + if (live < 0 || live > 1) throw new Error(`${scenario.name} on ${device.name}: live ${live}`) + }, + }) + expect(frames).toBeGreaterThan(10) + } + } + }) + + test('the same run twice is the same run', () => { + // Everything here is deterministic on purpose. A figure that wanders between runs is a figure nobody can + // tune against, and the first sign of state leaking between predictors would be exactly that. + for (const scenario of SCENARIOS.slice(0, 8)) { + const once = replay({ path: scenario.path, device: WIRED, durationMs: scenario.durationMs }) + const twice = replay({ path: scenario.path, device: WIRED, durationMs: scenario.durationMs }) + expect(twice).toEqual(once) + } + }) + + test('two predictors do not see each other', () => { + // The window and the eased lead are per-instance state, and a caller comparing two settings side by side + // has to be able to trust that. Feeding one a whole flick must leave the other exactly as it was. + const alone = createPointerPredictor() + const alongside = createPointerPredictor() + const noisy = createPointerPredictor() + + let now = 1000 + const results: number[][] = [[], []] + for (let f = 0; f < 60; f++) { + for (let s = 0; s < 8; s++) { + const at = now - 13 - FRAME + (s * FRAME) / 8 + alone.push(at, 4, 1) + alongside.push(at, 4, 1) + // A completely different hand, on its own instance. + noisy.push(at, -19, 7) + } + noisy.update({ nowMs: now, deltaMs: FRAME, ...shipped }) + results[0]!.push(alone.update({ nowMs: now, deltaMs: FRAME, ...shipped }).x) + results[1]!.push(alongside.update({ nowMs: now, deltaMs: FRAME, ...shipped }).x) + now += FRAME + } + expect(results[1]).toEqual(results[0]!) + expect(Math.max(...results[0]!)).toBeGreaterThan(5) + }) + + test('mirroring the hand mirrors the guess', () => { + // Nothing in the fit may prefer one side of the screen. A turn read with the wrong sign, or a floor + // subtracted rather than believed, tends to show up here first — the two runs stop being reflections. + for (const path of [steady(0.8, 30), circle(120, 400), flick(2.6, 20, 800, 150)]) { + for (const device of ALL_DEVICES) { + const upright = replay({ path, device, durationMs: 1600 }) + const flipped = replay({ path: mirror(path), device, durationMs: 1600 }) + expect(flipped.worstLead).toBeCloseTo(upright.worstLead, 9) + expect(flipped.error).toBeCloseTo(upright.error, 9) + expect(flipped.worstOvershoot).toBeCloseTo(upright.worstOvershoot, 9) + } + } + }) + + test('the same drag at any angle is the same drag', () => { + // Off an axis the two axes cross their count boundaries at different moments and the path arrives as a + // zig-zag staircase, so this cannot be exact — but it has to be close, or the guess has a favourite + // direction and a hand sweeping round finds it. + const runs = [0, 22.5, 45, 67.5, 90, 137, -113].map(angle => + replay({ path: steady(0.8, angle), device: WIRED, durationMs: 2000 }), + ) + const leads = runs.map(run => run.meanLead) + expect(Math.max(...leads) - Math.min(...leads)).toBeLessThan(0.6) + expect(Math.max(...runs.map(run => run.worstSizeStep))).toBeLessThan(0.12) + }) + + test('the returned object is the same object every frame', () => { + // Documented, and worth holding: a caller that keeps the reference gets the next frame's answer for + // free, which is a bug that hides for a long time. + const predictor = createPointerPredictor() + const first = predictor.update({ nowMs: 1000, deltaMs: FRAME, ...shipped }) + const second = predictor.update({ nowMs: 1016.7, deltaMs: FRAME, ...shipped }) + expect(second).toBe(first) + }) +}) + +describe('inputs that are not a hand', () => { + test('nothing at all', () => { + const predictor = createPointerPredictor() + let lead = predictor.update({ nowMs: 1000, deltaMs: FRAME, ...shipped }) + expect(lead.x).toBe(0) + expect(lead.y).toBe(0) + for (let f = 0; f < 30; f++) { + lead = predictor.update({ nowMs: 1000 + f * FRAME, deltaMs: FRAME, ...shipped }) + } + expect(lead.x).toBe(0) + expect(lead.live).toBe(0) + }) + + test('one sample, which is a span of nothing', () => { + const predictor = createPointerPredictor() + predictor.push(1000, 40, 40) + const lead = predictor.update({ nowMs: 1013, deltaMs: FRAME, ...uncapped }) + expect(lead.x).toBe(0) + expect(lead.y).toBe(0) + }) + + test('a pointer-lock recentre, which is one absurd delta', () => { + // Clamping these belongs on the way in — `createPointerBuffer`'s `maxDeltaPx` — but a caller who has not + // set one must still not be handed a lead of thousands of pixels, and the cap is what stands there. + const predictor = createPointerPredictor() + push(predictor, [ + [1000, 2, 0], + [1004, 2, 0], + [1008, 4000, 3000], + [1012, 2, 0], + ]) + const lead = predictor.update({ nowMs: 1013, deltaMs: FRAME, ...shipped }) + expect(Number.isFinite(lead.x)).toBe(true) + expect(Math.hypot(lead.x, lead.y)).toBeLessThanOrEqual(DEFAULT_MAX_LEAD_PX + 1e-9) + }) + + test('two frames landing on the same clock reading', () => { + // A frame loop that reads the clock twice, or a browser handing over two callbacks in one refresh. The + // ramp is an exponential of the interval, so zero is the one value it cannot be given carelessly. + const predictor = createPointerPredictor() + for (let s = 0; s < 20; s++) predictor.push(1000 + s, 3, 1) + const first = predictor.update({ nowMs: 1020, deltaMs: FRAME, ...shipped }) + const held = { x: first.x, y: first.y } + const again = predictor.update({ nowMs: 1020, deltaMs: 0, ...shipped }) + expect(Number.isFinite(again.x)).toBe(true) + expect(again.x).toBeCloseTo(held.x, 9) + expect(again.y).toBeCloseTo(held.y, 9) + }) + + test('a frame after the tab came back', () => { + // Seconds of nothing, then a frame claiming all of it. Nothing about the last motion may survive that. + const predictor = createPointerPredictor() + for (let s = 0; s < 40; s++) predictor.push(1000 + s, 5, 5) + expect(predictor.update({ nowMs: 1040, deltaMs: FRAME, ...shipped }).x).toBeGreaterThan(0) + const back = predictor.update({ nowMs: 9000, deltaMs: 7960, ...shipped }) + expect(Number.isFinite(back.x)).toBe(true) + expect(back.x).toBe(0) + expect(back.live).toBe(0) + }) + + test('a session that has been running for eleven days', () => { + // The fit centres its positions on the newest sample precisely so the normal equations stay conditioned + // however long the clock has been running. A timestamp of a billion milliseconds squared is 1e18, which + // is where a fit that did not centre would come apart. + const predictor = createPointerPredictor() + let now = 1e9 + let lead = 0 + for (let f = 0; f < 40; f++) { + for (let s = 1; s <= 16; s++) predictor.push(now - 13 - FRAME + (s * FRAME) / 16, 0.75, 0.25) + lead = predictor.update({ nowMs: now, deltaMs: FRAME, ...shipped }).x + now += FRAME + } + // The same hand at the start of a session leads by the same amount. + const fresh = createPointerPredictor() + let early = 1000 + let freshLead = 0 + for (let f = 0; f < 40; f++) { + for (let s = 1; s <= 16; s++) fresh.push(early - 13 - FRAME + (s * FRAME) / 16, 0.75, 0.25) + freshLead = fresh.update({ nowMs: early, deltaMs: FRAME, ...shipped }).x + early += FRAME + } + expect(lead).toBeCloseTo(freshLead, 6) + expect(lead).toBeGreaterThan(5) + }) + + test('more samples in one frame than the window can hold', () => { + // A 8KHz mouse and a long frame, or a device waking up and delivering a backlog. The oldest fall off the + // end, which costs a fit nothing because the window is shorter than the buffer. + const predictor = createPointerPredictor() + for (let s = 0; s < 400; s++) predictor.push(1000 + s * 0.5, 0.4, 0) + const lead = predictor.update({ nowMs: 1200, deltaMs: FRAME, ...shipped }) + expect(Number.isFinite(lead.x)).toBe(true) + expect(lead.x).toBeGreaterThan(0) + expect(lead.x).toBeLessThanOrEqual(DEFAULT_MAX_LEAD_PX) + }) + + test('samples that arrive out of order', () => { + const predictor = createPointerPredictor() + for (let i = 0; i < 40; i++) predictor.push(1000 - i * 0.5, 5, 0) + const lead = predictor.update({ nowMs: 1013, deltaMs: FRAME, ...uncapped }) + expect(Number.isFinite(lead.x)).toBe(true) + expect(Math.abs(lead.x)).toBeLessThan(100) + }) + + test('a hand that reports movement of nothing', () => { + // Some devices report an event per poll whether or not the hand moved. All the timestamps in the world + // do not make that motion. + const predictor = createPointerPredictor() + for (let s = 0; s < 60; s++) predictor.push(1000 + s, 0, 0) + const lead = predictor.update({ nowMs: 1060, deltaMs: FRAME, ...uncapped }) + expect(lead.x).toBe(0) + expect(lead.y).toBe(0) + }) + + test('a ceiling given the wrong way round', () => { + // Documented as the magnitude, since a negative ceiling has no other sensible reading and silently + // handing back nothing at all would be worse. + const predictor = createPointerPredictor() + for (let s = 0; s < 40; s++) predictor.push(1000 + s, 6, 0) + const lead = predictor.update({ nowMs: 1040, deltaMs: FRAME, leadFrames: 1, decayMs: 20, maxLeadPx: -25 }) + expect(lead.x).toBeGreaterThan(0) + expect(lead.x).toBeLessThanOrEqual(25 + 1e-9) + }) + + test('a reset leaves nothing of the session behind it', () => { + // Whatever held the lead — a menu, an unlocked pointer, a cutscene — the next frame has to start from + // nothing rather than fit a curve through motion from before the gap. Not approximately nothing: the + // same predictor after a reset has to answer exactly as a new one does. + const used = createPointerPredictor() + let now = 1000 + for (let f = 0; f < 40; f++) { + for (let s = 1; s <= 16; s++) used.push(now - 13 - FRAME + (s * FRAME) / 16, 2, -1) + used.update({ nowMs: now, deltaMs: FRAME, ...shipped }) + now += FRAME + } + used.reset() + expect(used.update({ nowMs: now, deltaMs: FRAME, ...shipped }).x).toBe(0) + + const fresh = createPointerPredictor() + let freshNow = now + FRAME + let usedNow = now + FRAME + for (let f = 0; f < 30; f++) { + for (let s = 1; s <= 16; s++) { + used.push(usedNow - 13 - FRAME + (s * FRAME) / 16, 0.9, 0.3) + fresh.push(freshNow - 13 - FRAME + (s * FRAME) / 16, 0.9, 0.3) + } + const a = used.update({ nowMs: usedNow, deltaMs: FRAME, ...shipped }) + const b = fresh.update({ nowMs: freshNow, deltaMs: FRAME, ...shipped }) + expect(a.x).toBeCloseTo(b.x, 9) + expect(a.y).toBeCloseTo(b.y, 9) + expect(a.live).toBeCloseTo(b.live, 9) + usedNow += FRAME + freshNow += FRAME + } + }) +}) + +describe('fed from the buffer rather than by hand', () => { + test('pushFrame and push agree, sample for sample', () => { + // The two ends of the library are meant to fit together without the caller unpacking anything, and every + // figure in these files was measured through `push`. If the frame path lost or reordered a sample, none + // of them would be measuring what ships. + const buffered = createPointerPredictor() + const direct = createPointerPredictor() + const buffer = createPointerBuffer() + + let now = 1000 + for (let f = 0; f < 40; f++) { + for (let s = 1; s <= 5; s++) { + const timeStamp = now - 13 - FRAME + (s * FRAME) / 5 + buffer.handle({ movementX: 3, movementY: -1, timeStamp }) + direct.push(timeStamp, 3, -1) + } + buffered.pushFrame(buffer.read(now)) + const a = buffered.update({ nowMs: now, deltaMs: FRAME, ...shipped }) + const b = direct.update({ nowMs: now, deltaMs: FRAME, ...shipped }) + expect(a.x).toBe(b.x) + expect(a.y).toBe(b.y) + expect(a.live).toBe(b.live) + now += FRAME + } + }) +}) diff --git a/library/src/__tests__/predictor-shapes.test.ts b/library/src/__tests__/predictor-shapes.test.ts new file mode 100644 index 0000000..9d8c66c --- /dev/null +++ b/library/src/__tests__/predictor-shapes.test.ts @@ -0,0 +1,186 @@ +import { describe, expect, test } from 'bun:test' + +import { DEVICES, replay, SCENARIOS, worstAcross, type Metrics } from './harness.js' + +// Every shape a hand makes, scored on every device, against a floor that was measured rather than hoped for. +// +// The point of this file is that the prediction can be changed without anyone having to sit at a desk with a +// mouse: a shape that got shakier, later, or further past the hand fails here by name. Each row is the worst +// reading across the four devices, so a change that only breaks the trackpad still fails — `bun run bench` +// then says which device and which column moved. +// +// **These are floors, not targets.** Every figure was read off the rig and given about a quarter more room, +// so it fails on a real regression rather than on rounding. A change that improves one should tighten it in +// the same commit, or the floor stops being a floor and the next regression walks straight under it. A +// change that trades one column for another — a stop seen sooner is always lead given up somewhere else — +// should move both numbers and say so. +// +// What each column means is written next to it in `harness.ts`. The short version: +// +// gain error against the error of not guessing at all. Under 1 is worth having. +// size how much the size of the guess may jump in one frame. Jitter, on the eye's terms. +// over how far past the hand the guess may ever sit. Every pixel of it has to be handed back. +// late how far behind it may sit. This is the lateness the library exists to remove. +// kick the worst single frame of screen motion against the way the hand is going. +// back all of that added up over the run. +// still screen motion over frames where the hand is not moving at all. +// lead how far out the guess may ever be. +type Limits = { + gain: number + size: number + over: number + late: number + kick: number + back: number + still: number + lead: number +} + +const LIMITS: Record = { + // Straight lines. Nothing here is hard to predict, so these are almost entirely about steadiness: the hand + // is doing nothing new, and every pixel the guess moves by is invented. + 'steady fast': { gain: 0.35, size: 0.75, over: 15, late: 25, kick: 0, back: 0, still: 0, lead: 30 }, + 'steady slow': { gain: 0.76, size: 0.3, over: 0, late: 8.5, kick: 0.1, back: 1, still: 0, lead: 3 }, + // Below the speed gate there is no lead at all, so this one is exact rather than generous: a single pixel + // of guess here is quantisation being mistaken for motion, and a caller keeping the growth keeps it. + crawl: { gain: 1.06, size: 0, over: 0, late: 5.5, kick: 0, back: 0, still: 0, lead: 0 }, + 'steady on-axis': { gain: 0.35, size: 0.8, over: 15, late: 25, kick: 0, back: 0, still: 0, lead: 30 }, + // Far past the cap, so what is being judged is that the ceiling holds and nothing rings against it. + whip: { gain: 0.44, size: 0.7, over: 35, late: 150, kick: 0.01, back: 0.01, still: 0, lead: 101 }, + 'ramp up': { gain: 0.4, size: 4, over: 45, late: 85, kick: 0, back: 0, still: 0, lead: 101 }, + + // Stops. The one moment the guess is asked to give everything back, and the moment it is furthest from + // being able to see it coming — `still` is what that costs on screen. + 'flick, snapped stop': { gain: 0.41, size: 25, over: 60, late: 65, kick: 20, back: 35, still: 40, lead: 95 }, + 'flick, eased stop': { gain: 0.4, size: 20, over: 45, late: 65, kick: 9, back: 30, still: 30, lead: 95 }, + 'abrupt stop': { gain: 0.55, size: 25, over: 30, late: 50, kick: 0, back: 0, still: 85, lead: 75 }, + 'abrupt start': { gain: 0.47, size: 10, over: 30, late: 150, kick: 0, back: 0, still: 0, lead: 75 }, + 'stop and go': { gain: 0.72, size: 20, over: 35, late: 85, kick: 15, back: 35, still: 250, lead: 55 }, + 'point to point': { gain: 0.63, size: 6, over: 20, late: 50, kick: 3.5, back: 15, still: 55, lead: 45 }, + + // Turns. A per-axis fit reads the extremum of every one of these as a hand that is stopping, which is + // exactly backwards, so these are the cases the 2D fit exists for. + 'sharp corner': { gain: 0.49, size: 8.5, over: 20, late: 55, kick: 0, back: 0, still: 0, lead: 45 }, + 'rounded corner': { gain: 0.49, size: 20, over: 20, late: 50, kick: 0, back: 0, still: 65, lead: 45 }, + circle: { gain: 0.33, size: 9.5, over: 30, late: 50, kick: 0, back: 0, still: 0, lead: 75 }, + 'circle, tight and fast': { gain: 0.32, size: 8.5, over: 25, late: 30, kick: 0.25, back: 0.25, still: 0, lead: 60 }, + // A stir: small and quick enough that a whole horizon of it sweeps past a right angle, which is where the + // arc has to stop being believed rather than wrap round. + stir: { gain: 0.94, size: 15, over: 15, late: 40, kick: 0, back: 0, still: 0, lead: 50 }, + 'circle, reversed': { gain: 0.33, size: 9.5, over: 30, late: 50, kick: 0, back: 0, still: 0, lead: 75 }, + 'figure eight': { gain: 0.86, size: 8.5, over: 15, late: 65, kick: 2, back: 4.5, still: 0, lead: 50 }, + spiral: { gain: 0.58, size: 8.5, over: 30, late: 75, kick: 0, back: 0, still: 0, lead: 95 }, + serpentine: { gain: 0.69, size: 4, over: 15, late: 45, kick: 0, back: 0, still: 0, lead: 30 }, + // A turn slow enough and wide enough that the angle between the window's two halves barely clears what + // quantisation alone produces. Believing it too readily and not at all are both visible here. + 'wide slow arc': { gain: 0.36, size: 0.45, over: 4.5, late: 15, kick: 0, back: 0, still: 0, lead: 15 }, + + // Reversals. Nothing in a window says a turn is coming when the hand is at its fastest, so the guess is + // bounded by how long this hand has lately been going between turning round rather than by evidence. + // Gain over 1 is honest here: there is nothing to win, and the job is not to lose much. + zigzag: { gain: 1.06, size: 15, over: 20, late: 75, kick: 30, back: 90, still: 0, lead: 30 }, + 'shake, 8Hz': { gain: 1.11, size: 9, over: 45, late: 50, kick: 30, back: 200, still: 40, lead: 15 }, + 'shake, fast and small': { gain: 1.11, size: 6, over: 30, late: 30, kick: 25, back: 350, still: 55, lead: 6.5 }, + 'overshoot and correct': { gain: 0.55, size: 15, over: 35, late: 85, kick: 7, back: 15, still: 35, lead: 65 }, + 'target acquire': { gain: 0.54, size: 20, over: 40, late: 100, kick: 20, back: 20, still: 30, lead: 75 }, + // Braking is believed on thin evidence and acted on at once, which is right for a stop and is exactly + // wrong for a hand that is only pausing between two throws. This is the case that pays for that choice. + 'double flick': { gain: 0.55, size: 25, over: 40, late: 150, kick: 20, back: 50, still: 45, lead: 85 }, + + // A hand is never still. The wobble is the size of the counts underneath it, which is where a second + // derivative reads it as motion and a caller keeping the growth turns it into rotation. + 'tremor on a drag': { gain: 0.39, size: 2, over: 5.5, late: 15, kick: 0.25, back: 2, still: 0, lead: 15 }, + 'tremor at rest': { gain: 1.06, size: 2, over: 2.5, late: 3.5, kick: 2, back: 4.5, still: 15, lead: 1.5 }, +} + +describe('movement shapes', () => { + test('every scenario has a floor and every floor has a scenario', () => { + // A shape added to the rig with nothing holding it is a shape nobody is watching, and a floor left + // behind by a renamed scenario is a number that can never fail. + const names = new Set(SCENARIOS.map(scenario => scenario.name)) + expect(SCENARIOS.filter(scenario => !LIMITS[scenario.name]).map(scenario => scenario.name)).toEqual([]) + expect(Object.keys(LIMITS).filter(name => !names.has(name))).toEqual([]) + }) + + for (const scenario of SCENARIOS) { + test(`${scenario.name}: ${scenario.note}`, () => { + const limits = LIMITS[scenario.name]! + const worst = worstAcross(DEVICES, { + path: scenario.path, + durationMs: scenario.durationMs, + scoreAfterMs: scenario.scoreAfterMs, + }) + expect(worst.frames).toBeGreaterThan(10) + expect(worst.gain).toBeLessThanOrEqual(limits.gain) + expect(worst.worstSizeStep).toBeLessThanOrEqual(limits.size) + expect(worst.worstOvershoot).toBeLessThanOrEqual(limits.over) + expect(worst.worstLate).toBeLessThanOrEqual(limits.late) + expect(worst.worstBackwards).toBeLessThanOrEqual(limits.kick) + expect(worst.backwards).toBeLessThanOrEqual(limits.back) + expect(worst.phantom).toBeLessThanOrEqual(limits.still) + expect(worst.worstLead).toBeLessThanOrEqual(limits.lead) + }) + } + + test('nothing produces a lead that is not a number', () => { + // Cheap, and the one failure that would make every figure above meaningless. + for (const scenario of SCENARIOS) { + let frames = 0 + replay({ + path: scenario.path, + device: DEVICES[0]!, + durationMs: scenario.durationMs, + onFrame: frame => { + frames++ + const { x, y, live } = frame.lead + if (!Number.isFinite(x) || !Number.isFinite(y) || !Number.isFinite(live)) { + throw new Error(`${scenario.name} produced ${frame.lead.x},${frame.lead.y} at frame ${frame.index}`) + } + }, + }) + expect(frames).toBeGreaterThan(10) + } + }) +}) + +describe('the shapes that must beat not guessing', () => { + // A gain over 1 is only ever acceptable where there is nothing to win: a hand reversing faster than the + // horizon, or one moving too slowly to be led at all. Everywhere else the guess has to earn its place, and + // by a margin — the lateness it removes is the whole product. + const EARNS_ITS_PLACE = [ + 'steady fast', + 'steady on-axis', + 'whip', + 'ramp up', + 'flick, snapped stop', + 'flick, eased stop', + 'circle', + 'circle, tight and fast', + 'spiral', + 'point to point', + 'sharp corner', + 'rounded corner', + 'overshoot and correct', + 'target acquire', + 'double flick', + 'wide slow arc', + 'tremor on a drag', + ] + + for (const name of EARNS_ITS_PLACE) { + test(`${name} is worth guessing on every device`, () => { + const scenario = SCENARIOS.find(candidate => candidate.name === name)! + for (const device of DEVICES) { + const metrics: Metrics = replay({ + path: scenario.path, + device, + durationMs: scenario.durationMs, + scoreAfterMs: scenario.scoreAfterMs, + }) + // Two thirds of the error, at worst, on the device that reports least often. The wired figures are + // several times better than this and are held to the table above. + expect(metrics.error).toBeLessThan(metrics.errorWithout * 0.67) + } + }) + } +}) diff --git a/library/src/__tests__/predictor-stops.test.ts b/library/src/__tests__/predictor-stops.test.ts new file mode 100644 index 0000000..46c5150 --- /dev/null +++ b/library/src/__tests__/predictor-stops.test.ts @@ -0,0 +1,296 @@ +import { describe, expect, test } from 'bun:test' + +import { + ALL_DEVICES, + BLUETOOTH, + COARSE, + DEVICES, + flick, + glide, + hold, + piecewise, + replay, + SPARSE, + TRACKPAD, + WIRED, + type ReplayOptions, +} from './harness.js' + +// Stopping, and the overshoot that comes with it. +// +// This is where a guess is most exposed, and it is asymmetric in a way the error figures alone do not show: +// lead that turns out not to have been needed has to be handed back, and handing it back walks the view +// backwards under the hand, where lead that was never taken is only the lateness the library exists to +// remove. So everything here is scored on which side of the hand the guess was on, and how much of the +// picture moved while the hand itself was not moving at all. +// +// Nothing in a window says a stop is coming until it is already under way, so none of these figures can go +// to zero. What they can do is get worse without anyone noticing, which is what they are here for. + +type Frame = Parameters>[0] + +const framesOf = (options: Omit): Frame[] => { + const list: Frame[] = [] + replay({ ...options, onFrame: frame => list.push(frame) }) + return list +} + +const size = (frame: Frame) => Math.hypot(frame.lead.x, frame.lead.y) + +/** A hand crossing the pad and coming off it between one sample and the next. */ +const stopsAt = (movingMs: number, quietMs = 700) => piecewise([glide(2, 40, movingMs), hold(quietMs)]) + +/** Everything about the run after the moment the hand stopped. */ +const afterTheStop = (movingMs: number, device: (typeof ALL_DEVICES)[number], quietMs = 700) => { + const frames = framesOf({ path: stopsAt(movingMs, quietMs), device, durationMs: movingMs + quietMs }).filter( + frame => frame.frameAt >= movingMs, + ) + let travel = 0 + let grew = 0 + for (let i = 1; i < frames.length; i++) { + const before = frames[i - 1]! + const now = frames[i]! + travel += Math.hypot(now.shown.x - before.shown.x, now.shown.y - before.shown.y) + grew = Math.max(grew, size(now) - size(before)) + } + const zeroAt = frames.findIndex(frame => size(frame) === 0) + const deadAt = frames.findIndex(frame => frame.lead.live === 0) + return { + peak: Math.max(...frames.map(size)), + /** Frames until the lead is exactly zero, and until the predictor calls the hand stopped. */ + zeroAt: zeroAt === -1 ? frames.length : zeroAt, + deadAt: deadAt === -1 ? frames.length : deadAt, + /** How far the picture travelled with the hand already still. */ + travel, + /** The most the guess ever grew on a frame after the hand had stopped. */ + grew, + } +} + +describe('stopping', () => { + test('the lead settles at exactly zero, on every device', () => { + // Not "near zero": a caller that keeps what it is given keeps a creeping remainder for the rest of the + // session, and a tenth of a pixel a frame is a degree of rotation a minute. + for (const device of ALL_DEVICES) { + const { zeroAt } = afterTheStop(900, device) + expect(zeroAt).toBeLessThanOrEqual(18) + expect(zeroAt).toBeGreaterThan(2) + } + }) + + test('the predictor calls the hand stopped within a few frames of it stopping', () => { + // `live` is the signal a caller watches to drop the lead outright. It has to tolerate the quiet each + // device keeps while moving — a mouse on a radio leaves a third of frames empty — without letting a + // real stop go unnoticed for longer than the eye takes to see it. + for (const device of ALL_DEVICES) { + expect(afterTheStop(900, device).deadAt).toBeLessThanOrEqual(7) + } + }) + + test('nothing grows once the hand has stopped', () => { + // A frame that carries no samples has learned nothing, and a guess that grows on it is inventing motion + // on the one frame the eye is most likely to get: the next one is already being discounted. + for (const device of ALL_DEVICES) { + expect(afterTheStop(900, device).grew).toBeLessThan(1) + } + }) + + test('how long the hand moved first does not change how the stop ends', () => { + // The window is a fixed forty milliseconds, so a flick and a long drag end the same way. Anything that + // makes a long drag unwind more slowly is state that has been accumulating where it should not. + for (const device of [WIRED, TRACKPAD, BLUETOOTH]) { + const brief = afterTheStop(200, device) + const long = afterTheStop(3000, device) + expect(Math.abs(long.zeroAt - brief.zeroAt)).toBeLessThanOrEqual(3) + expect(long.travel).toBeLessThan(brief.travel * 1.5 + 5) + } + }) + + test('the picture moves less with the hand still than the guess it was holding', () => { + // Everything the guess holds at the moment the hand stops has to be given back, so this can never be + // nothing. What it must not be is more than that: a lead that unwinds past zero, or rings, or is still + // being topped up from a stale window, all show up as the picture travelling further than the lead ever + // was. The sparse device is exempt from the tight figure — it can be two frames late learning the hand + // moved at all, let alone that it stopped. + for (const device of ALL_DEVICES) { + const { travel, peak } = afterTheStop(900, device) + expect(travel).toBeLessThan(peak * (device === SPARSE ? 1.9 : 1.45)) + } + }) +}) + +describe('coming off a flick', () => { + // A cosine fall from a hard cruise: 40ms is a hand snapping to a halt, 400ms is letting go of a long drag. + // The overshoot allowed is per device and per fall, because the slower a device reports the later it can + // possibly know — and it is measured along the way the hand is going, so it is overshoot by itself rather + // than the ordinary lateness rolled in with it. + const OVERSHOOT: Record>> = { + wired: { 40: 38, 80: 26, 150: 12, 250: 6, 400: 4.5 }, + trackpad: { 40: 50, 80: 44, 150: 24, 250: 18, 400: 18 }, + bluetooth: { 40: 65, 80: 53, 150: 35, 250: 33, 400: 31 }, + 'slow radio': { 40: 60, 80: 57, 150: 45, 250: 40, 400: 36 }, + } + + for (const fallMs of [40, 80, 150, 250, 400]) { + test(`a ${fallMs}ms stop is not led past where the hand ends up`, () => { + for (const device of DEVICES) { + const path = flick(2.6, 20, 800, fallMs) + const durationMs = 800 + fallMs + 400 + const metrics = replay({ path, device, durationMs }) + expect(metrics.worstOvershoot).toBeLessThan(OVERSHOOT[device.name]![fallMs]!) + // And none of it may be bought by simply not guessing, which would score perfectly on overshoot. + expect(metrics.error).toBeLessThan(metrics.errorWithout * 0.45) + // Nor by handing back more than was ever taken. + expect(metrics.backwards).toBeLessThan(metrics.worstLead) + } + }) + } + + test('the lead is already coming down while the hand is still moving', () => { + // The whole reason for fitting a curve rather than smoothing a velocity. An estimator that cannot see a + // stop coming is still near its peak when the hand is already still; this asks where the guess is at the + // moment the hand has given up half its speed, which is halfway through the fall. + // + // A 40ms fall is excluded: the hand is gone inside two frames and nothing can be read that fast. The + // slower the device reports, the later it can know, which is what the second figure is. + const HALFWAY: Record = { wired: 0.62, trackpad: 0.78, bluetooth: 0.88, 'slow radio': 0.82 } + for (const fallMs of [80, 150, 250]) { + for (const device of DEVICES) { + const path = flick(2.6, 20, 800, fallMs) + const frames = framesOf({ path, device, durationMs: 800 + fallMs + 400 }) + const peak = Math.max(...frames.filter(frame => frame.frameAt < 800).map(size)) + const halfway = frames.find(frame => frame.frameAt >= 800 + fallMs / 2)! + expect(size(halfway)).toBeLessThan(peak * HALFWAY[device.name]!) + } + } + }) + + test('the guess never points backwards through a stop', () => { + // Past the vertex of a fitted parabola the slope inverts, so the tail of a flick asks to be led + // backwards. That is a visible kick and no sample supports it: a guess may say the hand is about to + // slow, it may not say the hand is about to reverse. + // + // The low-DPI mouse is held to its own figure and it is not a good one — see the note on it in + // `predictor-devices.test.ts`. Its counts are four pixels wide, the turn rate is read against a floor + // written in units of one, and at the tail of a stop the held lead gets spun through a right angle by a + // turn that is entirely staircase. Four pixels of backwards lead is what that costs. + for (const device of ALL_DEVICES) { + const allowed = device === COARSE ? -6 : -0.05 + for (const fallMs of [40, 150, 400]) { + let worst = 0 + framesOf({ path: flick(2.6, 20, 800, fallMs), device, durationMs: 800 + fallMs + 400 }).forEach(frame => { + const along = frame.lead.x * Math.cos((20 * Math.PI) / 180) + frame.lead.y * Math.sin((20 * Math.PI) / 180) + worst = Math.min(worst, along) + }) + expect(worst).toBeGreaterThanOrEqual(allowed) + } + } + }) +}) + +describe('starting again', () => { + test('a hard reversal turns the lead round within a few frames', () => { + // Not a shake — the hand runs one way for most of a second and then the other, so there is a genuine + // reversal in the window rather than an oscillation to be bounded. The guess has to follow it, and the + // only thing that can carry it is the measured travel turning round. + const path = piecewise([glide(1.5, 0, 700), glide(1.5, 180, 700)]) + for (const device of DEVICES) { + const after = framesOf({ path, device, durationMs: 1400 }).filter(frame => frame.frameAt >= 700) + const flipped = after.findIndex(frame => frame.lead.x < 0) + expect(flipped).toBeGreaterThanOrEqual(0) + expect(flipped).toBeLessThanOrEqual(10) + // And it has to be worth something once it has turned, not merely the right sign. + expect(after.slice(0, 12).some(frame => frame.lead.x < -5)).toBe(true) + } + }) + + test('a start from rest arrives without inventing motion first', () => { + // The frame the hand starts moving carries a window that is almost entirely the stillness before it, so + // the guess has to grow into the motion rather than jump. Anything that jumps here is a fit reading a + // span of a fraction of a millisecond. + const path = piecewise([hold(500), glide(2, 40, 900)]) + for (const device of ALL_DEVICES) { + const frames = framesOf({ path, device, durationMs: 1400 }) + let worstGrowth = 0 + for (let i = 1; i < frames.length; i++) + worstGrowth = Math.max(worstGrowth, size(frames[i]!) - size(frames[i - 1]!)) + // A frame's worth of the motion itself is about 33px, and the guess is about 40px, so it may not + // arrive in one step. It also may not take for ever: the second figure is the point of the library. + expect(worstGrowth).toBeLessThan(12) + const settled = frames.filter(frame => frame.frameAt > 700) + expect(Math.max(...settled.map(size))).toBeGreaterThan(20) + } + }) + + test('repeated starts and stops do not accumulate into rotation', () => { + // Six flicks with a pause after each. A caller applying only the growth keeps every upward wobble and + // gives none of them back, so anything shaky here is not paid back at the stop — it is kept for the rest + // of the session. About one lead per cycle is the honest figure; several times that is a ratchet. + const cycle = [glide(1.6, 15, 200), hold(180)] + const path = piecewise([hold(200), ...Array.from({ length: 6 }, () => cycle).flat()]) + for (const device of DEVICES) { + const { kept, worstLead } = replay({ path, device, durationMs: 2700 }) + expect(kept).toBeLessThan(worstLead * 7) + } + }) + + test('a pause before a flick does not buy the flick a longer stop', () => { + // How long a device stays quiet while the hand is moving has to be learned. What is far too easy to + // learn by mistake is the hand sitting still, which looks identical and is many times longer — taken for + // a property of the device it says this one reports three times a second, and the flick after it sails + // on with the hand already still. + for (const device of DEVICES) { + const sail = (idleMs: number) => { + const path = piecewise([hold(idleMs), glide(2, 40, 500), hold(600)]) + const frames = framesOf({ path, device, durationMs: idleMs + 1100 }).filter( + frame => frame.frameAt >= idleMs + 500, + ) + const zeroAt = frames.findIndex(frame => size(frame) === 0) + return zeroAt === -1 ? frames.length : zeroAt + } + const straightIn = sail(50) + for (const idleMs of [200, 500, 1500, 4000]) { + expect(sail(idleMs)).toBeLessThanOrEqual(straightIn + 2) + } + } + }) +}) + +describe('the picture never fights the hand', () => { + // The one error a camera cannot hide is the view walking backwards while the hand is moving forwards. It + // costs nothing to be late; it costs the illusion to reverse. + // + // A hand that simply stops never produces any of it, and neither does a corner at speed: the lead comes + // off while the hand is still travelling, so the picture only slows. It is the eased stop that does, and + // by how much is entirely a question of how late the device is — the wired mouse gives back a seventh of a + // pixel in the worst frame and the slow radio gives back fourteen. That is the shape of the whole + // trade-off in one number, and it is the number to watch when the braking is touched. + const CASES = [ + { + name: 'a long drag stopping', + path: stopsAt(900), + durationMs: 1600, + kick: { wired: 0.5, trackpad: 0.5, bluetooth: 0.5, 'slow radio': 0.5 }, + }, + { + name: 'a flick easing out', + path: flick(2.6, 20, 800, 150), + durationMs: 1350, + kick: { wired: 0.4, trackpad: 3.5, bluetooth: 11, 'slow radio': 17 }, + }, + { + name: 'a corner at speed', + path: piecewise([glide(1.2, 0, 900), glide(1.2, 90, 900)]), + durationMs: 1800, + kick: { wired: 0.5, trackpad: 0.5, bluetooth: 0.5, 'slow radio': 0.5 }, + }, + ] as const + + for (const { name, path, durationMs, kick } of CASES) { + test(name, () => { + for (const device of DEVICES) { + expect(replay({ path, device, durationMs }).worstBackwards).toBeLessThan(kick[device.name as keyof typeof kick]) + } + }) + } +}) diff --git a/library/src/__tests__/predictor.test.ts b/library/src/__tests__/predictor.test.ts index 8fd7865..ed0dde2 100644 --- a/library/src/__tests__/predictor.test.ts +++ b/library/src/__tests__/predictor.test.ts @@ -1,14 +1,13 @@ import { beforeEach, describe, expect, test } from 'bun:test' import { createPointerPredictor, DEFAULT_MAX_LEAD_PX, DEFAULT_DECAY_MS, DEFAULT_LEAD_FRAMES } from '../predictor.js' +import { AGE, BLUETOOTH, circle, FRAME, flick, replay, shake, SLOW_RADIO, steady, TRACKPAD, WIRED } from './harness.js' -const FRAME = 16.7 // Track what ships, so the floor is the floor of the thing that ships. const LEAD_FRAMES = DEFAULT_LEAD_FRAMES const DECAY = 20 const CAP = 40 const SAMPLES_PER_FRAME = 16 // a 1000Hz mouse, which is what puts several samples in one frame -const AGE = 13 // what the newest sample's age measures at at 60Hz, per the example rig describe('pointer prediction', () => { let predictor = createPointerPredictor() @@ -502,187 +501,44 @@ describe('pointer prediction', () => { // changing, every bit of it is jitter, and jitter on a camera is nauseating. Accuracy is scored against // where the hand really is when the frame reaches the screen, and has to beat not guessing at all. // -// Breaking each part of the predictor in turn fails something here, with one exception worth knowing -// about: CURVE_TRUST_MS can be set to nothing and every figure below stays put. It settles how far the -// curve is believed, which also decides how hard the lead is smoothed, and left unsettled that judgement -// flips frame to frame and the smoothing switches on and off with it. The synthetic hands here move too -// perfectly for the bend to cross its noise floor, so they never produce the flicker. It was found by hand -// on a Bluetooth mouse and confirmed by removing it twice. Do not read its lack of a test as it having no -// job. +// This file is the original floor and stays as the shortest description of what the predictor owes. The +// broad sweep lives next door — `predictor-shapes`, `predictor-stops`, `predictor-devices` and +// `predictor-invariants`, all replayed through `harness.ts`, which is also what `bun run bench` prints. +// +// Every constant in `predictor.ts` was broken in turn against the whole suite and every one of them fails +// something, which is the property worth keeping as it grows. Two lines are not reached by any hand that +// can be modelled here, and are worth knowing about rather than testing for the sake of it: +// +// - `MAX_TURN_RAD`, which clamps how far one horizon of arc may sweep. By the time the turn rate has been +// through its own noise gate, a hand would have to be circling faster than nine revolutions a second to +// reach it. It is a guard against a fit that has run away, not against a hand. +// - The horizon being cut short at the point the fit says the hand stops. Past that point the distance is +// already floored at zero, so removing the cut moves nothing measurably. +// +// CURVE_TRUST_MS used to be a third: the synthetic hands in this file move too perfectly for the bend to +// sit near its noise floor, which is the flicker it exists to stop. The tremor and coarse-count cases next +// door do produce it, so setting it to nothing now fails seventeen tests. describe('pointer prediction quality floor', () => { - // How many refreshes after a frame is drawn it reaches the screen. No clock inside a page can see this, - // which is why the example rig exists — so it is an assumption, and the lead is set to match it. Tying - // the two together is the point: the guess is judged against the pipeline it is aimed at, and if the rig - // says that pipeline is a different depth, both move and these figures are re-measured. - const PRESENT_FRAMES = DEFAULT_LEAD_FRAMES - - /** How a device reports: how often, how many at a time, and how much its clock wanders. */ - type Device = { periodMs: number; burst: number; jitterMs: number } - const WIRED: Device = { periodMs: 1, burst: 1, jitterMs: 0.1 } - const TRACKPAD: Device = { periodMs: 11, burst: 1, jitterMs: 0.4 } - // A mouse on a radio: reports in clumps, and says nothing at all for a third of frames while moving. - const BLUETOOTH: Device = { periodMs: 8, burst: 3, jitterMs: 2.5 } - // Slower still, and the one whose gaps most resemble a hand that has stopped. - const SLOW_RADIO: Device = { periodMs: 16, burst: 2, jitterMs: 3 } - - /** `hitchEvery` stalls one frame in that many to three times its length, as a real one does. */ - const replay = ( - path: (t: number) => { x: number; y: number }, - device: Device, - durationMs: number, - hitchEvery = 0, - ) => { - const predictor = createPointerPredictor() - let seed = 20260808 - const rnd = () => (seed = (seed * 1103515245 + 12345) & 0x7fffffff) / 0x7fffffff - - let reportedX = 0 - let reportedY = 0 - let integratedX = 0 - let integratedY = 0 - let frameAt = 0 - let prevFrameAt = 0 - let nextReport = 0 - let previous: { x: number; y: number } | null = null - // What a caller applying only the growth accumulates. Jitter is rectified into rotation there: every - // upward wobble is kept and none of the downward ones give it back. - let keptX = 0 - let keptY = 0 - let appliedX = 0 - let appliedY = 0 - let worstStep = 0 - let worstLead = 0 - let worstOvershoot = 0 - let radialSum = 0 - let withSum = 0 - let withoutSum = 0 - let scored = 0 - - let frameIndex = 0 - while (frameAt <= durationMs) { - frameIndex++ - frameAt += FRAME * (hitchEvery > 0 && frameIndex % hitchEvery === 0 ? 3 : 1) - // Events are held until shortly before the callback, and arrive a burst at a time. - while (nextReport <= frameAt - 2) { - for (let b = 0; b < device.burst; b++) { - const t = nextReport - (device.burst - 1 - b) * device.periodMs + (rnd() - 0.5) * device.jitterMs - if (t <= 0) continue - const at = path(t) - // Whole mouse counts, as a real device reports. - const dx = Math.trunc(at.x) - reportedX - const dy = Math.trunc(at.y) - reportedY - if (dx !== 0 || dy !== 0) { - predictor.push(t, dx, dy) - reportedX += dx - reportedY += dy - integratedX += dx - integratedY += dy - } - } - nextReport += device.periodMs * device.burst - } - - const lead = predictor.update({ - nowMs: frameAt, - deltaMs: frameAt - prevFrameAt, - leadFrames: LEAD_FRAMES, - decayMs: DEFAULT_DECAY_MS, - maxLeadPx: DEFAULT_MAX_LEAD_PX, - }) - prevFrameAt = frameAt - - const growth = (value: number, applied: number) => - value * applied >= 0 && Math.abs(value) > Math.abs(applied) ? value - applied : 0 - keptX += growth(lead.x, appliedX) - keptY += growth(lead.y, appliedY) - appliedX = lead.x - appliedY = lead.y - - // Once the window is full of the motion rather than of starting up. - if (frameAt > 600) { - if (previous) worstStep = Math.max(worstStep, Math.hypot(lead.x - previous.x, lead.y - previous.y)) - const truth = path(frameAt + PRESENT_FRAMES * FRAME) - - // How far past the hand the guess sits, along the way the hand is going. Overshooting and falling - // short are not the same error and are not worth scoring as one: lead that was never there has to - // be handed back, and handing it back is a reversal on screen, where falling short is only the - // lateness this whole thing exists to remove. - const ahead = path(frameAt + PRESENT_FRAMES * FRAME + 1) - const behind = path(frameAt + PRESENT_FRAMES * FRAME - 1) - const towardsX = ahead.x - behind.x - const towardsY = ahead.y - behind.y - const towards = Math.hypot(towardsX, towardsY) - if (towards > 1e-9) { - const past = - ((integratedX + lead.x - truth.x) * towardsX + (integratedY + lead.y - truth.y) * towardsY) / towards - worstOvershoot = Math.max(worstOvershoot, past) - } - withSum += (integratedX + lead.x - truth.x) ** 2 + (integratedY + lead.y - truth.y) ** 2 - withoutSum += (integratedX - truth.x) ** 2 + (integratedY - truth.y) ** 2 - radialSum += Math.hypot(integratedX + lead.x, integratedY + lead.y) - worstLead = Math.max(worstLead, Math.hypot(lead.x, lead.y)) - scored++ - } - previous = { x: lead.x, y: lead.y } - } - - return { - worstStep, - error: Math.sqrt(withSum / Math.max(1, scored)), - errorWithout: Math.sqrt(withoutSum / Math.max(1, scored)), - meanRadius: radialSum / Math.max(1, scored), - kept: Math.hypot(keptX, keptY), - worstLead, - worstOvershoot, - } - } - - const steady = (speed: number, angleDeg: number) => (t: number) => ({ - x: speed * Math.cos((angleDeg * Math.PI) / 180) * t, - y: speed * Math.sin((angleDeg * Math.PI) / 180) * t, - }) - const shake = (amplitude: number, hz: number) => (t: number) => ({ - x: amplitude * Math.sin((2 * Math.PI * hz * t) / 1000), - y: 0, - }) - const circle = (r: number, periodMs: number) => (t: number) => ({ - x: r * Math.cos((2 * Math.PI * t) / periodMs), - y: r * Math.sin((2 * Math.PI * t) / periodMs), - }) - // A hand that throws the view somewhere and lets go: a cruise, then a cosine ease down to a dead stop - // over fallMs. The stop is the whole point — it is the one moment the guess is asked to give back - // everything it is holding, and the moment it is furthest from being able to see it coming. - const flick = (speed: number, angleDeg: number, cruiseMs: number, fallMs: number) => { - const ux = Math.cos((angleDeg * Math.PI) / 180) - const uy = Math.sin((angleDeg * Math.PI) / 180) - const travelled = (t: number) => { - if (t <= cruiseMs) return speed * t - const u = Math.min(t - cruiseMs, fallMs) - return speed * (cruiseMs + u / 2 + (fallMs / (2 * Math.PI)) * Math.sin((Math.PI * u) / fallMs)) - } - return (t: number) => ({ x: ux * travelled(t), y: uy * travelled(t) }) - } - // A diagonal is the hard case for steadiness: the two axes cross their whole-count boundaries at // different moments, so the path the fit sees zig-zags about the true line. test('a wired mouse on a steady diagonal barely moves the lead at all', () => { - expect(replay(steady(0.8, 30), WIRED, 2500).worstStep).toBeLessThan(0.15) + expect(replay({ path: steady(0.8, 30), device: WIRED, durationMs: 2500 }).worstStep).toBeLessThan(0.15) }) test('a trackpad on a steady diagonal is nearly as steady as a wired mouse', () => { - expect(replay(steady(0.8, 30), TRACKPAD, 2500).worstStep).toBeLessThan(0.4) + expect(replay({ path: steady(0.8, 30), device: TRACKPAD, durationMs: 2500 }).worstStep).toBeLessThan(0.4) }) test('a mouse on a radio is within reach of the other two', () => { // It reports a fifth as often and in clumps, so it will never match them; it must stay close enough // that no device makes the camera shake. - expect(replay(steady(0.8, 30), BLUETOOTH, 2500).worstStep).toBeLessThan(0.9) - expect(replay(steady(0.8, 30), SLOW_RADIO, 2500).worstStep).toBeLessThan(0.8) + expect(replay({ path: steady(0.8, 30), device: BLUETOOTH, durationMs: 2500 }).worstStep).toBeLessThan(0.9) + expect(replay({ path: steady(0.8, 30), device: SLOW_RADIO, durationMs: 2500 }).worstStep).toBeLessThan(0.8) }) test('slow motion is steady on every device, which is where a shaky guess shows most', () => { for (const device of [WIRED, TRACKPAD, BLUETOOTH, SLOW_RADIO]) { - expect(replay(steady(0.15, 30), device, 2500).worstStep).toBeLessThan(0.35) + expect(replay({ path: steady(0.15, 30), device, durationMs: 2500 }).worstStep).toBeLessThan(0.35) } }) @@ -827,7 +683,7 @@ describe('pointer prediction quality floor', () => { // four seconds at a crawl this used to reach nearly four times the lead it should have left behind. for (const device of [WIRED, TRACKPAD, BLUETOOTH, SLOW_RADIO]) { const speed = 0.06 - const { kept } = replay(steady(speed, 30), device, 4000) + const { kept } = replay({ path: steady(speed, 30), device, durationMs: 4000 }) const oneLead = speed * (13 + LEAD_FRAMES * FRAME) expect(kept).toBeLessThan(oneLead * 0.6) } @@ -838,7 +694,7 @@ describe('pointer prediction quality floor', () => { // taken for one. Believing it moves the horizon by two frames and hands over pixels of lead on the // frame after every stall. for (const device of [WIRED, TRACKPAD, BLUETOOTH]) { - expect(replay(steady(0.8, 30), device, 3000, 20).worstStep).toBeLessThan(1.15) + expect(replay({ path: steady(0.8, 30), device, durationMs: 3000, hitchEvery: 20 }).worstStep).toBeLessThan(1.15) } }) @@ -852,7 +708,7 @@ describe('pointer prediction quality floor', () => { [40, 8], [10, 12], ] as const) { - const { worstLead, error, errorWithout } = replay(shake(amplitude, hz), WIRED, 2500) + const { worstLead, error, errorWithout } = replay({ path: shake(amplitude, hz), device: WIRED, durationMs: 2500 }) // Never further out than the motion itself is wide. A shake is a stop twice a cycle, so reading the // braking sooner pulled both of these in and they were tightened to match. expect(worstLead).toBeLessThan(amplitude * 0.55) @@ -872,7 +728,7 @@ describe('pointer prediction quality floor', () => { [120, 300, 4], [200, 400, 3], ] as const) { - const { meanRadius } = replay(circle(r, periodMs), WIRED, 3000) + const { meanRadius } = replay({ path: circle(r, periodMs), device: WIRED, durationMs: 3000 }) expect(meanRadius - r).toBeLessThan(tolerance) expect(meanRadius - r).toBeGreaterThan(-tolerance) } @@ -898,7 +754,11 @@ describe('pointer prediction quality floor', () => { for (const [device, ...allowed] of limits) { const falls = [80, 150, 250] as const falls.forEach((fallMs, i) => { - const { worstOvershoot, error, errorWithout } = replay(flick(2.6, 20, 800, fallMs), device, 800 + fallMs + 300) + const { worstOvershoot, error, errorWithout } = replay({ + path: flick(2.6, 20, 800, fallMs), + device, + durationMs: 800 + fallMs + 300, + }) expect(worstOvershoot).toBeLessThan(allowed[i]!) // And none of that may be bought by simply not guessing, which would score perfectly here. expect(error).toBeLessThan(errorWithout * 0.45) @@ -908,10 +768,10 @@ describe('pointer prediction quality floor', () => { test('guessing beats not guessing, on every device and both kinds of path', () => { for (const device of [WIRED, TRACKPAD, BLUETOOTH, SLOW_RADIO]) { - const straight = replay(steady(0.8, 30), device, 2500) + const straight = replay({ path: steady(0.8, 30), device, durationMs: 2500 }) expect(straight.error).toBeLessThan(straight.errorWithout * 0.35) - const round = replay(circle(120, 400), device, 3000) + const round = replay({ path: circle(120, 400), device, durationMs: 3000 }) expect(round.error).toBeLessThan(round.errorWithout * 0.75) } }) diff --git a/library/src/__tests__/scoreboard.ts b/library/src/__tests__/scoreboard.ts new file mode 100644 index 0000000..918f918 --- /dev/null +++ b/library/src/__tests__/scoreboard.ts @@ -0,0 +1,117 @@ +// Every scenario against every device, printed as a table. This is the thing to run while changing the fit: +// the tests say whether anything got worse than it is allowed to be, and this says what actually moved. +// +// bun run bench every scenario, on the four devices the shipped figures are quoted at +// bun run bench circle only the scenarios whose name contains "circle" +// bun run bench --all every device, including the coarse and sparse ones +// bun run bench --save before.json +// bun run bench --diff before.json every number against that run, so a change is read as a change +// +// A tuning change almost always trades one column against another — a stop seen sooner is lead given up on +// the way in, a smoother lead is a later one. `--diff` is there so that trade is visible rather than +// discovered later on somebody's desk. + +import { readFileSync, writeFileSync } from 'node:fs' + +import { ALL_DEVICES, DEVICES, replay, SCENARIOS, type Device, type Metrics } from './harness.js' + +/** The columns worth watching, in the order they are worth watching them. `lower` is which way is better. */ +const COLUMNS = [ + { key: 'gain', label: 'gain', digits: 3, lower: true }, + { key: 'error', label: 'err', digits: 2, lower: true }, + { key: 'worstOvershoot', label: 'over', digits: 2, lower: true }, + { key: 'worstLate', label: 'late', digits: 2, lower: true }, + { key: 'worstSizeStep', label: 'size', digits: 3, lower: true }, + { key: 'worstTurnStep', label: 'turn°', digits: 2, lower: true }, + { key: 'worstStep', label: 'step', digits: 3, lower: true }, + { key: 'backwards', label: 'back', digits: 1, lower: true }, + { key: 'worstBackwards', label: 'kick', digits: 2, lower: true }, + { key: 'phantom', label: 'still', digits: 1, lower: true }, + { key: 'worstLead', label: 'lead', digits: 1, lower: false }, + { key: 'bias', label: 'bias', digits: 2, lower: true }, + { key: 'kept', label: 'kept', digits: 1, lower: true }, +] as const satisfies readonly { key: keyof Metrics; label: string; digits: number; lower: boolean }[] + +const run = (filter: string, devices: readonly Device[]) => { + const rows: { scenario: string; device: string; metrics: Metrics }[] = [] + for (const scenario of SCENARIOS) { + if (filter && !scenario.name.toLowerCase().includes(filter.toLowerCase())) continue + for (const device of devices) { + rows.push({ + scenario: scenario.name, + device: device.name, + metrics: replay({ + path: scenario.path, + device, + durationMs: scenario.durationMs, + scoreAfterMs: scenario.scoreAfterMs, + }), + }) + } + } + return rows +} + +const pad = (value: string, width: number, left = false) => (left ? value.padStart(width) : value.padEnd(width)) + +const format = (value: number, digits: number) => + Number.isFinite(value) ? value.toFixed(digits) : value > 0 ? 'inf' : '-inf' + +const main = () => { + const args = process.argv.slice(2) + const after = (flag: string) => (args.includes(flag) ? args[args.indexOf(flag) + 1] : undefined) + const saveTo = after('--save') + const diffWith = after('--diff') + const filter = args.find(arg => !arg.startsWith('--') && arg !== saveTo && arg !== diffWith) ?? '' + const devices = args.includes('--all') ? ALL_DEVICES : DEVICES + + const rows = run(filter, devices) + if (rows.length === 0) { + console.log(`no scenario matches "${filter}"`) + return + } + + const baseline: Record | null = diffWith ? JSON.parse(readFileSync(diffWith, 'utf8')) : null + + const nameWidth = Math.max(...rows.map(row => row.scenario.length), 8) + 2 + const deviceWidth = Math.max(...rows.map(row => row.device.length), 6) + 2 + const cellWidth = 9 + + console.log( + pad('scenario', nameWidth) + + pad('device', deviceWidth) + + COLUMNS.map(column => pad(column.label, cellWidth, true)).join(''), + ) + + let previousScenario = '' + for (const row of rows) { + const before = baseline?.[`${row.scenario}|${row.device}`] + const cells = COLUMNS.map(column => { + const now = row.metrics[column.key] + const cell = format(now, column.digits) + if (!before) return pad(cell, cellWidth, true) + const was = before[column.key] + const change = now - was + // Only what moved enough to be worth reading. Sub-percent wander is not a result. + if (Math.abs(change) <= Math.abs(was) * 0.01 + 1e-9) return pad(cell, cellWidth, true) + const better = column.lower ? change < 0 : change > 0 + return pad(`${cell}${better ? '↓' : '↑'}`, cellWidth + 1, true) + }) + const label = row.scenario === previousScenario ? '' : row.scenario + previousScenario = row.scenario + console.log(pad(label, nameWidth) + pad(row.device, deviceWidth) + cells.join('')) + } + + if (saveTo) { + const out: Record = {} + for (const row of rows) out[`${row.scenario}|${row.device}`] = row.metrics + writeFileSync(saveTo, `${JSON.stringify(out, null, 2)}\n`) + console.log(`\nsaved ${rows.length} rows to ${saveTo}`) + } + + if (baseline) { + console.log('\n↓ better than the baseline, ↑ worse. Anything unmarked moved by less than a percent.') + } +} + +main() diff --git a/library/tsconfig.json b/library/tsconfig.json index e9fde86..430a3fd 100644 --- a/library/tsconfig.json +++ b/library/tsconfig.json @@ -2,6 +2,9 @@ "compilerOptions": { // Environment "lib": ["ESNext", "DOM"], + // The replay rig and the scoreboard under `src/__tests__` are run by Bun, not shipped, and the + // scoreboard reads a baseline file and its own arguments. + "types": ["bun"], "target": "ESNext", "module": "Preserve", "moduleDetection": "force", diff --git a/package.json b/package.json index 4b7c837..5a53d85 100644 --- a/package.json +++ b/package.json @@ -17,6 +17,7 @@ "format": "oxfmt .", "format:check": "oxfmt --check .", "test": "bun test", + "bench": "bun run library/src/__tests__/scoreboard.ts", "warden": "warden", "all": "bun run format:check && bun run lint && bun run typecheck && bun run warden && bun run test", "prepare": "husky", From c267b3e4db05d6b9fe7bbd92671666ce6afc4368 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 03:18:08 +0000 Subject: [PATCH 2/6] fix: measure what a count is worth instead of assuming a pixel MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every noise floor in the fit is written in units of one count, and a count was assumed to be a pixel. That holds for a mouse at ordinary sensitivity and not for a low-DPI one, a page at a device pixel ratio other than 1, or an OS scaling the counts on the way through. On a mouse stepping four pixels a count, every floor sat four times too low and its staircase was believed as motion: the guess swung 28 degrees of heading a frame on a dead straight path, jumped thirty times as far in size as the same hand on a 125Hz mouse reporting just as often, ratcheted nine times as much, spun the held lead through a right angle at the tail of a stop, and led a crawl it should have refused. Deltas are the only thing that knows what a count is worth, and every delta is a whole number of them, so the grid they sit on is their greatest common divisor. Unlike sample counts or smallest deltas, that reading does not confuse a coarse device with a fast hand. Two windows of 250ms have to agree before the floors move, since a hand can hold a factor by luck for one of them but not two, and the answer is held at a pixel from below: a device finer than that is quieter than the floors assume, so following it down would mean believing more of a second derivative on the strength of an estimate. That is what makes this safe rather than a retune — the answer can only move floors up, and eight of the nine modelled devices are bit-identical to before. The speed gate takes the extra uncertainty off the measured travel rather than putting it on the threshold. Multiplying the threshold refuses an ordinary 0.35px/ms drag on such a device; four counts across a window are far too few to fit a curve to and plenty to say how fast a hand is going. On the low-DPI mouse: size step 4.69 -> 0.59, heading swing 27.9 -> 1.1 degrees a frame, ratchet 198 -> 53, crawl lead 6.3 -> 1.6, backwards lead at a stop 4.25 -> none, and error on a straight drag 0.315 -> 0.185 of not guessing. It costs the curvature, which four-pixel counts genuinely cannot carry: a second derivative there sits 1.3x its own noise, so the arc is mostly given up and circles fall back to a straight line that points half a window of turning behind the tangent. That trade is pinned in predictor-devices.test.ts with the numbers a fix would have to beat — advancing the swept heading by the turn already measured would give most of it back without believing any more curvature, and that is a change to how every device extrapolates rather than to this one. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HctBLnHDXwx1XURcvnEB9T --- README.md | 7 + library/src/__tests__/harness.ts | 11 + .../src/__tests__/predictor-devices.test.ts | 227 +++++++++++++++--- .../src/__tests__/predictor-shapes.test.ts | 5 + library/src/__tests__/predictor-stops.test.ts | 12 +- library/src/__tests__/predictor.test.ts | 6 + library/src/predictor.ts | 97 +++++++- 7 files changed, 315 insertions(+), 50 deletions(-) diff --git a/README.md b/README.md index 9d3c420..5d31613 100644 --- a/README.md +++ b/README.md @@ -234,6 +234,13 @@ The rest is mostly about not believing noise: - **Below a crawl there is no lead at all.** Whole mouse counts are all a slow hand produces, and a fit over four of them is mostly quantisation. +- **What a count is worth is measured, not assumed.** Every noise floor is written in units of one count, and + a count is one pixel only on a mouse at ordinary sensitivity — a low-DPI one steps four, and so does + anything the OS or a device pixel ratio scales on the way through. Assumed, that put every floor four times + too low on such a device: it swung the guess 28° of heading a frame on a dead straight path and led a crawl + it should have refused. The deltas are the only thing that knows, and every delta is a whole number of + counts — so the grid they sit on is their greatest common divisor, which is the one reading of it that does + not mistake a coarse device for a fast hand. - **The parabola is believed only as far as the bend stands above the noise floor** — how hard a path of whole counts appears to bend when it is dead straight — and slides back to the straight line as that margin closes. Sample count is the wrong question: it counts the samples that carried motion, so a fast diff --git a/library/src/__tests__/harness.ts b/library/src/__tests__/harness.ts index 6409da9..dc419a0 100644 --- a/library/src/__tests__/harness.ts +++ b/library/src/__tests__/harness.ts @@ -64,6 +64,10 @@ export const USB_8K = device('8KHz mouse', 0.125, 1, 0.05) /** Coarse counts: four pixels a step, which is what a low-DPI mouse reports. The path arrives as a * staircase, and every noise floor in the fit is written in units of one count. */ export const COARSE = device('low-DPI mouse', 8, 1, 0.5, 4) +/** Counts finer than a pixel, which is what a page at a device pixel ratio of two reports. The floors were + * tuned at a pixel and are held there rather than followed down: a device quieter than they assume costs a + * little sensitivity, and one noisier than they assume is the bug this is the other side of. */ +export const FINE = device('half-pixel mouse', 4, 1, 0.3, 0.5) /** The sparsest thing that still calls itself a pointer: less than one report a frame, so most frames see * nothing and the window holds two or three samples. */ export const SPARSE = device('sparse', 25, 1, 4) @@ -80,6 +84,7 @@ export const ALL_DEVICES: readonly Device[] = [ BLUETOOTH, SLOW_RADIO, COARSE, + FINE, SPARSE, ] @@ -763,6 +768,12 @@ export const SCENARIOS: readonly Scenario[] = [ circle(25, 110), 2500, ), + scenario( + 'circle, hand-drawn', + 'the same circle with a hand on it, so the quantisation error stops being a function of the angle', + withTremor(circle(120, 400), 1.5, 7), + 3000, + ), scenario('circle, reversed', 'and the other way round, so a turn sign error cannot hide', circle(120, 400, -1), 3000), scenario( 'figure eight', diff --git a/library/src/__tests__/predictor-devices.test.ts b/library/src/__tests__/predictor-devices.test.ts index 58c39c6..cb55862 100644 --- a/library/src/__tests__/predictor-devices.test.ts +++ b/library/src/__tests__/predictor-devices.test.ts @@ -1,19 +1,27 @@ import { describe, expect, test } from 'bun:test' +import { createPointerPredictor, DEFAULT_DECAY_MS, DEFAULT_LEAD_FRAMES, DEFAULT_MAX_LEAD_PX } from '../predictor.js' import { ALL_DEVICES, BLUETOOTH, COARSE, + FINE, flick, + glide, HZ_120, HZ_144, HZ_30, HZ_60, + FRAME, + hold, + piecewise, replay, + SCENARIOS, SPARSE, steady, TRACKPAD, USB_125, + USB_500, WIRED, } from './harness.js' @@ -26,10 +34,13 @@ import { // refreshes because the delay it covers is a count of refreshes, so the same setting has to mean half as // long on a 120Hz screen and the figures have to hold there too. +const SHIPPED = { leadFrames: DEFAULT_LEAD_FRAMES, decayMs: DEFAULT_DECAY_MS, maxLeadPx: DEFAULT_MAX_LEAD_PX } + describe('every device', () => { // How much the size of the guess may jump in one frame, on a hand that is doing nothing new. Measured, - // then given room. The ordering is the interesting part: it tracks how often the device reports, and the - // one device that breaks the ordering breaks it for a reason worth knowing (see below). + // then given room. The ordering is the interesting part: it tracks how often the device reports and + // nothing else — the low-DPI mouse sits with the other devices of its rate rather than thirty times worse + // than them, which is the whole of what measuring the count size bought. const STEADY: Record = { wired: { fast: 0.05, slow: 0.09, gain: 0.06 }, '8KHz mouse': { fast: 0.04, slow: 0.07, gain: 0.05 }, @@ -38,7 +49,8 @@ describe('every device', () => { trackpad: { fast: 0.23, slow: 0.18, gain: 0.17 }, bluetooth: { fast: 0.73, slow: 0.25, gain: 0.28 }, 'slow radio': { fast: 0.64, slow: 0.29, gain: 0.36 }, - 'low-DPI mouse': { fast: 5.9, slow: 3, gain: 0.4 }, + 'low-DPI mouse': { fast: 0.75, slow: 0.65, gain: 0.24 }, + 'half-pixel mouse': { fast: 0.06, slow: 0.08, gain: 0.08 }, sparse: { fast: 0.95, slow: 0.29, gain: 0.29 }, } @@ -58,14 +70,19 @@ describe('every device', () => { test('a crawl is led by nothing at all', () => { // Below the gate there is nothing in the samples a fit can tell from quantisation, and a caller applying // only the growth keeps every upward wobble of it for as long as the drag lasts. Exactly nothing, not - // nearly nothing — the low-DPI mouse is the exception and it is a bug rather than a tolerance, so it is - // held on its own below. + // nearly nothing. for (const device of ALL_DEVICES) { if (device === COARSE) continue const { worstLead, kept } = replay({ path: steady(0.06, 30), device, durationMs: 4000 }) expect(worstLead).toBe(0) expect(kept).toBe(0) } + // The low-DPI mouse takes about half a second to be recognised as one, and until then its counts are + // assumed to be pixels and a stray pair of them reads as motion. What it may not do is go on doing that + // for the rest of the drag. + const coarse = replay({ path: steady(0.06, 30), device: COARSE, durationMs: 4000 }) + expect(coarse.worstLead).toBeLessThan(2) + expect(coarse.kept).toBeLessThan(12) }) test('a stalling frame clock does not throw the lead about', () => { @@ -80,7 +97,8 @@ describe('every device', () => { trackpad: 0.55, bluetooth: 1.2, 'slow radio': 1.9, - 'low-DPI mouse': 12, + 'low-DPI mouse': 0.8, + 'half-pixel mouse': 0.4, sparse: 1.7, } for (const device of ALL_DEVICES) { @@ -128,44 +146,179 @@ describe('every device', () => { }) describe('a device whose counts are wider than a pixel', () => { - // **A known weak spot, written down as numbers so that fixing it shows up as numbers.** - // - // Every noise floor in the fit is written in units of one count and one count is assumed to be about one - // pixel — `QUANTISATION_PX`. That holds for a mouse at default sensitivity, and it does not hold for a - // low-DPI one, a page at a device pixel ratio other than 1, or an OS scaling the counts on the way - // through: this device reports four pixels a step, so the bend, the turn and the trend all sit four times - // higher above floors that have not moved, and noise gets believed as motion. + // Every noise floor in the fit is written in units of one count, and what a count is worth in pixels is + // not something a browser reports: it is the mouse's resolution, the OS's scaling and the page's device + // pixel ratio multiplied together. Assumed to be one pixel, as it was, a device reporting four made every + // floor four times too low and its noise was believed as motion — the guess swung 28 degrees of heading a + // frame on a dead straight path, jumped thirty times as far in size as the same hand on a 125Hz mouse + // reporting just as often, ratcheted nine times as much, and led a crawl it should have refused. // - // What that costs, against the same hand on a 125Hz mouse that reports just as often: the lead's size - // jumps thirty times as far in a frame, it swings through 28° of heading on a path with no turn in it at - // all, and a caller keeping the growth accumulates nine times as much. None of that is a device problem — - // it is one constant that does not know what a count is worth. - // - // These are ceilings on a known-bad case, not a floor anyone is proud of. Scaling the floors by the count - // size — from the deltas the device actually reports, since nothing else knows — should collapse all four. - test('is the case the noise floors do not fit', () => { + // The count is measured now, off the only thing that knows: every delta a device reports is a whole number + // of counts, so the grid they sit on is their greatest common divisor. These are the same four figures + // afterwards, and the control below is what they are supposed to look like. + test('is measured off the deltas, and the floors follow it', () => { const fast = replay({ path: steady(0.8, 30), device: COARSE, durationMs: 2500 }) - expect(fast.worstSizeStep).toBeLessThan(5.9) - expect(fast.worstTurnStep).toBeLessThan(35) - expect(fast.kept).toBeLessThan(250) + const fine = replay({ path: steady(0.8, 30), device: USB_125, durationMs: 2500 }) + expect(fast.worstSizeStep).toBeLessThan(0.75) + expect(fast.worstTurnStep).toBeLessThan(1.5) + expect(fast.kept).toBeLessThan(65) + // Not as good as one-pixel counts — four times the quantisation is four times the quantisation — but + // within a few times of it rather than a couple of orders. + expect(fast.worstSizeStep).toBeLessThan(fine.worstSizeStep * 5) + expect(fast.worstTurnStep).toBeLessThan(fine.worstTurnStep * 5) + expect(fast.kept).toBeLessThan(fine.kept * 3) const slow = replay({ path: steady(0.15, 30), device: COARSE, durationMs: 2500 }) - expect(slow.worstTurnStep).toBeLessThan(175) + expect(slow.worstTurnStep).toBeLessThan(1) + expect(slow.worstSizeStep).toBeLessThan(0.65) + }) - // And the same constant is what lets a crawl through the speed gate: at 0.06px/ms the hand covers less - // than a count a frame, so two counts landing close together read as a hand going six times faster than - // it is. Every other device is led by exactly nothing here. - const crawl = replay({ path: steady(0.06, 30), device: COARSE, durationMs: 4000 }) - expect(crawl.worstLead).toBeLessThan(8) - expect(crawl.kept).toBeLessThan(50) + test('is not paid for in lead on a hand moving normally', () => { + // The floors going up is the point; the gate going up with them would not be. A drag at 0.8px/ms puts + // twenty counts across the window on this device, which is far too few to fit a curve to and plenty to + // say how fast the hand is going — so the travel is read with the extra uncertainty taken off it rather + // than the threshold being multiplied, and an ordinary hand keeps its lead. + const drag = replay({ path: steady(0.8, 30), device: COARSE, durationMs: 2500 }) + expect(drag.meanLead).toBeGreaterThan(14) + expect(drag.gain).toBeLessThan(0.24) + const aiming = replay({ path: steady(0.35, 25), device: COARSE, durationMs: 2500 }) + expect(aiming.meanLead).toBeGreaterThan(5) + expect(aiming.gain).toBeLessThan(0.45) }) - test('a mouse reporting just as often but in single pixels is steady', () => { - // The control for the case above: same period, same burst, same wander, one-pixel counts. - const fine = replay({ path: steady(0.8, 30), device: USB_125, durationMs: 2500 }) - expect(fine.worstSizeStep).toBeLessThan(0.2) - expect(fine.worstTurnStep).toBeLessThan(1) - expect(fine.kept).toBeLessThan(25) + test('costs the curvature, which four-pixel counts genuinely cannot carry', () => { + // **The trade, pinned so it can be argued with.** A second derivative over a window holding a handful of + // four-pixel steps sits about 1.3 times its own noise, and a floor that knows what a count is worth now + // says so — where before it read five times its noise and was believed outright. So the arc is mostly + // given up on this device and the guess falls back to the straight line, which on a circle points half a + // window's worth of turning behind the tangent. Guessing is still better than not guessing, and not by + // much. + // + // It is the honest reading of what the samples support, and it is not the end of the matter: the heading + // the arc is swept from is the window's average rather than its newest, and advancing it by the turn + // already measured would give most of this back without believing any more of the curvature. That is a + // change to how every device extrapolates, so it is a separate one, and these are the numbers it would + // have to beat. + for (const [name, gain] of [ + ['circle', 0.94], + ['circle, hand-drawn', 0.94], + ['circle, tight and fast', 1], + ['spiral', 0.81], + ['figure eight', 0.84], + ['serpentine', 0.6], + ['wide slow arc', 0.31], + ] as const) { + const scenario = SCENARIOS.find(candidate => candidate.name === name)! + const metrics = replay({ + path: scenario.path, + device: COARSE, + durationMs: scenario.durationMs, + scoreAfterMs: scenario.scoreAfterMs, + }) + expect(metrics.gain).toBeLessThanOrEqual(gain) + // Whatever it costs in lateness, it may not be bought back as shake. + expect(metrics.worstSizeStep).toBeLessThan(3) + } + }) + + test('counts finer than a pixel do not lower the floors under them', () => { + // The other side of the same assumption, and deliberately not symmetric. A device reporting half-pixel + // steps is quieter than the floors expect, and following it down would mean believing more of what a + // second derivative reads — on the strength of an estimate made from deltas. Holding at a pixel costs + // that device a little sensitivity and can break nothing, so this asks that it behaves like the + // one-pixel device of its own rate rather than better or worse than it. + const fine = replay({ path: steady(0.8, 30), device: FINE, durationMs: 2500 }) + const pixels = replay({ path: steady(0.8, 30), device: USB_500, durationMs: 2500 }) + expect(fine.worstSizeStep).toBeLessThan(pixels.worstSizeStep * 2) + expect(fine.gain).toBeLessThan(pixels.gain * 1.5) + // And a crawl is still refused outright, which is the figure a lowered floor would spend first. + const crawl = replay({ path: steady(0.06, 30), device: FINE, durationMs: 4000 }) + expect(crawl.worstLead).toBe(0) + }) + + test('one window of deltas that share a factor is not enough to move the floors', () => { + // A quarter second is long enough for a hand to be regular by accident — a steady stretch reporting + // threes and nothing else. Believed on its own that puts the floors up threefold and takes two pixels + // off every reading of the travel, which throttles a slow hand by a third for as long as it lasts. Two + // windows have to agree, and the second one here does not. + const predictor = createPointerPredictor() + let now = 1000 + const step = (deltaPx: number) => { + predictor.push(now - 13, deltaPx, 0) + const lead = predictor.update({ nowMs: now, deltaMs: FRAME, ...SHIPPED }) + now += FRAME + return lead.x + } + // 0.18px/ms, a slow deliberate drag, reported in threes for the first window and honestly after it. + for (let frame = 0; frame < 16; frame++) step(3) + let worst = Infinity + // The window straight after the regular one, which is the only one a single-window answer would reach. + for (let frame = 0; frame < 15; frame++) worst = Math.min(worst, step(frame % 3 === 0 ? 4 : 3)) + expect(worst).toBeGreaterThan(2.8) + }) + + test('is found on whichever axis the hand is using', () => { + // A hand dragging straight down reports nothing on X at all, so a count looked for on X alone would + // never be found and the whole drag would be spent assuming pixels. Both axes read the same here, which + // is the point: an axis-aligned drag is a hand steadying itself, not a special case. + const downwards = replay({ path: steady(0.8, 90), device: COARSE, durationMs: 2500 }) + const sideways = replay({ path: steady(0.8, 0), device: COARSE, durationMs: 2500 }) + expect(downwards.worstSizeStep).toBeLessThan(0.4) + expect(sideways.worstSizeStep).toBeLessThan(0.4) + expect(downwards.worstSizeStep).toBeCloseTo(sideways.worstSizeStep, 6) + }) + + test('a device that keeps stopping is not forgotten between movements', () => { + // What a count is worth is learned from windows that carried motion, and a hand that moves in bursts + // leaves most windows empty. Letting an empty one answer would drop the device back to being assumed to + // report in pixels every time it paused, and it would spend the first half second of every burst there. + const path = piecewise([hold(200), ...Array.from({ length: 6 }, () => [glide(0.5, 15, 260), hold(300)]).flat()]) + const metrics = replay({ path, device: COARSE, durationMs: 3600, scoreAfterMs: 300 }) + expect(metrics.worstTurnStep).toBeLessThan(12) + expect(metrics.kept).toBeLessThan(72) + }) + + test('a hand whose deltas happen to share a factor is not mistaken for one', () => { + // The risk the estimator runs: a hand at a steady speed on an ordinary mouse can report the same delta + // over and over, and a run of fours has a common divisor of four whatever the device underneath. Two + // windows have to agree before the floors move, and a real hand does not hold a factor that long — but + // when it does, the cost has to be small, so this is the pathological case with the wobble turned off. + const marching = createPointerPredictor() + let now = 1000 + let lead = 0 + for (let frame = 0; frame < 120; frame++) { + // Exactly four pixels every millisecond, on a device whose counts are one. + for (let s = 1; s <= 16; s++) marching.push(now - 13 - FRAME + (s * FRAME) / 16, 4, 0) + lead = marching.update({ nowMs: now, deltaMs: FRAME, ...SHIPPED }).x + now += FRAME + } + // 4px/ms over about 30ms of horizon is 120px, which the cap holds at 100 either way. What a mistaken + // count would cost here is the gate taking three pixels off the travel, which is under a percent of it. + expect(lead).toBeGreaterThan(99) + }) + + test('what a count is worth survives a reset', () => { + // It is a property of the device, not of the session: dropping it would put the floors back to assuming + // pixels for the first half second after every unlock, which is exactly when the hand is moving again. + const predictor = createPointerPredictor() + let now = 1000 + // Long enough to learn, on a hand slow enough that assuming pixels would lead it. + for (let frame = 0; frame < 90; frame++) { + if (frame % 4 === 0) predictor.push(now - 13, 4, 0) + predictor.update({ nowMs: now, deltaMs: FRAME, ...SHIPPED }) + now += FRAME + } + predictor.reset() + + let worst = 0 + for (let frame = 0; frame < 60; frame++) { + if (frame % 4 === 0) predictor.push(now - 13, 4, 0) + const lead = predictor.update({ nowMs: now, deltaMs: FRAME, ...SHIPPED }) + worst = Math.max(worst, Math.hypot(lead.x, lead.y)) + now += FRAME + } + // One count every four frames is 0.06px/ms, which is a crawl however wide the count is. + expect(worst).toBeLessThan(2) }) }) diff --git a/library/src/__tests__/predictor-shapes.test.ts b/library/src/__tests__/predictor-shapes.test.ts index 9d8c66c..ee8d59a 100644 --- a/library/src/__tests__/predictor-shapes.test.ts +++ b/library/src/__tests__/predictor-shapes.test.ts @@ -67,6 +67,10 @@ const LIMITS: Record = { // A stir: small and quick enough that a whole horizon of it sweeps past a right angle, which is where the // arc has to stop being believed rather than wrap round. stir: { gain: 0.94, size: 15, over: 15, late: 40, kick: 0, back: 0, still: 0, lead: 50 }, + // The same circle with a hand on it, so the quantisation error stops being a function of the angle and + // starts being noise. It scores like the clean one, which is what says the clean one is not being flattered + // by its own regularity. + 'circle, hand-drawn': { gain: 0.34, size: 15, over: 25, late: 55, kick: 0, back: 0, still: 0, lead: 75 }, 'circle, reversed': { gain: 0.33, size: 9.5, over: 30, late: 50, kick: 0, back: 0, still: 0, lead: 75 }, 'figure eight': { gain: 0.86, size: 8.5, over: 15, late: 65, kick: 2, back: 4.5, still: 0, lead: 50 }, spiral: { gain: 0.58, size: 8.5, over: 30, late: 75, kick: 0, back: 0, still: 0, lead: 95 }, @@ -156,6 +160,7 @@ describe('the shapes that must beat not guessing', () => { 'flick, eased stop', 'circle', 'circle, tight and fast', + 'circle, hand-drawn', 'spiral', 'point to point', 'sharp corner', diff --git a/library/src/__tests__/predictor-stops.test.ts b/library/src/__tests__/predictor-stops.test.ts index 46c5150..888accd 100644 --- a/library/src/__tests__/predictor-stops.test.ts +++ b/library/src/__tests__/predictor-stops.test.ts @@ -3,7 +3,6 @@ import { describe, expect, test } from 'bun:test' import { ALL_DEVICES, BLUETOOTH, - COARSE, DEVICES, flick, glide, @@ -170,19 +169,18 @@ describe('coming off a flick', () => { // backwards. That is a visible kick and no sample supports it: a guess may say the hand is about to // slow, it may not say the hand is about to reverse. // - // The low-DPI mouse is held to its own figure and it is not a good one — see the note on it in - // `predictor-devices.test.ts`. Its counts are four pixels wide, the turn rate is read against a floor - // written in units of one, and at the tail of a stop the held lead gets spun through a right angle by a - // turn that is entirely staircase. Four pixels of backwards lead is what that costs. + // Every device, with nothing set aside. The low-DPI mouse used to need four pixels of allowance here: + // its counts are four pixels wide, the turn rate was read against a floor written in units of one, and + // at the tail of a stop the held lead was spun through a right angle by a turn that was entirely + // staircase. With the count measured rather than assumed there is nothing left to allow. for (const device of ALL_DEVICES) { - const allowed = device === COARSE ? -6 : -0.05 for (const fallMs of [40, 150, 400]) { let worst = 0 framesOf({ path: flick(2.6, 20, 800, fallMs), device, durationMs: 800 + fallMs + 400 }).forEach(frame => { const along = frame.lead.x * Math.cos((20 * Math.PI) / 180) + frame.lead.y * Math.sin((20 * Math.PI) / 180) worst = Math.min(worst, along) }) - expect(worst).toBeGreaterThanOrEqual(allowed) + expect(worst).toBeGreaterThanOrEqual(-0.05) } } }) diff --git a/library/src/__tests__/predictor.test.ts b/library/src/__tests__/predictor.test.ts index ed0dde2..b635246 100644 --- a/library/src/__tests__/predictor.test.ts +++ b/library/src/__tests__/predictor.test.ts @@ -514,6 +514,12 @@ describe('pointer prediction', () => { // reach it. It is a guard against a fit that has run away, not against a hand. // - The horizon being cut short at the point the fit says the hand stops. Past that point the distance is // already floored at zero, so removing the cut moves nothing measurably. +// - The measured count size being held at a pixel from below. A device reporting finer than a pixel is +// quieter than the floors assume, so following it down would mean believing more of what a second +// derivative reads on the strength of an estimate — and holding costs it nothing measurable on any hand +// or device here. It is what makes measuring the count safe rather than a retune of every floor: the +// answer can only ever move them up, which is why eight of the nine devices are bit-identical to how +// they were before it existed. // // CURVE_TRUST_MS used to be a third: the synthetic hands in this file move too perfectly for the bend to // sit near its noise floor, which is the flicker it exists to stop. The tremor and coarse-count cases next diff --git a/library/src/predictor.ts b/library/src/predictor.ts index 7f4841a..9ec5991 100644 --- a/library/src/predictor.ts +++ b/library/src/predictor.ts @@ -54,6 +54,14 @@ // most, while a steady drag looks thick when there is no bend there to find. Asking how far the bend // stands above the noise instead is both more accurate and steadier, and it is the same question whatever // the device: a hand crossing a mouse pad reads the same on a wired mouse and one on a radio. +// +// **What a count is worth is measured, not assumed.** Every one of those floors is written in units of one +// count, and one count is a pixel only on a mouse at ordinary sensitivity — a low-DPI one steps four, and so +// does anything scaled on the way through by the OS or by a page that is not at a device pixel ratio of 1. +// Assumed, that put every floor four times too low on such a device and its staircase was believed as +// motion. So the count is read off the deltas, which are the only thing that knows: every delta is a whole +// number of counts, so the grid they sit on is their greatest common divisor, and unlike every other +// statistic to hand that reading does not confuse a coarse device with a fast hand. import type { PointerFrame } from './pointer.js' @@ -79,9 +87,31 @@ const TURN_SNR = 2 // A guess that bends further than this over one horizon is not a turn a hand makes; it is a fit that has // run away. const MAX_TURN_RAD = Math.PI / 2 -// About one mouse count, which is how far off a chord that ends on whole counts can point. Used to work out -// how much of a measured turn is quantisation before any of it is believed. +// How far off a chord that ends on whole counts can point, and the unit every noise floor below is written +// in: one count. What a count is worth in pointer pixels is the assumption, and it is only one pixel on a +// mouse at ordinary sensitivity — a low-DPI one moves several, and so does any device whose counts are +// scaled on the way through, by the OS or by a page that is not at a device pixel ratio of 1. Every floor is +// then several times too low, and noise gets believed as motion: measured on a mouse reporting four pixels a +// count, the guess swung 28 degrees of heading a frame on a dead straight path and led a crawl it should +// have refused outright. +// +// So the count is measured rather than assumed, and the deltas are the only thing that knows. Every delta a +// device reports is a whole number of counts, so the grid they all sit on is their greatest common divisor — +// the one reading of it that does not care how fast the hand is going. A hand crossing the pad reports 5 and +// 6 and 5, which share nothing but 1; the same hand on a low-DPI mouse reports 20 and 24 and 20, which share +// 4. Sample counts, smallest deltas and every other obvious statistic all say "coarse device" and "fast +// hand" with the same voice, and the difference is the whole point. const QUANTISATION_PX = 1 +// Long enough to hold a few dozen deltas of an ordinary drag, and two of them have to agree before the +// floors move: over one window a hand can be regular enough that its deltas share a factor by luck, and +// twice running it cannot. Learning therefore takes about half a second of motion, during which the counts +// are assumed to be pixels, which is where they started. +const COUNT_WINDOW_MS = 250 +// Wider than this is not a count size, it is a caller scaling deltas into something that is not pixels, and +// the floors have no business following it there. +const MAX_COUNT_PX = 16 +// Deltas are whole counts, so a remainder this far under one is the division coming out even. +const COUNT_EPSILON = 0.05 // How much a least-squares curvature wobbles for a given amount of that quantisation, over a window of a // given length and sample count. Tuned on a steady drag, where every bit of curvature read is noise. const ACCEL_NOISE = 20 @@ -144,6 +174,20 @@ const REVERSAL_HORIZON = 0.4 // talking about the same number or the correction is for a filter that is not there. const rampLength = (decayMs: number, confidence: number) => Math.max(decayMs * (1 + NOISE_EASE * (1 - confidence)), 1) +/** Euclid, to a tolerance, because a count is not always a whole number of pixels either. The iteration + * count is bounded because a pair of deltas that do not converge is a pair to give up on rather than to keep + * dividing: falling back to a common divisor of one is the assumption this started from. */ +const sharedDivisor = (a: number, b: number) => { + let x = a + let y = b + for (let i = 0; i < 8 && y > COUNT_EPSILON; i++) { + const remainder = x % y + x = y + y = remainder + } + return y > COUNT_EPSILON ? QUANTISATION_PX : x +} + export type PredictorOptions = { /** How many display refreshes ahead to guess. This is a count of refreshes rather than a duration * because the delay it covers is a count of refreshes: the same setting has to mean half as long on a @@ -226,11 +270,31 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { let headingY = 0 let msSinceReversal = 1e4 let reversalPeriodMs = 1e4 + // What one count of this device is worth in pointer pixels, and the two windows of deltas behind that + // answer. Believed only as far as the assumption it replaces is safe: a device reporting finer than a + // pixel is quieter than the floors expect, which costs a little sensitivity and can break nothing, so + // this only ever moves them up. + let countPx = QUANTISATION_PX + let countThisWindow = 0 + let countLastWindow = 0 + let countWindowAt = -1 const out: Lead = { x: 0, y: 0, live: 1 } + const observeCount = (delta: number) => { + const size = Math.abs(delta) + if (size < COUNT_EPSILON) return + countThisWindow = countThisWindow === 0 ? size : sharedDivisor(countThisWindow, size) + } + /** One coalesced sample: when the motion happened, and how far it went on each axis. */ const push = (timeStampMs: number, deltaX: number, deltaY: number) => { + // Once this window's deltas have nothing bigger than a pixel in common there is nothing left to learn + // from it, which is the case on every device that reports in pixels and costs them a comparison. + if (countThisWindow === 0 || countThisWindow > QUANTISATION_PX) { + observeCount(deltaX) + observeCount(deltaY) + } cumulativeX += deltaX cumulativeY += deltaY times[head] = timeStampMs @@ -317,7 +381,14 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { // the hand is going lets every spike through amplified. Under a crawl the lead is dropped entirely — // a caller that keeps what it is given keeps every upward wobble too, so jitter at a slow steady // drag accumulates for as long as the drag lasts. - const travel = Math.hypot(travelX, travelY) + // Travel is only known to within a count, and the gate below is a speed tuned against a device whose + // counts are pixels — so the extra uncertainty of a wider one comes off the travel before it is read. + // Left in, a hand well under the gate reads several times over it on a low-DPI mouse whenever two counts + // happen to land close together, and a crawl gets led by a fit that is entirely staircase. Taking it off + // the travel rather than putting it on the threshold is what keeps an ordinary drag on that device led: + // four counts across a window are few enough that no curve can be fitted to them and plenty to say how + // fast the hand is going. + const travel = Math.max(0, Math.hypot(travelX, travelY) - (countPx - QUANTISATION_PX)) const speed = span > 0 ? travel / span : 0 const trust = Math.max(0, Math.min(1, (speed - SLOW_PX_PER_MS) / (SURE_PX_PER_MS - SLOW_PX_PER_MS))) if (trust === 0) return false @@ -360,7 +431,7 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { // mouse looks exactly as thin as a slow device, and that is the moment the bend matters most. // How much a least-squares curvature wobbles for this much quantisation over a window this long and // this well populated. Every judgement about whether a bend is real is asked against it. - const bendFloor = (ACCEL_NOISE * QUANTISATION_PX) / (span * span * Math.sqrt(used)) + const bendFloor = (ACCEL_NOISE * countPx) / (span * span * Math.sqrt(used)) const measured = bendFloor > 0 ? Math.max(0, Math.min(1, (Math.hypot(bendX, bendY) / bendFloor - 1) / (CURVE_SNR - 1))) : 0 // A bend sitting near its own noise floor answers this differently every frame, and the answer decides @@ -438,7 +509,7 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { speedTrend += ((speedNow - lastSpeed) / deltaMs - speedTrend) * (1 - Math.exp(-deltaMs / TREND_MS)) } lastSpeed = speedNow - const trendFloor = (TREND_NOISE * QUANTISATION_PX) / (span * Math.sqrt(used) * TREND_MS) + const trendFloor = (TREND_NOISE * countPx) / (span * Math.sqrt(used) * TREND_MS) if (speedTrend < 0 && trendFloor > 0) { const believed = Math.max(0, Math.min(1, (-speedTrend / trendFloor - 1) / (CURVE_SNR - 1))) alongAcceleration = Math.min(alongAcceleration, speedTrend * believed) @@ -511,7 +582,7 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { // it is a flat tax: it removes as much from a turn a hand really made as from one only the counts // made up, and on a circle a hand does make that came to a third of the turn — which is most of // why the guess used to sit outside the circle. - const floor = QUANTISATION_PX * (1 / newLength + 1 / oldLength) + const floor = countPx * (1 / newLength + 1 / oldLength) const angle = Math.atan2(cross, dot) const believed = floor > 0 ? Math.max(0, Math.min(1, (Math.abs(angle) / floor - 1) / (TURN_SNR - 1))) : 0 turn = (angle / (span / 2)) * believed * curveTrust @@ -551,6 +622,17 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { // real refresh change to walk there over a few frames and a stall to barely register. if (deltaMs > 0) frameMs += (Math.min(deltaMs, frameMs * MAX_FRAME_STEP) - frameMs) * FRAME_SMOOTHING + // What a count of this device is worth, settled a window at a time. A window that carried no motion has + // nothing to say and is simply extended, so a hand sitting still cannot make the answer expire. + if (countWindowAt < 0) countWindowAt = nowMs + else if (nowMs - countWindowAt >= COUNT_WINDOW_MS && countThisWindow > 0) { + const agreed = countLastWindow > 0 ? Math.min(countThisWindow, countLastWindow) : QUANTISATION_PX + countPx = Math.min(MAX_COUNT_PX, Math.max(QUANTISATION_PX, agreed)) + countLastWindow = countThisWindow + countThisWindow = 0 + countWindowAt = nowMs + } + const newestT = count > 0 ? times[(head - 1 + CAPACITY) % CAPACITY]! : 0 const hadInput = count > 0 && newestT > lastSeenNewest // Whether these samples landed on a window that was still standing, or on one the last stop had @@ -716,6 +798,9 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { headingY = 0 msSinceReversal = 1e4 reversalPeriodMs = 1e4 + // What a count is worth is not part of the session. It is a property of the device, it does not change + // across a menu, and dropping it would put the noise floors back to assuming pixels for the first half + // second after every unlock — which is exactly when the hand is moving again. } return { push, pushFrame, update, reset } From c291ad19dc4334b2896ce318cf9420a86207ac8d Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 04:02:45 +0000 Subject: [PATCH 3/6] fix: a corner is not a stop, and tell them apart by the other axis MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reported as bad jumps on sharp turns. Traced with a frame-by-frame trace of the fit: on the frame a right-angle corner enters the window, the guess asks for a lead of exactly zero, from a hand that has not slowed by a pixel per second. Seen along the heading alone a corner and a stop are the same event — the speed the hand had is going away — and a corner takes it away harder and sooner than any stop does. Braking is deliberately believed on thin evidence and acted on at once, so both readings of it fire, and the ramp-lag subtraction then takes the whole guess with it. This is the mistake the module header already describes for a per-axis fit at the extremum of a circle, one level in, and it has the same answer: look at the other axis. A hand coming to a stop pushes straight back along its path; a hand turning pushes sideways. Measured through a right angle at speed, a hundredth of the bend pointed back along the path and the rest of it across. So both brake readings are gated by how much of the bend points back along the path. Below its own noise floor the bend has no direction worth reading and the gate reads as one, which leaves every device too thin to carry a curvature braking exactly as it did — that is the case the speed trend exists for and it is worth 16px of stop overshoot on the sparsest device. The guess through a corner now jumps 2-3x less in size: 6.4 -> 4.5px wired, 6.4 -> 3.0 on bluetooth, and the same on every curve. 140 cells of the board improve against 70 that give up a little, all of them small and on turning paths, where less braking means slightly more overshoot. Three things measured and NOT changed, each written down where it will be found again: - Believing a bend sooner when it stands far above its noise. The parabola knows the new heading the moment the corner is in the window and the 150ms settling is what stops it being used, but acting on it concentrates the whole catch-up into one frame: the camera moved 102px on a frame where the hand moved 42. - Easing the lead in polar rather than world axes, so a direction change does not cross through a shorter vector. It moves the corner by nothing and costs 305 cells elsewhere. - Clamping how far the held lead may be spun in one frame. It is currently a right angle, which looks alarming and measures as nothing. The corner itself turns out to be close to what a 40ms window allows: against the same hand going straight, a right angle adds 0-11px to the worst uneven frame, and on a mouse reporting in clumps it adds nothing at all. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HctBLnHDXwx1XURcvnEB9T --- README.md | 7 + library/src/__tests__/harness.ts | 87 ++++++++++- .../src/__tests__/predictor-devices.test.ts | 62 ++++++++ .../src/__tests__/predictor-shapes.test.ts | 136 ++++++++++++++---- library/src/__tests__/predictor-stops.test.ts | 85 +++++++++++ library/src/predictor.ts | 32 ++++- 6 files changed, 373 insertions(+), 36 deletions(-) diff --git a/README.md b/README.md index 5d31613..88c6a2b 100644 --- a/README.md +++ b/README.md @@ -223,6 +223,13 @@ window, the second differences first derivatives over several frames and still r its own noise. Whichever brakes harder wins, so a thick window is unaffected and a thin one gets a stop it would otherwise never have seen. +**A corner is not a stop, and telling them apart needs the other axis.** Along the heading the hand was +going, a corner takes its speed away faster than any stop does — so braking, which is deliberately believed +on thin evidence and acted on at once, fires on a hand that has not slowed by a pixel per second. What +separates them is where the bend points: a hand coming to a stop pushes straight back along its path, and a +hand turning pushes sideways. Through a right angle at speed, a hundredth of the bend pointed back along the +path and the rest of it across. + **Most of the overshoot was never in the fit — it was the ramp behind it.** A first-order lag trails its target by its own time constant times how fast that target is moving, and coming off a flick the target falls faster than at any other time. That lag is a known quantity rather than something to be differentiated out of diff --git a/library/src/__tests__/harness.ts b/library/src/__tests__/harness.ts index dc419a0..33d702b 100644 --- a/library/src/__tests__/harness.ts +++ b/library/src/__tests__/harness.ts @@ -11,6 +11,7 @@ // the figure measured on the next machine. Any number in a test file was read off this rig and given room. import { createPointerPredictor, DEFAULT_DECAY_MS, DEFAULT_LEAD_FRAMES, DEFAULT_MAX_LEAD_PX } from '../predictor.js' +import { createLeadRatchet } from '../ratchet.js' export const FRAME = 16.7 /** What the newest sample's age measures at at 60Hz, per the example rig. */ @@ -167,6 +168,18 @@ export type Metrics = { * direction. A guess still holding lead when the hand stops has to give every pixel of it back, and this * is what that costs. */ phantom: number + /** How unevenly the picture moves, in pixels: the worst frame's difference between how far the picture + * travelled and how far it should have. A hand at a constant speed makes a picture that moves the same + * distance every frame, and the eye reads any departure from that as a jump — which is not the same + * question as how far the guess is from the truth, and not the same question as how much the lead moved. + * A lead that jumps about to cancel a device reporting in clumps is doing its job. + * + * Two of them, because there are two ways to spend a lead and they judge it differently. The first is a + * caller with a true position to add the lead to. The second is a locked pointer, which has none and can + * only ever be pushed further along — `createLeadRatchet`, and the shape most of this library's callers + * are. A guess can be smooth for one and not the other. */ + worstJump: number + worstRatchetJump: number worstLead: number meanLead: number /** How far the guessed position sits from the origin, averaged. Only means something on a circle, where @@ -216,6 +229,7 @@ export const replay = (options: ReplayOptions): Metrics => { } = options const predictor = createPointerPredictor() + const ratchet = createLeadRatchet() let rndState = seed const rnd = () => (rndState = (rndState * 1103515245 + 12345) & 0x7fffffff) / 0x7fffffff @@ -245,6 +259,14 @@ export const replay = (options: ReplayOptions): Metrics => { let previousRelativeAngle: number | null = null let previousHandAngle: number | null = null let phantom = 0 + let worstJump = 0 + let worstRatchetJump = 0 + let previousTruth: Vec | null = null + let previousReported: Vec | null = null + let ratchetX = 0 + let ratchetY = 0 + let previousRatchetX = 0 + let previousRatchetY = 0 let worstLead = 0 let leadSum = 0 let worstOvershoot = 0 @@ -375,6 +397,26 @@ export const replay = (options: ReplayOptions): Metrics => { phantom += Math.hypot(movedX, movedY) } } + // How far the picture moved this frame against how far it should have. Both cameras are fed the same + // guess; they differ in what they are allowed to do with it. + const grew = ratchet.step(lead) + ratchetX += reportedX - (previousReported?.x ?? reportedX) + grew.x + ratchetY += reportedY - (previousReported?.y ?? reportedY) + grew.y + if (previousTruth && previousShown && previousReported) { + const wanted = Math.hypot(truth.x - previousTruth.x, truth.y - previousTruth.y) + worstJump = Math.max( + worstJump, + Math.abs(Math.hypot(shownX - previousShown.x, shownY - previousShown.y) - wanted), + ) + worstRatchetJump = Math.max( + worstRatchetJump, + Math.abs(Math.hypot(ratchetX - previousRatchetX, ratchetY - previousRatchetY) - wanted), + ) + } + previousRatchetX = ratchetX + previousRatchetY = ratchetY + previousTruth = truth + previousReported = { x: reportedX, y: reportedY } withSum += (shownX - truth.x) ** 2 + (shownY - truth.y) ** 2 withoutSum += (integratedX - truth.x) ** 2 + (integratedY - truth.y) ** 2 radialSum += Math.hypot(shownX, shownY) @@ -415,6 +457,8 @@ export const replay = (options: ReplayOptions): Metrics => { backwards, worstBackwards, phantom, + worstJump, + worstRatchetJump, worstLead, meanLead: leadSum / frames, meanRadius: radialSum / frames, @@ -448,6 +492,8 @@ export const worstAcross = (devices: readonly Device[], options: Omit (degrees * Math.PI) / 180 export type Segment = { durationMs: number; at: (u: number) => Vec } /** Segments laid end to end, each starting where the last one finished, holding the final position for ever - * after — which is what a hand that has stopped does, and what the score has to be able to ask about. */ + * after — which is what a hand that has stopped does, and what the score has to be able to ask about. + * + * A path meant to be still moving when the run ends therefore has to carry a segment past the end of it, or + * the last frame reads as a hand stopping dead and every figure about jumps picks that up instead of the + * motion it was asked about. */ export const piecewise = (segments: readonly Segment[]): Path => { const starts: number[] = [] const offsets: Vec[] = [] @@ -746,13 +796,44 @@ export const SCENARIOS: readonly Scenario[] = [ scenario( 'sharp corner', 'a right angle at full speed with no slowing: the heading inverts on one axis and holds on the other', - piecewise([glide(1.2, 0, 900), glide(1.2, 90, 900)]), + piecewise([glide(1.2, 0, 900), glide(1.2, 90, 1100)]), 1800, ), + scenario( + 'corner, 45 degrees', + 'a change of direction with no slowing at all, which is where a fit reads a stop that is not there', + piecewise([glide(2.5, 0, 700), glide(2.5, 45, 900)]), + 1400, + ), + scenario( + 'corner, 90 degrees', + 'the same, square: half the window says one heading and half says the other', + piecewise([glide(2.5, 0, 700), glide(2.5, 90, 900)]), + 1400, + ), + scenario( + 'corner, 135 degrees', + 'past square, where the travel across the window is shorter than either leg of it', + piecewise([glide(2.5, 0, 700), glide(2.5, 135, 900)]), + 1400, + ), + scenario( + 'corner, slow', + 'the same square corner taken gently, where the window holds far fewer counts to read it from', + piecewise([glide(0.5, 0, 900), glide(0.5, 90, 1100)]), + 1800, + ), + scenario( + 'zigzag, sharp and fast', + 'corner after corner with no straight between them to recover in', + piecewise([...Array.from({ length: 10 }, (_, i) => glide(2.2, i % 2 === 0 ? 60 : -60, 150))]), + 1200, + 200, + ), scenario( 'rounded corner', 'the same corner taken round a radius, which is what a hand actually does', - piecewise([glide(1.2, 0, 900), arc(1.2, 140, 90, 0), glide(1.2, 90, 700)]), + piecewise([glide(1.2, 0, 900), arc(1.2, 140, 90, 0), glide(1.2, 90, 900)]), 1900, ), scenario('circle', 'a hand going round, where a per-axis fit reads a stop twice a lap', circle(120, 400), 3000), diff --git a/library/src/__tests__/predictor-devices.test.ts b/library/src/__tests__/predictor-devices.test.ts index cb55862..756c681 100644 --- a/library/src/__tests__/predictor-devices.test.ts +++ b/library/src/__tests__/predictor-devices.test.ts @@ -135,6 +135,68 @@ describe('every device', () => { } }) + test('the picture moves as unevenly as the device reports, and no worse', () => { + // **The biggest jump in this library is not in this library.** A hand at a dead constant speed should + // make a picture that moves the same distance every frame; on every device that coalesces, it does not, + // because the device's report interval and the display's refresh interval do not divide into each other. + // A trackpad reporting every 11ms into a 16.7ms frame delivers two samples on some frames and one on + // others, so the picture steps 55px and then 28px with the hand moving perfectly steadily. + // + // The lead neither causes it nor removes it: the two columns below are the picture with the guess + // applied and the reported position on its own, and they are the same figure. What the lead *could* do + // is remove it — the newest sample says exactly how stale the reported position is, so the motion + // missing from it is arithmetic rather than a guess. It is not done, for a reason worth writing down: + // that correction has to alternate every frame, and half this library's callers spend the lead through + // `createLeadRatchet`, which keeps growth and never gives it back. Feeding an alternating term through a + // ratchet turns it into permanent drift — measured at thirteen times the accumulation on a trackpad. + // Fixing it properly means smoothing the delivery on the way in, where a ratchet never sees it, and that + // is a change to the buffer rather than to the guess. + const BEAT: Record = { + wired: { withLead: 1.2, bare: 0.9 }, + '8KHz mouse': { withLead: 1.2, bare: 0.9 }, + '500Hz mouse': { withLead: 2.3, bare: 2 }, + '125Hz mouse': { withLead: 6.8, bare: 6.4 }, + trackpad: { withLead: 5.8, bare: 5.3 }, + bluetooth: { withLead: 13.3, bare: 13.4 }, + 'slow radio': { withLead: 14.6, bare: 14.5 }, + 'low-DPI mouse': { withLead: 7.3, bare: 6.7 }, + 'half-pixel mouse': { withLead: 3.1, bare: 2.8 }, + sparse: { withLead: 13.5, bare: 13.4 }, + } + for (const device of ALL_DEVICES) { + const limits = BEAT[device.name]! + const path = steady(0.8, 30) + const guessed = replay({ path, device, durationMs: 2500 }) + // What the same picture does with no lead applied at all: the beat by itself. + let bare = 0 + let previousReported: { x: number; y: number } | null = null + let previousTruth: { x: number; y: number } | null = null + replay({ + path, + device, + durationMs: 2500, + onFrame: frame => { + const truth = path(frame.frameAt + FRAME) + if (previousReported && previousTruth && frame.frameAt > 600) { + bare = Math.max( + bare, + Math.abs( + Math.hypot(frame.reported.x - previousReported.x, frame.reported.y - previousReported.y) - + Math.hypot(truth.x - previousTruth.x, truth.y - previousTruth.y), + ), + ) + } + previousReported = { x: frame.reported.x, y: frame.reported.y } + previousTruth = truth + }, + }) + expect(bare).toBeLessThan(limits.bare + 0.05) + expect(guessed.worstJump).toBeLessThan(limits.withLead) + // The guess may not make it worse than the device already is by more than a fraction of a pixel. + expect(guessed.worstJump).toBeLessThan(bare + 1) + } + }) + test('a device nobody would call fast still gets a guess worth having', () => { // One report every 25ms with 4ms of wander: most frames see nothing at all and the window holds two or // three samples. It cannot be as good as a wired mouse and it must not be useless. diff --git a/library/src/__tests__/predictor-shapes.test.ts b/library/src/__tests__/predictor-shapes.test.ts index ee8d59a..c928da3 100644 --- a/library/src/__tests__/predictor-shapes.test.ts +++ b/library/src/__tests__/predictor-shapes.test.ts @@ -25,6 +25,10 @@ import { DEVICES, replay, SCENARIOS, worstAcross, type Metrics } from './harness // back all of that added up over the run. // still screen motion over frames where the hand is not moving at all. // lead how far out the guess may ever be. +// jump the worst frame where the picture moved further, or less far, than it should have. A hand at a +// constant speed makes a picture that moves the same distance every frame, and the eye reads any +// departure from that as a jump — which is the artefact a user actually reports, and not the same +// question as how far the guess is from the truth or how much the lead moved. type Limits = { gain: number size: number @@ -34,67 +38,133 @@ type Limits = { back: number still: number lead: number + jump: number } const LIMITS: Record = { // Straight lines. Nothing here is hard to predict, so these are almost entirely about steadiness: the hand // is doing nothing new, and every pixel the guess moves by is invented. - 'steady fast': { gain: 0.35, size: 0.75, over: 15, late: 25, kick: 0, back: 0, still: 0, lead: 30 }, - 'steady slow': { gain: 0.76, size: 0.3, over: 0, late: 8.5, kick: 0.1, back: 1, still: 0, lead: 3 }, + 'steady fast': { gain: 0.35, size: 0.75, over: 15, late: 25, kick: 0, back: 0, still: 0, lead: 30, jump: 20 }, + 'steady slow': { gain: 0.76, size: 0.3, over: 0, late: 8.5, kick: 0.1, back: 1, still: 0, lead: 3, jump: 4.5 }, // Below the speed gate there is no lead at all, so this one is exact rather than generous: a single pixel // of guess here is quantisation being mistaken for motion, and a caller keeping the growth keeps it. - crawl: { gain: 1.06, size: 0, over: 0, late: 5.5, kick: 0, back: 0, still: 0, lead: 0 }, - 'steady on-axis': { gain: 0.35, size: 0.8, over: 15, late: 25, kick: 0, back: 0, still: 0, lead: 30 }, + crawl: { gain: 1.06, size: 0, over: 0, late: 5.5, kick: 0, back: 0, still: 0, lead: 0, jump: 1.5 }, + 'steady on-axis': { gain: 0.35, size: 0.8, over: 15, late: 25, kick: 0, back: 0, still: 0, lead: 30, jump: 20 }, // Far past the cap, so what is being judged is that the ceiling holds and nothing rings against it. - whip: { gain: 0.44, size: 0.7, over: 35, late: 150, kick: 0.01, back: 0.01, still: 0, lead: 101 }, - 'ramp up': { gain: 0.4, size: 4, over: 45, late: 85, kick: 0, back: 0, still: 0, lead: 101 }, + whip: { gain: 0.44, size: 0.7, over: 35, late: 150, kick: 0.01, back: 0.01, still: 0, lead: 101, jump: 90 }, + 'ramp up': { gain: 0.4, size: 4, over: 45, late: 85, kick: 0, back: 0, still: 0, lead: 101, jump: 85 }, // Stops. The one moment the guess is asked to give everything back, and the moment it is furthest from // being able to see it coming — `still` is what that costs on screen. - 'flick, snapped stop': { gain: 0.41, size: 25, over: 60, late: 65, kick: 20, back: 35, still: 40, lead: 95 }, - 'flick, eased stop': { gain: 0.4, size: 20, over: 45, late: 65, kick: 9, back: 30, still: 30, lead: 95 }, - 'abrupt stop': { gain: 0.55, size: 25, over: 30, late: 50, kick: 0, back: 0, still: 85, lead: 75 }, - 'abrupt start': { gain: 0.47, size: 10, over: 30, late: 150, kick: 0, back: 0, still: 0, lead: 75 }, - 'stop and go': { gain: 0.72, size: 20, over: 35, late: 85, kick: 15, back: 35, still: 250, lead: 55 }, - 'point to point': { gain: 0.63, size: 6, over: 20, late: 50, kick: 3.5, back: 15, still: 55, lead: 45 }, + 'flick, snapped stop': { + gain: 0.41, + size: 25, + over: 60, + late: 65, + kick: 20, + back: 35, + still: 40, + lead: 95, + jump: 70, + }, + 'flick, eased stop': { gain: 0.4, size: 20, over: 45, late: 65, kick: 9.5, back: 30, still: 30, lead: 95, jump: 55 }, + 'abrupt stop': { gain: 0.55, size: 25, over: 30, late: 50, kick: 0, back: 0, still: 85, lead: 75, jump: 80 }, + 'abrupt start': { gain: 0.47, size: 10, over: 30, late: 150, kick: 0, back: 0, still: 0, lead: 75, jump: 60 }, + 'stop and go': { gain: 0.72, size: 20, over: 35, late: 85, kick: 15, back: 35, still: 250, lead: 55, jump: 60 }, + 'point to point': { gain: 0.63, size: 6, over: 20, late: 50, kick: 3.5, back: 15, still: 55, lead: 45, jump: 30 }, // Turns. A per-axis fit reads the extremum of every one of these as a hand that is stopping, which is // exactly backwards, so these are the cases the 2D fit exists for. - 'sharp corner': { gain: 0.49, size: 8.5, over: 20, late: 55, kick: 0, back: 0, still: 0, lead: 45 }, - 'rounded corner': { gain: 0.49, size: 20, over: 20, late: 50, kick: 0, back: 0, still: 65, lead: 45 }, - circle: { gain: 0.33, size: 9.5, over: 30, late: 50, kick: 0, back: 0, still: 0, lead: 75 }, - 'circle, tight and fast': { gain: 0.32, size: 8.5, over: 25, late: 30, kick: 0.25, back: 0.25, still: 0, lead: 60 }, + 'sharp corner': { gain: 0.49, size: 6, over: 20, late: 55, kick: 0, back: 0, still: 0, lead: 45, jump: 35 }, + 'corner, 45 degrees': { gain: 0.42, size: 4, over: 35, late: 85, kick: 0, back: 0, still: 0, lead: 90, jump: 60 }, + 'corner, 90 degrees': { gain: 0.55, size: 15, over: 35, late: 150, kick: 0, back: 0, still: 0, lead: 90, jump: 65 }, + 'corner, 135 degrees': { + gain: 0.65, + size: 20, + over: 35, + late: 200, + kick: 40, + back: 40, + still: 0, + lead: 90, + jump: 75, + }, + 'corner, slow': { gain: 0.5, size: 2.5, over: 7, late: 25, kick: 0, back: 0, still: 0, lead: 20, jump: 15 }, + 'zigzag, sharp and fast': { + gain: 0.89, + size: 15, + over: 50, + late: 150, + kick: 35, + back: 50, + still: 0, + lead: 65, + jump: 60, + }, + 'rounded corner': { gain: 0.47, size: 2, over: 20, late: 50, kick: 0, back: 0, still: 0, lead: 45, jump: 30 }, + circle: { gain: 0.33, size: 8.5, over: 30, late: 50, kick: 0, back: 0, still: 0, lead: 75, jump: 50 }, + 'circle, tight and fast': { + gain: 0.32, + size: 7.5, + over: 25, + late: 30, + kick: 0.4, + back: 0.4, + still: 0, + lead: 60, + jump: 45, + }, // A stir: small and quick enough that a whole horizon of it sweeps past a right angle, which is where the // arc has to stop being believed rather than wrap round. - stir: { gain: 0.94, size: 15, over: 15, late: 40, kick: 0, back: 0, still: 0, lead: 50 }, + stir: { gain: 0.94, size: 15, over: 15, late: 40, kick: 0, back: 0, still: 0, lead: 50, jump: 35 }, // The same circle with a hand on it, so the quantisation error stops being a function of the angle and // starts being noise. It scores like the clean one, which is what says the clean one is not being flattered // by its own regularity. - 'circle, hand-drawn': { gain: 0.34, size: 15, over: 25, late: 55, kick: 0, back: 0, still: 0, lead: 75 }, - 'circle, reversed': { gain: 0.33, size: 9.5, over: 30, late: 50, kick: 0, back: 0, still: 0, lead: 75 }, - 'figure eight': { gain: 0.86, size: 8.5, over: 15, late: 65, kick: 2, back: 4.5, still: 0, lead: 50 }, - spiral: { gain: 0.58, size: 8.5, over: 30, late: 75, kick: 0, back: 0, still: 0, lead: 95 }, - serpentine: { gain: 0.69, size: 4, over: 15, late: 45, kick: 0, back: 0, still: 0, lead: 30 }, + 'circle, hand-drawn': { gain: 0.34, size: 10, over: 25, late: 50, kick: 0, back: 0, still: 0, lead: 75, jump: 50 }, + 'circle, reversed': { gain: 0.33, size: 8.5, over: 30, late: 50, kick: 0, back: 0, still: 0, lead: 75, jump: 50 }, + 'figure eight': { gain: 0.86, size: 7.5, over: 15, late: 65, kick: 2.5, back: 3.5, still: 0, lead: 50, jump: 35 }, + spiral: { gain: 0.58, size: 8.5, over: 30, late: 75, kick: 0, back: 0, still: 0, lead: 95, jump: 60 }, + serpentine: { gain: 0.69, size: 3.5, over: 15, late: 45, kick: 0, back: 0, still: 0, lead: 30, jump: 25 }, // A turn slow enough and wide enough that the angle between the window's two halves barely clears what // quantisation alone produces. Believing it too readily and not at all are both visible here. - 'wide slow arc': { gain: 0.36, size: 0.45, over: 4.5, late: 15, kick: 0, back: 0, still: 0, lead: 15 }, + 'wide slow arc': { gain: 0.36, size: 0.45, over: 4.5, late: 15, kick: 0, back: 0, still: 0, lead: 15, jump: 8 }, // Reversals. Nothing in a window says a turn is coming when the hand is at its fastest, so the guess is // bounded by how long this hand has lately been going between turning round rather than by evidence. // Gain over 1 is honest here: there is nothing to win, and the job is not to lose much. - zigzag: { gain: 1.06, size: 15, over: 20, late: 75, kick: 30, back: 90, still: 0, lead: 30 }, - 'shake, 8Hz': { gain: 1.11, size: 9, over: 45, late: 50, kick: 30, back: 200, still: 40, lead: 15 }, - 'shake, fast and small': { gain: 1.11, size: 6, over: 30, late: 30, kick: 25, back: 350, still: 55, lead: 6.5 }, - 'overshoot and correct': { gain: 0.55, size: 15, over: 35, late: 85, kick: 7, back: 15, still: 35, lead: 65 }, - 'target acquire': { gain: 0.54, size: 20, over: 40, late: 100, kick: 20, back: 20, still: 30, lead: 75 }, + zigzag: { gain: 1.06, size: 15, over: 20, late: 75, kick: 30, back: 90, still: 0, lead: 30, jump: 35 }, + 'shake, 8Hz': { gain: 1.11, size: 9, over: 45, late: 50, kick: 30, back: 200, still: 40, lead: 15, jump: 40 }, + 'shake, fast and small': { + gain: 1.11, + size: 6, + over: 30, + late: 30, + kick: 25, + back: 350, + still: 55, + lead: 6.5, + jump: 25, + }, + 'overshoot and correct': { + gain: 0.55, + size: 15, + over: 35, + late: 85, + kick: 7, + back: 15, + still: 35, + lead: 65, + jump: 45, + }, + 'target acquire': { gain: 0.54, size: 20, over: 40, late: 100, kick: 20, back: 20, still: 30, lead: 75, jump: 55 }, // Braking is believed on thin evidence and acted on at once, which is right for a stop and is exactly // wrong for a hand that is only pausing between two throws. This is the case that pays for that choice. - 'double flick': { gain: 0.55, size: 25, over: 40, late: 150, kick: 20, back: 50, still: 45, lead: 85 }, + 'double flick': { gain: 0.55, size: 25, over: 40, late: 150, kick: 20, back: 50, still: 45, lead: 85, jump: 65 }, // A hand is never still. The wobble is the size of the counts underneath it, which is where a second // derivative reads it as motion and a caller keeping the growth turns it into rotation. - 'tremor on a drag': { gain: 0.39, size: 2, over: 5.5, late: 15, kick: 0.25, back: 2, still: 0, lead: 15 }, - 'tremor at rest': { gain: 1.06, size: 2, over: 2.5, late: 3.5, kick: 2, back: 4.5, still: 15, lead: 1.5 }, + 'tremor on a drag': { gain: 0.39, size: 2, over: 5.5, late: 15, kick: 0.25, back: 2, still: 0, lead: 15, jump: 15 }, + 'tremor at rest': { gain: 1.06, size: 2, over: 2.5, late: 3.5, kick: 2, back: 4.5, still: 15, lead: 1.5, jump: 2 }, } describe('movement shapes', () => { @@ -123,6 +193,10 @@ describe('movement shapes', () => { expect(worst.backwards).toBeLessThanOrEqual(limits.back) expect(worst.phantom).toBeLessThanOrEqual(limits.still) expect(worst.worstLead).toBeLessThanOrEqual(limits.lead) + // Both ways of spending a lead: a caller with a true position to add it to, and a locked pointer that + // can only ever be pushed further along. + expect(worst.worstJump).toBeLessThanOrEqual(limits.jump) + expect(worst.worstRatchetJump).toBeLessThanOrEqual(limits.jump) }) } diff --git a/library/src/__tests__/predictor-stops.test.ts b/library/src/__tests__/predictor-stops.test.ts index 888accd..035a7d7 100644 --- a/library/src/__tests__/predictor-stops.test.ts +++ b/library/src/__tests__/predictor-stops.test.ts @@ -10,6 +10,7 @@ import { piecewise, replay, SPARSE, + worstAcross, TRACKPAD, WIRED, type ReplayOptions, @@ -145,6 +146,16 @@ describe('coming off a flick', () => { }) } + test('a device too thin to carry a curvature still gets its stop', () => { + // The window here holds two or three samples, so the bend has no direction worth reading and the stop is + // seen entirely in the fitted speed falling across frames. Telling a turn from a stop by where the bend + // points must not take that away: below its own noise floor the bend says nothing, and a reading that + // says nothing may not be allowed to argue. Letting it, this stop went from 41 to 56px past the hand. + const stop = replay({ path: flick(2.6, 20, 800, 150), device: SPARSE, durationMs: 1350 }) + expect(stop.worstOvershoot).toBeLessThan(45) + expect(stop.phantom).toBeLessThan(48) + }) + test('the lead is already coming down while the hand is still moving', () => { // The whole reason for fitting a curve rather than smoothing a velocity. An estimator that cannot see a // stop coming is still near its peak when the hand is already still; this asks where the guess is at the @@ -254,6 +265,80 @@ describe('starting again', () => { }) }) +describe('turning is not stopping', () => { + // Seen along the heading alone, a corner and a stop are the same event: the speed in the direction the + // hand was going collapses. A corner collapses it harder and sooner than any real stop, and braking is + // deliberately believed on thin evidence and acted on at once — so a hand turning a right angle at speed + // had the guess asked for *nothing at all* on the frame the corner entered the window. Zero, from a hand + // that had not slowed by a single pixel per second. + // + // What tells them apart is the other axis, which is the same answer this library gives for a per-axis fit + // at the extremum of a circle. A hand coming to a stop pushes straight back along its path; a hand turning + // pushes sideways. Measured through a right angle at speed, a hundredth of the bend pointed back along the + // path and the rest of it across. + const corner = (turnDeg: number, speed = 2.5) => piecewise([glide(speed, 0, 700), glide(speed, turnDeg, 900)]) + + // How far the guess may dip through the turn, against what it held going in. The sharper the corner the + // more of it is honest — the hand's own lead across a 135 degree corner really is short, because the two + // legs cancel — so each angle is held to its own figure. + const DIP: Record = { 45: 0.85, 90: 0.62, 135: 0.28 } + + test('a corner does not read as a hand that has stopped', () => { + // The guess through the turn, against the guess before it. A corner costs some of it — the window holds + // two headings and the fitted speed across it is the chord — but it may not cost all of it. + for (const device of DEVICES) { + for (const turnDeg of [45, 90, 135]) { + const frames = framesOf({ path: corner(turnDeg), device, durationMs: 1400 }) + const before = Math.max(...frames.filter(frame => frame.frameAt < 690).map(size)) + const through = frames.filter(frame => frame.frameAt >= 690 && frame.frameAt < 820) + expect(Math.min(...through.map(size))).toBeGreaterThan(before * DIP[turnDeg]!) + } + } + }) + + test('the guess does not jump in size going round a corner', () => { + // How much the size of the guess may move in one frame while the hand turns. Reading the corner as + // braking put this at 6.4px on a hand whose speed never changed. + const SIZE: Record = { 45: 4, 90: 9.5, 135: 17 } + for (const turnDeg of [45, 90, 135]) { + const worst = worstAcross(DEVICES, { path: corner(turnDeg), durationMs: 1400 }) + expect(worst.worstSizeStep).toBeLessThan(SIZE[turnDeg]!) + } + }) + + test('a corner adds little to what the device is already doing', () => { + // The honest scoring of a corner. A picture that moves unevenly is what a user reports as a jump, and + // most of it on any device slower than a wired mouse is the device reporting in clumps rather than + // anything the corner did — so what a corner may add is measured against the same hand going straight. + // The figure is what the corner is worth on top of that, and it is small. + const ADDED: Record = { wired: 9, trackpad: 6, bluetooth: 4, 'slow radio': 13 } + for (const device of DEVICES) { + const turning = replay({ path: corner(90), device, durationMs: 1400 }) + const straight = replay({ path: piecewise([glide(2.5, 0, 1600)]), device, durationMs: 1400 }) + expect(turning.worstJump - straight.worstJump).toBeLessThan(ADDED[device.name]!) + } + }) + + test('a corner is not led round further than the hand went', () => { + // The arc extrapolation swings the guess through the turn it believes the path has. A corner is not an + // arc: the heading changes once and then holds, and a turn rate read off it is a rate that will not + // continue. Bending the guess through the corner and out the other side is a visible kick. + for (const device of ALL_DEVICES) { + for (const turnDeg of [45, 90, 135]) { + const frames = framesOf({ path: corner(turnDeg), device, durationMs: 1400 }) + const after = frames.filter(frame => frame.frameAt > 780) + const heading = (turnDeg * Math.PI) / 180 + for (const frame of after) { + if (size(frame) < 1) continue + // Never further round than the new heading, give or take what the ramp is still carrying. + const across = -frame.lead.x * Math.sin(heading) + frame.lead.y * Math.cos(heading) + expect(across).toBeLessThan(size(frame) * 0.75) + } + } + } + }) +}) + describe('the picture never fights the hand', () => { // The one error a camera cannot hide is the view walking backwards while the hand is moving forwards. It // costs nothing to be late; it costs the illusion to reverse. diff --git a/library/src/predictor.ts b/library/src/predictor.ts index 9ec5991..33ef1d3 100644 --- a/library/src/predictor.ts +++ b/library/src/predictor.ts @@ -20,6 +20,14 @@ // the inward part of the guess, which is why circling used to throw the lead outwards. In 2D none of them // fire, because a circling hand never slows and never turns around. It turns. // +// The same mistake waits one level in, where the path is split into along and across. Along the heading +// alone a corner and a stop are again the same event — the speed the hand had is going away — and a corner +// takes it away harder and sooner than any stop does, so a hand turning a right angle at speed had the +// guess asked for nothing at all on the frame the corner entered the window. The answer is the same as +// before: look at the other axis. A hand coming to a stop pushes straight back along its path; a hand +// turning pushes sideways, and through a right angle a hundredth of the bend pointed back along the path +// and the rest of it across. +// // So the extrapolation follows an arc rather than a straight line. Speed and heading come from the fit; // the turn rate is measured off the path, because over a wide arc a parabola fits the curvature badly but // the heading of the window's newer half against its older half is just two chords. The arc degenerates to @@ -480,6 +488,20 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { // going at a dead constant speed. Subtract what quantisation alone would produce, so constant speed // reads as constant and a flick still reads as a flick. const bendAlong = bendX * tx + bendY * ty + // How much of the bend is the hand slowing rather than the hand turning. Seen along the heading alone, + // a corner is indistinguishable from a stop: the speed in the direction the hand was going collapses, + // and it collapses harder and sooner than any real stop does. It is the same blind spot a per-axis fit + // has at the extremum of a circle, one level in — and it has the same answer, which is to look at the + // other axis. A hand coming to a stop pushes straight back along its path. A hand turning pushes + // sideways, and a corner is almost entirely sideways: measured through a right angle at speed, a + // hundredth of the bend pointed back along the path and the rest of it across. + // + // Where the bend is too small to have a direction at all this says nothing and must not be allowed to: + // below its own noise floor it reads as one, which leaves every device too thin to carry a curvature + // braking exactly as it did before. That is the case the speed trend exists for, and this may not take + // it away. + const bendMagnitude = Math.hypot(bendX, bendY) + const slowing = bendMagnitude > bendFloor ? Math.abs(bendAlong) / bendMagnitude : 1 const alongRaw = bendAlong * curveTrust let alongAcceleration = Math.sign(alongRaw) * Math.max(0, Math.abs(alongRaw) - bendFloor) @@ -491,7 +513,9 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { // Only ever taken when it brakes harder than the settled reading does, so this can subtract lead and // never add it: a stop is seen sooner or exactly as before, never later. const brakeMeasured = - bendAlong < 0 && bendFloor > 0 ? Math.max(0, Math.min(1, (-bendAlong / bendFloor - 1) / (CURVE_SNR - 1))) : 0 + bendAlong < 0 && bendFloor > 0 + ? Math.max(0, Math.min(1, (-bendAlong / bendFloor - 1) / (CURVE_SNR - 1))) * slowing + : 0 brakeConfidence += (brakeMeasured - brakeConfidence) * (1 - Math.exp(-deltaMs / BRAKE_TRUST_MS)) const brakeTrust = brakeConfidence if (bendAlong < 0) { @@ -510,8 +534,12 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { } lastSpeed = speedNow const trendFloor = (TREND_NOISE * countPx) / (span * Math.sqrt(used) * TREND_MS) + // The fitted speed falls at a corner as surely as it falls at a stop, and for a reason that has nothing + // to do with the hand: the straight line through a window that bends is the chord of it, and a chord is + // shorter than the path. So this is held to the same question as the curvature above — through a right + // angle the fitted speed dropped by a third with the hand at a dead constant speed throughout. if (speedTrend < 0 && trendFloor > 0) { - const believed = Math.max(0, Math.min(1, (-speedTrend / trendFloor - 1) / (CURVE_SNR - 1))) + const believed = Math.max(0, Math.min(1, (-speedTrend / trendFloor - 1) / (CURVE_SNR - 1))) * slowing alongAcceleration = Math.min(alongAcceleration, speedTrend * believed) } From 3add0ae26fdc09c462409b8408889bf1260b356e Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 04:24:13 +0000 Subject: [PATCH 4/6] fix: do not guess further ahead than the path turns a quarter circle over MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reported as small circular motions coming apart, and worse the higher `leadFrames` goes. Measured, that is exactly the shape of it: a 25px circle stirred every 110ms put the guess 14px off the circle at one refresh of lead, 42 at two, 58 at three and 65 at four — two diameters from anywhere the hand ever goes — and moved the picture 60px in a frame getting there. Every extra refresh cost more than the last. Past about a quarter turn of horizon the answer is mostly the turn rate, which is the noisiest thing in the fit, and the chord being asked for has stopped growing with the horizon and started coming back towards the hand — so a wobble in the rate moves the guess around the circle rather than along it. Worse, the arc sweep was clamped at a right angle while the distance went on being the whole arc length, so a guess that should have been a short chord left the circle altogether. So the horizon is bounded by how far the path turns over it, which is the same rule the reversal horizon already applies to a hand doubling back. The small circles now read the same at one refresh of lead and at four: 9px off instead of 65, and 13px of uneven picture motion instead of 60. Two things the measurements insisted on: - The rate is settled over 100ms before it bounds anything. Bounding on the raw reading hands the horizon the turn estimate's own wobble, and a horizon that wobbles is a lead whose length wobbles, which is the picture moving backwards under the hand — 2.3px of it on a tight circle, the artefact this exists to remove rather than one to add. - The bound is eased in rather than applied as a ceiling. A hard ceiling has a knee, and a path whose turn sits on the knee steps across it and back every frame; the eased shape leaves a gentle turn alone, takes a sixth off where the bound is reached, and settles on it past that. A quarter turn was chosen by sweeping: at a half turn the small circles are still 23px off, and at an eighth an ordinary tight circle starts losing lead it should have. 158 cells of the board improve against 72 that give up a little, all of the latter being more lateness on the two tightest circles, which is the trade being made on purpose. Wide circles keep every pixel of lead the setting asks for — they never turn far enough to reach the bound — and a wired mouse stays within 6px of one at four refreshes of lead. On a device reporting four or five samples a window they still drift outside a wide circle, by up to 39px at four refreshes; that is the turn being believed less than it is on short chords rather than the horizon, it is unchanged by this, and it is pinned with the numbers a fix would have to beat. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HctBLnHDXwx1XURcvnEB9T --- README.md | 6 + .../src/__tests__/predictor-shapes.test.ts | 110 +++++++++++++++--- library/src/predictor.ts | 110 ++++++++++++------ 3 files changed, 177 insertions(+), 49 deletions(-) diff --git a/README.md b/README.md index 88c6a2b..0c211a7 100644 --- a/README.md +++ b/README.md @@ -264,6 +264,12 @@ The rest is mostly about not believing noise: grew a fifth on it, which is the one frame the eye gets before freshness starts discounting it. - **A hitch is not a change of refresh interval**, and the age of the newest sample is averaged rather than taken raw — a few milliseconds of horizon is pixels of lead on a hand doing nothing new. +- **Nor further ahead than the path turns a quarter circle over.** Past that the answer is mostly the turn + rate — the noisiest thing in the fit — and the chord being asked for has stopped growing with the horizon + and started coming back towards the hand, so a wobble in the rate moves the guess around the circle rather + than along it. Unbounded, a hand stirring a small circle came apart as `leadFrames` went up: at three + refreshes of lead the guess sat 58px off a 25px circle. Bounded, it reads the same at one refresh and at + four. - **The guess is never led past its own reversal.** A quick shake turns round before the horizon is out, and at the moment it is fastest there is no acceleration to give the turn away. diff --git a/library/src/__tests__/predictor-shapes.test.ts b/library/src/__tests__/predictor-shapes.test.ts index c928da3..4d2555d 100644 --- a/library/src/__tests__/predictor-shapes.test.ts +++ b/library/src/__tests__/predictor-shapes.test.ts @@ -1,6 +1,6 @@ import { describe, expect, test } from 'bun:test' -import { DEVICES, replay, SCENARIOS, worstAcross, type Metrics } from './harness.js' +import { circle, DEVICES, replay, SCENARIOS, WIRED, worstAcross, type Metrics } from './harness.js' // Every shape a hand makes, scored on every device, against a floor that was measured rather than hoped for. // @@ -51,8 +51,8 @@ const LIMITS: Record = { crawl: { gain: 1.06, size: 0, over: 0, late: 5.5, kick: 0, back: 0, still: 0, lead: 0, jump: 1.5 }, 'steady on-axis': { gain: 0.35, size: 0.8, over: 15, late: 25, kick: 0, back: 0, still: 0, lead: 30, jump: 20 }, // Far past the cap, so what is being judged is that the ceiling holds and nothing rings against it. - whip: { gain: 0.44, size: 0.7, over: 35, late: 150, kick: 0.01, back: 0.01, still: 0, lead: 101, jump: 90 }, - 'ramp up': { gain: 0.4, size: 4, over: 45, late: 85, kick: 0, back: 0, still: 0, lead: 101, jump: 85 }, + whip: { gain: 0.44, size: 0.7, over: 35, late: 150, kick: 0.01, back: 0.01, still: 0, lead: 150, jump: 90 }, + 'ramp up': { gain: 0.4, size: 4, over: 45, late: 85, kick: 0, back: 0, still: 0, lead: 150, jump: 85 }, // Stops. The one moment the guess is asked to give everything back, and the moment it is furthest from // being able to see it coming — `still` is what that costs on screen. @@ -102,28 +102,28 @@ const LIMITS: Record = { jump: 60, }, 'rounded corner': { gain: 0.47, size: 2, over: 20, late: 50, kick: 0, back: 0, still: 0, lead: 45, jump: 30 }, - circle: { gain: 0.33, size: 8.5, over: 30, late: 50, kick: 0, back: 0, still: 0, lead: 75, jump: 50 }, + circle: { gain: 0.34, size: 7.5, over: 25, late: 50, kick: 0, back: 0, still: 0, lead: 70, jump: 50 }, 'circle, tight and fast': { - gain: 0.32, - size: 7.5, - over: 25, - late: 30, - kick: 0.4, - back: 0.4, + gain: 0.39, + size: 5, + over: 20, + late: 35, + kick: 0.75, + back: 0.75, still: 0, - lead: 60, + lead: 55, jump: 45, }, // A stir: small and quick enough that a whole horizon of it sweeps past a right angle, which is where the // arc has to stop being believed rather than wrap round. - stir: { gain: 0.94, size: 15, over: 15, late: 40, kick: 0, back: 0, still: 0, lead: 50, jump: 35 }, + stir: { gain: 0.95, size: 15, over: 0, late: 40, kick: 0, back: 0, still: 0, lead: 35, jump: 30 }, // The same circle with a hand on it, so the quantisation error stops being a function of the angle and // starts being noise. It scores like the clean one, which is what says the clean one is not being flattered // by its own regularity. - 'circle, hand-drawn': { gain: 0.34, size: 10, over: 25, late: 50, kick: 0, back: 0, still: 0, lead: 75, jump: 50 }, - 'circle, reversed': { gain: 0.33, size: 8.5, over: 30, late: 50, kick: 0, back: 0, still: 0, lead: 75, jump: 50 }, + 'circle, hand-drawn': { gain: 0.35, size: 9, over: 25, late: 50, kick: 0, back: 0, still: 0, lead: 75, jump: 50 }, + 'circle, reversed': { gain: 0.34, size: 7.5, over: 25, late: 50, kick: 0, back: 0, still: 0, lead: 70, jump: 50 }, 'figure eight': { gain: 0.86, size: 7.5, over: 15, late: 65, kick: 2.5, back: 3.5, still: 0, lead: 50, jump: 35 }, - spiral: { gain: 0.58, size: 8.5, over: 30, late: 75, kick: 0, back: 0, still: 0, lead: 95, jump: 60 }, + spiral: { gain: 0.58, size: 8, over: 30, late: 75, kick: 0, back: 0, still: 0, lead: 95, jump: 60 }, serpentine: { gain: 0.69, size: 3.5, over: 15, late: 45, kick: 0, back: 0, still: 0, lead: 30, jump: 25 }, // A turn slow enough and wide enough that the angle between the window's two halves barely clears what // quantisation alone produces. Believing it too readily and not at all are both visible here. @@ -142,7 +142,7 @@ const LIMITS: Record = { kick: 25, back: 350, still: 55, - lead: 6.5, + lead: 5.5, jump: 25, }, 'overshoot and correct': { @@ -221,6 +221,84 @@ describe('movement shapes', () => { }) }) +describe('raising the lead does not throw a small circle apart', () => { + // A hand stirring a small circle quickly is the one shape where a whole horizon is most of a lap, and it + // used to come apart as `leadFrames` went up: at three refreshes of lead the guess sat 58px off a 25px + // circle — two diameters from anywhere the hand ever goes — and moved 60px in a frame getting there. Every + // extra refresh cost more than the last, which is the shape of a guess being extrapolated past the point + // anything about it is still true. + // + // The horizon is bounded by how far the path turns over it now, so the answer stops growing with the + // setting instead: what a longer lead cannot buy, it no longer spends. Each row is the worst reading + // across the four devices, and the figures hold at one refresh of lead and at four alike. + const CIRCLES = [ + { r: 25, periodMs: 110, off: 10, jump: 24 }, + { r: 15, periodMs: 150, off: 3, jump: 15 }, + { r: 25, periodMs: 200, off: 3.5, jump: 19 }, + ] + + for (const { r, periodMs, off, jump } of CIRCLES) { + test(`a ${r}px circle every ${periodMs}ms stays on the circle at any lead`, () => { + for (const leadFrames of [1, 2, 3, 4]) { + const run = worstAcross(DEVICES, { path: circle(r, periodMs), durationMs: 3000, leadFrames }) + expect(Math.abs(run.meanRadius - r)).toBeLessThan(off) + expect(run.worstJump).toBeLessThan(jump) + } + }) + } + + test('and the lead setting barely moves any of it', () => { + // The figures above would pass on a guess that was equally bad at every setting. This is the part that + // says the setting stopped mattering: four refreshes of lead may not be meaningfully worse than one. + for (const { r, periodMs } of CIRCLES) { + const one = worstAcross(DEVICES, { path: circle(r, periodMs), durationMs: 3000, leadFrames: 1 }) + const four = worstAcross(DEVICES, { path: circle(r, periodMs), durationMs: 3000, leadFrames: 4 }) + expect(Math.abs(four.meanRadius - r)).toBeLessThan(Math.abs(one.meanRadius - r) + 1.5) + expect(four.worstJump).toBeLessThan(one.worstJump * 1.15 + 1) + } + }) + + test('a circle a hand actually draws is untouched by the bound', () => { + // The bound is on the turn, not on turning. A wide circle turns a fraction of a right angle over one + // horizon, so it never reaches the bound and keeps every pixel of lead the setting asks for — the lead + // has to go on growing with `leadFrames`, or this has quietly become a cap on how far anything is led. + for (const [r, periodMs] of [ + [60, 400], + [200, 900], + ] as const) { + const leads = [1, 2, 4].map( + leadFrames => replay({ path: circle(r, periodMs), device: WIRED, durationMs: 3000, leadFrames }).meanLead, + ) + expect(leads[1]!).toBeGreaterThan(leads[0]! * 1.4) + expect(leads[2]!).toBeGreaterThan(leads[1]! * 1.25) + // And it stays on the circle while doing it. + for (const leadFrames of [1, 2, 4]) { + const run = replay({ path: circle(r, periodMs), device: WIRED, durationMs: 3000, leadFrames }) + expect(Math.abs(run.meanRadius - r)).toBeLessThan(8) + } + } + }) + + test('a wide circle on a device that reports slowly still drifts outside it', () => { + // **Known, unfixed, and a different mechanism.** The bound above is on how far the path turns over the + // horizon, and a wide circle never turns far enough to reach it. What puts the guess outside one of + // those is the turn being believed less than it is: on a window holding four or five samples the two + // chords it is read from are short, the quantisation floor they are judged against is correspondingly + // large, and an under-believed turn bends the arc less than the path bends, which leaves along the + // tangent. A wired mouse on the same circle sits within 6px of it at four refreshes of lead. + // + // Fixing it means believing a turn on thinner evidence, which is the same knob that decides whether a + // slow drag shakes — a different change with a different blast radius. These are the numbers it would + // have to beat. + const DRIFT: Record = { trackpad: 34, bluetooth: 24, 'slow radio': 39 } + for (const device of DEVICES) { + const run = replay({ path: circle(60, 400), device, durationMs: 3000, leadFrames: 4 }) + const allowed = DRIFT[device.name] ?? 3 + expect(run.meanRadius - 60).toBeLessThan(allowed) + } + }) +}) + describe('the shapes that must beat not guessing', () => { // A gain over 1 is only ever acceptable where there is nothing to win: a hand reversing faster than the // horizon, or one moving too slowly to be led at all. Everywhere else the guess has to earn its place, and diff --git a/library/src/predictor.ts b/library/src/predictor.ts index 33ef1d3..66f7c3a 100644 --- a/library/src/predictor.ts +++ b/library/src/predictor.ts @@ -33,6 +33,14 @@ // the heading of the window's newer half against its older half is just two chords. The arc degenerates to // the straight line as the turn rate goes to zero, so straight motion is bit-identical. // +// **And the horizon is bounded by how far the path turns over it.** Past about a quarter turn the answer is +// mostly the turn rate, which is the noisiest thing here, and the chord being asked for has stopped growing +// with the horizon and started coming back towards the hand — so a wobble in the rate moves the guess around +// the circle rather than along it. Unbounded, a hand stirring a small circle quickly came apart as the lead +// setting went up: at three refreshes the guess sat 58px off a 25px circle, two diameters from anywhere the +// hand goes. Bounded, the answer stops growing with the setting instead of getting worse with it — what a +// longer lead cannot buy, it no longer spends. +// // Freshness is the only stop signal once the hand has already stopped. A stopped pointer sends nothing, so // the lead fades out over STOP_MS rather than sailing on at the last fitted speed. // @@ -95,6 +103,18 @@ const TURN_SNR = 2 // A guess that bends further than this over one horizon is not a turn a hand makes; it is a fit that has // run away. const MAX_TURN_RAD = Math.PI / 2 +// And how far the path may turn over the horizon before the horizon itself is cut short. Past this the +// answer is mostly the turn rate, which is the noisiest thing in the fit, and the chord being asked for has +// stopped growing with the horizon and started coming back towards the hand — so a wobble in the rate moves +// the guess around the circle rather than along it. Measured on small circles stirred quickly, which is +// where one horizon is most of a lap: at a quarter turn the guess sat 9px off a 25px circle where trusting a +// half turn put it 23px off and trusting the lot put it 58px off, two diameters from anywhere the hand goes. +// Tighter than this starts costing an ordinary tight circle its lead, and buys little. +const MAX_TRUSTED_TURN_RAD = Math.PI / 4 +// How long to settle the turn rate the cap above is read from. +const TURN_RATE_MS = 100 +// How sharply the horizon gives way once the path turns further than that over it. +const TURN_CAP_KNEE = 4 // How far off a chord that ends on whole counts can point, and the unit every noise floor below is written // in: one count. What a count is worth in pointer pixels is the assumption, and it is only one pixel on a // mouse at ordinary sensitivity — a low-DPI one moves several, and so does any device whose counts are @@ -268,8 +288,10 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { // previous speed to difference against, which is not the same as a previous speed of zero. let lastSpeed = -1 let speedTrend = 0 - // Believed turn rate of the path, rad/ms, so the eased lead can be carried around with the heading. + // Believed turn rate of the path, rad/ms, so the eased lead can be carried around with the heading, and + // how fast the path has lately been turning, which is what bounds how far ahead the turn may be trusted. let carriedTurn = 0 + let turnRate = 0 // A hand that has lately been doubling back, and how long it went between doing so. At the moment a // shake is fastest it is locally indistinguishable from a hand crossing the pad: same speed, and no // acceleration, because the turn is still ahead. Only the memory that it turned round a moment ago @@ -552,6 +574,38 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { const givenUp = Math.min(0, curvedX * tx + curvedY * ty - (straightX * tx + straightY * ty)) const speedAlong = Math.max(0, speedNow + givenUp * Math.max(0, brakeTrust - curveTrust)) + // Turn rate straight off the path: the heading of the newer half of the window against the heading of + // its older half, which is two chords rather than a second derivative. + let turn = 0 + if (used >= 3 && span > 0) { + const middle = (newest - (used >> 1) + CAPACITY) % CAPACITY + const oldest = (newest - (used - 1) + CAPACITY) % CAPACITY + const newX = newestX - xs[middle]! + const newY = newestY - ys[middle]! + const oldX = xs[middle]! - xs[oldest]! + const oldY = ys[middle]! - ys[oldest]! + const newLength = Math.hypot(newX, newY) + const oldLength = Math.hypot(oldX, oldY) + if (newLength > 1e-6 && oldLength > 1e-6) { + const cross = (oldX * newY - oldY * newX) / (newLength * oldLength) + const dot = (oldX * newX + oldY * newY) / (newLength * oldLength) + // A chord ending on whole counts can point up to about half a count either side of where it + // really points, so short chords carry an angle error that is pure quantisation. Off an axis the + // two axes cross their count boundaries at different moments, the path becomes a zig-zag about + // the true line, and a turn read straight off it wanders by more than any hand does. + // + // So the floor decides how far to believe the angle rather than being taken out of it. Subtracting + // it is a flat tax: it removes as much from a turn a hand really made as from one only the counts + // made up, and on a circle a hand does make that came to a third of the turn — which is most of + // why the guess used to sit outside the circle. + const floor = countPx * (1 / newLength + 1 / oldLength) + const angle = Math.atan2(cross, dot) + const believed = floor > 0 ? Math.max(0, Math.min(1, (Math.abs(angle) / floor - 1) / (TURN_SNR - 1))) : 0 + turn = (angle / (span / 2)) * believed * curveTrust + carriedTurn = turn + } + } + // How hard to smooth the lead. Read off the bend alone this asks whether the path is straight, and // answers a dead-steady cruise with the longest ramp there is — four times decayMs at the exact moment // the hand is most likely to come off the flick and the target about to move faster than at any other @@ -563,6 +617,27 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { // guess is extrapolating through a reversal it has no way to see, and on a quick shake it ran further // than the whole width of the motion. let horizon = Math.min(horizonMs, reversalPeriodMs * REVERSAL_HORIZON) + // Nor further ahead than the path turns a quarter circle over. Past that the answer is mostly the turn + // rate — the noisiest thing in the fit — and the chord it is asking for has stopped growing with the + // horizon and started shrinking back towards the hand, so a wobble in the rate moves the guess about the + // circle rather than along it. It is the same rule as the reversal above, for a hand that is turning + // rather than doubling back, and it binds on exactly the case that has no answer: a small circle stirred + // quickly, where one horizon is most of a lap. Left uncapped the sweep was clamped to a quarter turn + // while the distance went on being the whole arc, so the guess left the circle altogether — 58px off a + // 25px circle at three frames of lead, which is two diameters away from any point the hand visits. + // Against a settled reading of the rate, not this frame's. Capping on the raw one hands the horizon the + // turn estimate's own frame-to-frame wobble, and a horizon that wobbles is a lead whose length wobbles, + // which is the picture moving backwards under the hand — 2.3px of it on a tight circle, which is the + // artefact this exists to remove rather than one to introduce. How fast a hand is going round changes + // over tenths of a second; the noise on reading it changes every frame. + turnRate += (Math.abs(turn) - turnRate) * (1 - Math.exp(-deltaMs / TURN_RATE_MS)) + if (turnRate > 0) { + // Eased rather than cut, because a hard ceiling has a knee, and a path whose turn sits on the knee + // steps across it and back every frame. The shape below leaves a gentle turn alone, takes a sixth off + // at the point the cap is reached, and settles on the cap itself past it. + const sweep = (turnRate * horizon) / MAX_TRUSTED_TURN_RAD + horizon /= (1 + sweep ** TURN_CAP_KNEE) ** (1 / TURN_CAP_KNEE) + } if (alongAcceleration < 0 && speedAlong + alongAcceleration * horizon < 0) { horizon = -speedAlong / alongAcceleration } @@ -586,38 +661,6 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { const rampLag = alongAcceleration < 0 ? alongAcceleration * horizon * rampMs : 0 const distance = Math.max(0, travelled + rampLag) - // Turn rate straight off the path: the heading of the newer half of the window against the heading of - // its older half, which is two chords rather than a second derivative. - let turn = 0 - if (used >= 3 && span > 0) { - const middle = (newest - (used >> 1) + CAPACITY) % CAPACITY - const oldest = (newest - (used - 1) + CAPACITY) % CAPACITY - const newX = newestX - xs[middle]! - const newY = newestY - ys[middle]! - const oldX = xs[middle]! - xs[oldest]! - const oldY = ys[middle]! - ys[oldest]! - const newLength = Math.hypot(newX, newY) - const oldLength = Math.hypot(oldX, oldY) - if (newLength > 1e-6 && oldLength > 1e-6) { - const cross = (oldX * newY - oldY * newX) / (newLength * oldLength) - const dot = (oldX * newX + oldY * newY) / (newLength * oldLength) - // A chord ending on whole counts can point up to about half a count either side of where it - // really points, so short chords carry an angle error that is pure quantisation. Off an axis the - // two axes cross their count boundaries at different moments, the path becomes a zig-zag about - // the true line, and a turn read straight off it wanders by more than any hand does. - // - // So the floor decides how far to believe the angle rather than being taken out of it. Subtracting - // it is a flat tax: it removes as much from a turn a hand really made as from one only the counts - // made up, and on a circle a hand does make that came to a third of the turn — which is most of - // why the guess used to sit outside the circle. - const floor = countPx * (1 / newLength + 1 / oldLength) - const angle = Math.atan2(cross, dot) - const believed = floor > 0 ? Math.max(0, Math.min(1, (Math.abs(angle) / floor - 1) / (TURN_SNR - 1))) : 0 - turn = (angle / (span / 2)) * believed * curveTrust - carriedTurn = turn - } - } - const swept = Math.max(-MAX_TURN_RAD, Math.min(MAX_TURN_RAD, turn * horizon)) let along = distance let across = 0 @@ -826,6 +869,7 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { headingY = 0 msSinceReversal = 1e4 reversalPeriodMs = 1e4 + turnRate = 0 // What a count is worth is not part of the session. It is a property of the device, it does not change // across a menu, and dropping it would put the noise floors back to assuming pixels for the first half // second after every unlock — which is exactly when the hand is moving again. From aa1d4f501a9fb38291ecc8be981c1206d48a7136 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 9 Aug 2026 04:48:03 +0000 Subject: [PATCH 5/6] fix: believe a turn on its own evidence, and on several readings of it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reported as the marker orbiting outside a 120px circle. Measured on one at a range of speeds, worst across the four quoted devices: 1.9px out at 300ms a lap, 19.7 at 600ms, 13.2 at 900ms. Slow circles were worse than fast ones, which is the tell — it is not the extrapolation running away, it is the turn not being believed. Two gates were multiplying. The turn is read from two chords, which is a good measurement even on a window holding four samples, and it was then multiplied by how far the parabola was believed — a worse measurement of the same thing, and one that sits near a half on exactly that window. Two half-open gates left the arc bending at a quarter of the rate the hand was turning, and a lead that lags the heading on a circle points outwards. That is the marker sitting outside the circle. So the turn is judged on its own evidence now. That evidence is the average of several readings rather than one: a hand going round turns the same way on every frame while quantisation flips sign, so averaging keeps the turn whole, takes the noise down with the root of how many independent windows went into it, and is more evidence rather than a lower bar. Windows overlap, so it is windows that are counted and not frames. Two more things the measurements insisted on: - A reading that contradicts a turn already worth believing is taken at once rather than averaged in. A hand can stop going round one way and start going round the other — a figure eight does it twice a lap — and averaging across that says it is doing neither. The test is against a believed turn rather than against zero, or a straight path, whose readings flip sign every frame, would take the quick road every time and the average would follow the noise it exists to cancel. - The floor a turn is judged against is what quantisation typically does to a chord's heading, not the worst it could ever do. A position on a grid is off by a twelfth of a count in variance, which is about four tenths of the worst case; used as the divisor of a signal to noise ratio, the worst case asked for a five-sigma turn before believing one, which a 120px circle at a comfortable speed does not produce. A twentieth below the figure here is where a stalled frame's noise starts reading as a turn, so it sits just the safe side of that. The 120px circle now reads within 3px of itself at every speed on every device that reports in single pixels, against 20px before. 351 cells of the board improve against 205 that give up a little. What it costs is what believing a turn always costs a locked pointer: rotation is growth on both axes in turn, and a ratchet keeps growth. On shapes whose curvature reverses that accumulation roughly triples — a figure eight goes from 18 to 68px over three seconds. Every other shape's is unchanged, including a plain circle and a straight drag. `kept` is now a column of the shapes table so the next change to any of this has to say what it did to that. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HctBLnHDXwx1XURcvnEB9T --- README.md | 7 + .../src/__tests__/predictor-shapes.test.ts | 359 ++++++++++++++++-- library/src/__tests__/predictor-stops.test.ts | 6 +- library/src/predictor.ts | 67 +++- 4 files changed, 393 insertions(+), 46 deletions(-) diff --git a/README.md b/README.md index 0c211a7..8697f94 100644 --- a/README.md +++ b/README.md @@ -264,6 +264,13 @@ The rest is mostly about not believing noise: grew a fifth on it, which is the one frame the eye gets before freshness starts discounting it. - **A hitch is not a change of refresh interval**, and the age of the newest sample is averaged rather than taken raw — a few milliseconds of horizon is pixels of lead on a hand doing nothing new. +- **The turn is believed on its own evidence, and on several readings of it.** It used to be gated by how far + the parabola was believed as well — a worse measurement of the same thing, which sits near a half on a + window holding four samples. Two half-open gates left the arc bending at a quarter of the rate the hand was + turning, and a lead that lags the heading on a circle points outwards: the guess orbited 20px outside a + 120px circle. Averaging the readings before judging them is more evidence rather than a lower bar — a hand + going round turns the same way on every frame and quantisation flips sign — and a reading that contradicts + a turn already believed is taken at once, because a hand can start going round the other way. - **Nor further ahead than the path turns a quarter circle over.** Past that the answer is mostly the turn rate — the noisiest thing in the fit — and the chord being asked for has stopped growing with the horizon and started coming back towards the hand, so a wobble in the rate moves the guess around the circle rather diff --git a/library/src/__tests__/predictor-shapes.test.ts b/library/src/__tests__/predictor-shapes.test.ts index 4d2555d..e378986 100644 --- a/library/src/__tests__/predictor-shapes.test.ts +++ b/library/src/__tests__/predictor-shapes.test.ts @@ -25,6 +25,9 @@ import { circle, DEVICES, replay, SCENARIOS, WIRED, worstAcross, type Metrics } // back all of that added up over the run. // still screen motion over frames where the hand is not moving at all. // lead how far out the guess may ever be. +// kept what a locked-pointer caller accumulates, which keeps the growth of the lead and never gives it +// back. Believing a turn rotates the guess, and rotation is growth on both axes in turn, so this is +// where every decision about how far to believe a turn is paid for. // jump the worst frame where the picture moved further, or less far, than it should have. A hand at a // constant speed makes a picture that moves the same distance every frame, and the eye reads any // departure from that as a jump — which is the artefact a user actually reports, and not the same @@ -39,20 +42,65 @@ type Limits = { still: number lead: number jump: number + kept: number } const LIMITS: Record = { // Straight lines. Nothing here is hard to predict, so these are almost entirely about steadiness: the hand // is doing nothing new, and every pixel the guess moves by is invented. - 'steady fast': { gain: 0.35, size: 0.75, over: 15, late: 25, kick: 0, back: 0, still: 0, lead: 30, jump: 20 }, - 'steady slow': { gain: 0.76, size: 0.3, over: 0, late: 8.5, kick: 0.1, back: 1, still: 0, lead: 3, jump: 4.5 }, + 'steady fast': { + gain: 0.35, + size: 0.75, + over: 15, + late: 25, + kick: 0, + back: 0, + still: 0, + lead: 30, + jump: 20, + kept: 60, + }, + 'steady slow': { + gain: 0.76, + size: 0.3, + over: 0, + late: 8.5, + kick: 0.1, + back: 1, + still: 0, + lead: 3, + jump: 4.5, + kept: 8, + }, // Below the speed gate there is no lead at all, so this one is exact rather than generous: a single pixel // of guess here is quantisation being mistaken for motion, and a caller keeping the growth keeps it. - crawl: { gain: 1.06, size: 0, over: 0, late: 5.5, kick: 0, back: 0, still: 0, lead: 0, jump: 1.5 }, - 'steady on-axis': { gain: 0.35, size: 0.8, over: 15, late: 25, kick: 0, back: 0, still: 0, lead: 30, jump: 20 }, + crawl: { gain: 1.06, size: 0, over: 0, late: 5.5, kick: 0, back: 0, still: 0, lead: 0, jump: 1.5, kept: 0 }, + 'steady on-axis': { + gain: 0.35, + size: 0.8, + over: 15, + late: 25, + kick: 0, + back: 0, + still: 0, + lead: 30, + jump: 20, + kept: 60, + }, // Far past the cap, so what is being judged is that the ceiling holds and nothing rings against it. - whip: { gain: 0.44, size: 0.7, over: 35, late: 150, kick: 0.01, back: 0.01, still: 0, lead: 150, jump: 90 }, - 'ramp up': { gain: 0.4, size: 4, over: 45, late: 85, kick: 0, back: 0, still: 0, lead: 150, jump: 85 }, + whip: { + gain: 0.44, + size: 0.7, + over: 35, + late: 150, + kick: 0.05, + back: 0.05, + still: 0, + lead: 150, + jump: 90, + kept: 150, + }, + 'ramp up': { gain: 0.4, size: 4, over: 45, late: 85, kick: 0, back: 0, still: 0, lead: 150, jump: 85, kept: 150 }, // Stops. The one moment the guess is asked to give everything back, and the moment it is furthest from // being able to see it coming — `still` is what that costs on screen. @@ -66,84 +114,243 @@ const LIMITS: Record = { still: 40, lead: 95, jump: 70, + kept: 150, + }, + 'flick, eased stop': { + gain: 0.4, + size: 20, + over: 45, + late: 65, + kick: 9.5, + back: 30, + still: 30, + lead: 95, + jump: 55, + kept: 150, + }, + 'abrupt stop': { + gain: 0.55, + size: 25, + over: 30, + late: 50, + kick: 0, + back: 0, + still: 85, + lead: 75, + jump: 80, + kept: 95, + }, + 'abrupt start': { + gain: 0.47, + size: 10, + over: 30, + late: 150, + kick: 0, + back: 0, + still: 0, + lead: 75, + jump: 60, + kept: 95, + }, + 'stop and go': { + gain: 0.72, + size: 20, + over: 35, + late: 85, + kick: 15, + back: 35, + still: 250, + lead: 55, + jump: 60, + kept: 350, + }, + 'point to point': { + gain: 0.63, + size: 6, + over: 20, + late: 50, + kick: 3.5, + back: 15, + still: 55, + lead: 45, + jump: 30, + kept: 55, }, - 'flick, eased stop': { gain: 0.4, size: 20, over: 45, late: 65, kick: 9.5, back: 30, still: 30, lead: 95, jump: 55 }, - 'abrupt stop': { gain: 0.55, size: 25, over: 30, late: 50, kick: 0, back: 0, still: 85, lead: 75, jump: 80 }, - 'abrupt start': { gain: 0.47, size: 10, over: 30, late: 150, kick: 0, back: 0, still: 0, lead: 75, jump: 60 }, - 'stop and go': { gain: 0.72, size: 20, over: 35, late: 85, kick: 15, back: 35, still: 250, lead: 55, jump: 60 }, - 'point to point': { gain: 0.63, size: 6, over: 20, late: 50, kick: 3.5, back: 15, still: 55, lead: 45, jump: 30 }, // Turns. A per-axis fit reads the extremum of every one of these as a hand that is stopping, which is // exactly backwards, so these are the cases the 2D fit exists for. - 'sharp corner': { gain: 0.49, size: 6, over: 20, late: 55, kick: 0, back: 0, still: 0, lead: 45, jump: 35 }, - 'corner, 45 degrees': { gain: 0.42, size: 4, over: 35, late: 85, kick: 0, back: 0, still: 0, lead: 90, jump: 60 }, - 'corner, 90 degrees': { gain: 0.55, size: 15, over: 35, late: 150, kick: 0, back: 0, still: 0, lead: 90, jump: 65 }, + 'sharp corner': { + gain: 0.48, + size: 6, + over: 20, + late: 55, + kick: 0.25, + back: 0.25, + still: 0, + lead: 45, + jump: 35, + kept: 80, + }, + 'corner, 45 degrees': { + gain: 0.42, + size: 4, + over: 35, + late: 85, + kick: 0, + back: 0, + still: 0, + lead: 90, + jump: 60, + kept: 150, + }, + 'corner, 90 degrees': { + gain: 0.53, + size: 15, + over: 35, + late: 150, + kick: 0, + back: 0, + still: 0, + lead: 90, + jump: 70, + kept: 150, + }, 'corner, 135 degrees': { - gain: 0.65, + gain: 0.61, size: 20, over: 35, late: 200, - kick: 40, - back: 40, + kick: 45, + back: 45, still: 0, lead: 90, jump: 75, + kept: 150, }, - 'corner, slow': { gain: 0.5, size: 2.5, over: 7, late: 25, kick: 0, back: 0, still: 0, lead: 20, jump: 15 }, + 'corner, slow': { gain: 0.5, size: 2.5, over: 7, late: 25, kick: 0, back: 0, still: 0, lead: 20, jump: 15, kept: 35 }, 'zigzag, sharp and fast': { - gain: 0.89, + gain: 0.94, size: 15, - over: 50, + over: 45, late: 150, kick: 35, - back: 50, + back: 45, still: 0, lead: 65, - jump: 60, + jump: 65, + kept: 150, + }, + 'rounded corner': { + gain: 0.4, + size: 2, + over: 20, + late: 40, + kick: 0, + back: 0, + still: 0, + lead: 45, + jump: 30, + kept: 80, }, - 'rounded corner': { gain: 0.47, size: 2, over: 20, late: 50, kick: 0, back: 0, still: 0, lead: 45, jump: 30 }, - circle: { gain: 0.34, size: 7.5, over: 25, late: 50, kick: 0, back: 0, still: 0, lead: 70, jump: 50 }, + circle: { gain: 0.34, size: 7.5, over: 25, late: 50, kick: 0, back: 0, still: 0, lead: 70, jump: 50, kept: 75 }, 'circle, tight and fast': { gain: 0.39, size: 5, - over: 20, + over: 15, late: 35, - kick: 0.75, - back: 0.75, + kick: 0.2, + back: 0.2, still: 0, lead: 55, jump: 45, + kept: 40, }, // A stir: small and quick enough that a whole horizon of it sweeps past a right angle, which is where the // arc has to stop being believed rather than wrap round. - stir: { gain: 0.95, size: 15, over: 0, late: 40, kick: 0, back: 0, still: 0, lead: 35, jump: 30 }, + stir: { gain: 0.95, size: 15, over: 0, late: 40, kick: 0, back: 0, still: 0, lead: 35, jump: 30, kept: 50 }, // The same circle with a hand on it, so the quantisation error stops being a function of the angle and // starts being noise. It scores like the clean one, which is what says the clean one is not being flattered // by its own regularity. - 'circle, hand-drawn': { gain: 0.35, size: 9, over: 25, late: 50, kick: 0, back: 0, still: 0, lead: 75, jump: 50 }, - 'circle, reversed': { gain: 0.34, size: 7.5, over: 25, late: 50, kick: 0, back: 0, still: 0, lead: 70, jump: 50 }, - 'figure eight': { gain: 0.86, size: 7.5, over: 15, late: 65, kick: 2.5, back: 3.5, still: 0, lead: 50, jump: 35 }, - spiral: { gain: 0.58, size: 8, over: 30, late: 75, kick: 0, back: 0, still: 0, lead: 95, jump: 60 }, - serpentine: { gain: 0.69, size: 3.5, over: 15, late: 45, kick: 0, back: 0, still: 0, lead: 30, jump: 25 }, + 'circle, hand-drawn': { + gain: 0.35, + size: 9, + over: 25, + late: 50, + kick: 0, + back: 0, + still: 0, + lead: 75, + jump: 50, + kept: 90, + }, + 'circle, reversed': { + gain: 0.34, + size: 7.5, + over: 25, + late: 50, + kick: 0, + back: 0, + still: 0, + lead: 70, + jump: 50, + kept: 75, + }, + 'figure eight': { + gain: 0.75, + size: 7.5, + over: 9.5, + late: 60, + kick: 0.4, + back: 0.4, + still: 0, + lead: 45, + jump: 35, + kept: 90, + }, + spiral: { gain: 0.37, size: 8, over: 30, late: 65, kick: 0, back: 0, still: 0, lead: 95, jump: 55, kept: 75 }, + serpentine: { gain: 0.66, size: 4, over: 9.5, late: 45, kick: 0, back: 0, still: 0, lead: 35, jump: 25, kept: 200 }, // A turn slow enough and wide enough that the angle between the window's two halves barely clears what // quantisation alone produces. Believing it too readily and not at all are both visible here. - 'wide slow arc': { gain: 0.36, size: 0.45, over: 4.5, late: 15, kick: 0, back: 0, still: 0, lead: 15, jump: 8 }, + 'wide slow arc': { + gain: 0.36, + size: 0.45, + over: 4.5, + late: 15, + kick: 0, + back: 0, + still: 0, + lead: 15, + jump: 8, + kept: 30, + }, // Reversals. Nothing in a window says a turn is coming when the hand is at its fastest, so the guess is // bounded by how long this hand has lately been going between turning round rather than by evidence. // Gain over 1 is honest here: there is nothing to win, and the job is not to lose much. - zigzag: { gain: 1.06, size: 15, over: 20, late: 75, kick: 30, back: 90, still: 0, lead: 30, jump: 35 }, - 'shake, 8Hz': { gain: 1.11, size: 9, over: 45, late: 50, kick: 30, back: 200, still: 40, lead: 15, jump: 40 }, + zigzag: { gain: 1.03, size: 15, over: 20, late: 75, kick: 30, back: 85, still: 0, lead: 30, jump: 35, kept: 25 }, + 'shake, 8Hz': { + gain: 1.1, + size: 9, + over: 45, + late: 50, + kick: 30, + back: 200, + still: 40, + lead: 15, + jump: 40, + kept: 30, + }, 'shake, fast and small': { gain: 1.11, - size: 6, + size: 5.5, over: 30, late: 30, kick: 25, back: 350, still: 55, - lead: 5.5, + lead: 6.5, jump: 25, + kept: 20, }, 'overshoot and correct': { gain: 0.55, @@ -155,16 +362,61 @@ const LIMITS: Record = { still: 35, lead: 65, jump: 45, + kept: 65, + }, + 'target acquire': { + gain: 0.54, + size: 20, + over: 40, + late: 100, + kick: 20, + back: 20, + still: 30, + lead: 75, + jump: 55, + kept: 85, }, - 'target acquire': { gain: 0.54, size: 20, over: 40, late: 100, kick: 20, back: 20, still: 30, lead: 75, jump: 55 }, // Braking is believed on thin evidence and acted on at once, which is right for a stop and is exactly // wrong for a hand that is only pausing between two throws. This is the case that pays for that choice. - 'double flick': { gain: 0.55, size: 25, over: 40, late: 150, kick: 20, back: 50, still: 45, lead: 85, jump: 65 }, + 'double flick': { + gain: 0.55, + size: 25, + over: 40, + late: 150, + kick: 20, + back: 50, + still: 45, + lead: 85, + jump: 65, + kept: 200, + }, // A hand is never still. The wobble is the size of the counts underneath it, which is where a second // derivative reads it as motion and a caller keeping the growth turns it into rotation. - 'tremor on a drag': { gain: 0.39, size: 2, over: 5.5, late: 15, kick: 0.25, back: 2, still: 0, lead: 15, jump: 15 }, - 'tremor at rest': { gain: 1.06, size: 2, over: 2.5, late: 3.5, kick: 2, back: 4.5, still: 15, lead: 1.5, jump: 2 }, + 'tremor on a drag': { + gain: 0.39, + size: 2, + over: 5.5, + late: 15, + kick: 0.25, + back: 1.5, + still: 0, + lead: 15, + jump: 15, + kept: 35, + }, + 'tremor at rest': { + gain: 1.06, + size: 2, + over: 2.5, + late: 3.5, + kick: 2, + back: 4.5, + still: 15, + lead: 1.5, + jump: 2, + kept: 2, + }, } describe('movement shapes', () => { @@ -197,6 +449,7 @@ describe('movement shapes', () => { // can only ever be pushed further along. expect(worst.worstJump).toBeLessThanOrEqual(limits.jump) expect(worst.worstRatchetJump).toBeLessThanOrEqual(limits.jump) + expect(worst.kept).toBeLessThanOrEqual(limits.kept) }) } @@ -258,6 +511,28 @@ describe('raising the lead does not throw a small circle apart', () => { } }) + test('a medium circle is drawn on the circle, at every speed a hand draws one at', () => { + // The one a hand actually makes: 120px across, at everything from a quick loop to a slow deliberate one. + // The guess used to sit well outside it in the middle of that range — 20px out at 600ms a lap on a + // trackpad, and the marker visibly orbiting outside the path the hand drew. + // + // The cause was two gates multiplying. The turn is read from two chords, which is a good measurement + // even on a thin window, and it was then multiplied by how far the *parabola* was believed — a worse + // measurement of the same thing, and one that sits near half on a window holding four samples. Two + // half-open gates left the arc bending at a quarter of the rate the hand was turning, and a lead that + // lags the heading on a circle points outwards, which is exactly where the marker was. + // + // So the turn is judged on its own evidence now, and that evidence is the average of several readings + // rather than one: a hand going round turns the same way on every frame while quantisation flips, so + // averaging keeps the turn whole and takes the noise down with the root of how many independent windows + // went into it. + const DRIFT: Record = { 300: 3, 400: 2, 600: 5, 900: 13, 1400: 9 } + for (const periodMs of [300, 400, 600, 900, 1400]) { + const run = worstAcross(DEVICES, { path: circle(120, periodMs), durationMs: 4000 }) + expect(Math.abs(run.meanRadius - 120)).toBeLessThan(DRIFT[periodMs]!) + } + }) + test('a circle a hand actually draws is untouched by the bound', () => { // The bound is on the turn, not on turning. A wide circle turns a fraction of a right angle over one // horizon, so it never reaches the bound and keeps every pixel of lead the setting asks for — the lead diff --git a/library/src/__tests__/predictor-stops.test.ts b/library/src/__tests__/predictor-stops.test.ts index 035a7d7..f79787d 100644 --- a/library/src/__tests__/predictor-stops.test.ts +++ b/library/src/__tests__/predictor-stops.test.ts @@ -311,7 +311,11 @@ describe('turning is not stopping', () => { // most of it on any device slower than a wired mouse is the device reporting in clumps rather than // anything the corner did — so what a corner may add is measured against the same hand going straight. // The figure is what the corner is worth on top of that, and it is small. - const ADDED: Record = { wired: 9, trackpad: 6, bluetooth: 4, 'slow radio': 13 } + // Believing a turn on the evidence of the chords alone rather than on the parabola's as well — which is + // what keeps the guess on a medium circle — makes the swing round a corner more energetic, and these + // went up by a few pixels for it. It buys 16px of drift off a 120px circle, which is a shape a hand + // makes far more often than it makes a right angle at 2.5px/ms. + const ADDED: Record = { wired: 12, trackpad: 15, bluetooth: 4, 'slow radio': 17 } for (const device of DEVICES) { const turning = replay({ path: corner(90), device, durationMs: 1400 }) const straight = replay({ path: piecewise([glide(2.5, 0, 1600)]), device, durationMs: 1400 }) diff --git a/library/src/predictor.ts b/library/src/predictor.ts index 66f7c3a..94b6482 100644 --- a/library/src/predictor.ts +++ b/library/src/predictor.ts @@ -33,6 +33,18 @@ // the heading of the window's newer half against its older half is just two chords. The arc degenerates to // the straight line as the turn rate goes to zero, so straight motion is bit-identical. // +// **The turn is believed on its own evidence, and on several readings of it.** It used to be multiplied by +// how far the parabola was believed as well, which is a worse measurement of the same thing — on a window +// holding four samples it sits near a half — and two half-open gates left the arc bending at a quarter of +// the rate the hand was turning. A lead that lags the heading on a circle points outwards, so that is a +// marker orbiting outside the circle the hand drew: 20px outside a 120px one. +// +// What replaced it is more evidence rather than a lower bar. A hand going round turns the same way on every +// frame while quantisation flips sign, so the readings are averaged before they are judged, and the noise on +// the average falls with the root of how many independent windows went into it. A reading that contradicts a +// turn already believed is taken at once instead, because a hand can start going round the other way and +// averaging across that says it is doing neither. +// // **And the horizon is bounded by how far the path turns over it.** Past about a quarter turn the answer is // mostly the turn rate, which is the noisiest thing here, and the chord being asked for has stopped growing // with the horizon and started coming back towards the hand — so a wobble in the rate moves the guess around @@ -100,6 +112,11 @@ const SURE_PX_PER_MS = 0.25 // in full. Both are the same question asked of a different part of the fit. const CURVE_SNR = 2 const TURN_SNR = 2 +// How far a chord's heading typically misses by, per count of quantisation at each of its ends, as a +// fraction of the furthest it could ever miss by. A position on a grid is off by a twelfth of a count in +// variance, which works out at about four tenths of the worst case; a twentieth above that is where a +// stalled frame's noise starts being believed as a turn, so this sits just the safe side of it. +const TURN_NOISE = 0.5 // A guess that bends further than this over one horizon is not a turn a hand makes; it is a fit that has // run away. const MAX_TURN_RAD = Math.PI / 2 @@ -113,6 +130,11 @@ const MAX_TURN_RAD = Math.PI / 2 const MAX_TRUSTED_TURN_RAD = Math.PI / 4 // How long to settle the turn rate the cap above is read from. const TURN_RATE_MS = 100 +// How long the turn readings are averaged over before they are believed. Long enough for the average to be +// worth more than one reading, short enough to follow a hand that changes which way it is going round. +const TURN_TRUST_MS = 120 +// And how quickly it gives way when the path starts turning the other way instead. +const TURN_CHANGE_MS = 30 // How sharply the horizon gives way once the path turns further than that over it. const TURN_CAP_KNEE = 4 // How far off a chord that ends on whole counts can point, and the unit every noise floor below is written @@ -292,6 +314,7 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { // how fast the path has lately been turning, which is what bounds how far ahead the turn may be trusted. let carriedTurn = 0 let turnRate = 0 + let turnAverage = 0 // A hand that has lately been doubling back, and how long it went between doing so. At the moment a // shake is fastest it is locally indistinguishable from a hand crossing the pad: same speed, and no // acceleration, because the turn is still ahead. Only the memory that it turned round a moment ago @@ -598,10 +621,47 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { // it is a flat tax: it removes as much from a turn a hand really made as from one only the counts // made up, and on a circle a hand does make that came to a third of the turn — which is most of // why the guess used to sit outside the circle. - const floor = countPx * (1 / newLength + 1 / oldLength) + // Not the worst a count can do to a chord's heading, which is what this used to be, but what it does + // typically: a position quantised to a grid is off by a twelfth of a count in variance, so a chord + // with a count of grid at each end points about four tenths of a count over its length away from + // where it really points. The worst case is two and a half times that, and using it as the divisor + // of a signal-to-noise ratio asked for a five-sigma turn before believing one — which a 120px circle + // at a comfortable speed does not produce, and it was left sitting outside the circle for it. + const floor = TURN_NOISE * countPx * (1 / newLength + 1 / oldLength) const angle = Math.atan2(cross, dot) - const believed = floor > 0 ? Math.max(0, Math.min(1, (Math.abs(angle) / floor - 1) / (TURN_SNR - 1))) : 0 - turn = (angle / (span / 2)) * believed * curveTrust + + // One reading a frame, and a noisy one. What separates a hand going round from a path of whole + // counts that only looks like it is going round is not how big any single reading was — a slow drag + // throws up ninety degrees of apparent turn from nothing at all — it is that a real turn goes the + // same way on every frame and a made-up one flips. So the readings are averaged before they are + // judged: a circle keeps its whole rate through that, quantisation averages to nothing, and the + // noise on what is left falls by the root of how many independent windows went into it. + // + // Windows overlap, so frames are not independent readings; only a window's worth of new samples is. + // That is what the count below is, and it is the whole of why this is more evidence rather than a + // lower bar. + const rate = angle / (span / 2) + const independent = Math.max(1, TURN_TRUST_MS / WINDOW_MS) + const floorRate = floor / (span / 2) / Math.sqrt(independent) + // A reading that disagrees with a turn already worth believing is news rather than noise, and is + // taken far sooner — the same asymmetry braking gets, for the same reason: a hand can stop going + // round one way and start going round the other, and averaging across that says it is doing neither. + // A figure eight does it twice a lap, and carrying the old way across each one rotated the lead + // against the hand for a tenth of a second at a time, which a caller keeping the growth keeps. + // + // The test is against a turn that was believed, not against zero, or a straight path — whose + // readings flip sign every frame and mean nothing — would take the quick road every time and the + // average would follow the noise it is there to cancel. + const contradicts = rate * turnAverage < 0 && Math.abs(turnAverage) > floorRate + turnAverage += (rate - turnAverage) * (1 - Math.exp(-deltaMs / (contradicts ? TURN_CHANGE_MS : TURN_TRUST_MS))) + const believed = + floorRate > 0 ? Math.max(0, Math.min(1, (Math.abs(turnAverage) / floorRate - 1) / (TURN_SNR - 1))) : 0 + // Not gated by how far the parabola is believed as well. The chords are measured off the path + // precisely because the parabola fits an arc badly, and on a window holding four samples the + // parabola is the weaker of the two — letting it veto the stronger left the arc bending at a quarter + // of the rate the hand was turning, which on a circle is a lead that lags the heading, and a lead + // that lags on a circle points outwards. That is the marker sitting outside the circle. + turn = turnAverage * believed carriedTurn = turn } } @@ -870,6 +930,7 @@ export const createPointerPredictor = (options: PredictorOptions = {}) => { msSinceReversal = 1e4 reversalPeriodMs = 1e4 turnRate = 0 + turnAverage = 0 // What a count is worth is not part of the session. It is a property of the device, it does not change // across a menu, and dropping it would put the noise floors back to assuming pixels for the first half // second after every unlock — which is exactly when the hand is moving again. From 3ef775ec6ddfaa0bdf8977b79942c70d50e1f2b2 Mon Sep 17 00:00:00 2001 From: verekia <522007+verekia@users.noreply.github.com> Date: Sun, 9 Aug 2026 12:56:47 +0700 Subject: [PATCH 6/6] feat(rig): draw the OS pointer as a white dot instead of an arrow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An arrow is a bad shape to judge alignment against: it is not centred on the position it means, so the eye compares a corner to a circle and the answer depends on which way the arrow points. A dot centred on the hotspot is the same shape as the markers chasing it, and a slow-motion frame reads as the dot being inside the ring, on its edge, or outside it. Radius 8 against the green ring's 11, so the two never touch when they agree and the gap between them stays readable. It is still the OS pointer — a cursor image handed to the window server and drawn by the compositor at the true position, exactly as the arrow was. Drawing a dot into the canvas instead would put it at the *reported* position, which is what the red square already is, and the lag this page exists to show would vanish along with the reference for it. Falls back to the ordinary arrow rather than to `none`: a rig with no pointer is worse than one with the wrong pointer, and there is already a button for hiding it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HctBLnHDXwx1XURcvnEB9T --- README.md | 10 +++++---- example/pages/index.tsx | 50 ++++++++++++++++++++++++++++++++--------- 2 files changed, 46 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 8697f94..14ed54d 100644 --- a/README.md +++ b/README.md @@ -282,16 +282,18 @@ The rest is mostly about not believing noise: ## The rig -`example/` is a cursor-to-photon rig: markers chase the OS cursor so a slow-motion recording can measure the +`example/` is a cursor-to-photon rig: markers chase the OS pointer so a slow-motion recording can measure the gap to it. That is the only way to see presentation latency — no clock inside the page can observe it. ```bash bun i && bun dev ``` -The red square is the reported position, the green circle is the prediction, and the OS cursor is the truth -(the compositor draws it, so it is never late). `?compare=1` shows two leads at once; `?nocursor=1`, -`?noreported=1` and `?nopredicted=1` strip the frame down to what you want to read. +The red square is the reported position, the green circle is the prediction, and the white dot is the truth: +it is the OS pointer wearing a cursor image, so the compositor still draws it and it is still never late — an +arrow is simply a poor shape to judge a circle against, being pointed rather than centred on the position it +means. `?compare=1` shows two leads at once; `?nocursor=1`, `?noreported=1` and `?nopredicted=1` strip the +frame down to what you want to read. ## Changing the prediction diff --git a/example/pages/index.tsx b/example/pages/index.tsx index 5fab9fb..1533993 100644 --- a/example/pages/index.tsx +++ b/example/pages/index.tsx @@ -10,8 +10,9 @@ import { import Head from 'next/head' -// Cursor-to-photon rig. Markers chase the OS cursor so a slow-motion recording can measure the gap to it, -// which is the only way to see presentation latency — no clock inside the page can observe it. +// Cursor-to-photon rig. Markers chase the OS pointer — drawn here as a white dot, see CURSOR_DOT_RADIUS — +// so a slow-motion recording can measure the gap to it, which is the only way to see presentation latency: +// no clock inside the page can observe it. // // Getting the input position is not the problem: the frame asked for at one refresh is presented one to // four refreshes later, and the hand has moved by then. Prediction is the only thing that closes that, and @@ -73,6 +74,30 @@ const COMPARE_COLOURS: [number, number, number][] = [ ] const RING_RADII = [11, 16] +// The OS pointer, re-skinned as a white dot a little inside the green ring's radius. +// +// It is still the OS pointer: a cursor image is handed to the window server and drawn by the compositor at +// the true position, exactly as the arrow was, so it stays the one thing in the frame that is never late. +// Drawing a dot into the canvas instead would draw it at the *reported* position, which is the red square — +// the lag this page exists to show would vanish along with the reference for it. +// +// An arrow is a bad shape to judge alignment against: it is not centred on the position it means, so the +// eye compares a corner to a circle and the answer depends on which way the arrow points. A dot centred on +// the hotspot is the same shape as the markers chasing it, and a slow-motion frame reads as the dot being +// inside the ring, on its edge, or outside it. Smaller than the ring so that the two never touch when they +// agree, which is what makes the gap between them readable at all. +const CURSOR_DOT_RADIUS = 8 +const cursorDot = () => { + const size = CURSOR_DOT_RADIUS * 2 + 4 + const centre = size / 2 + const svg = + `` + + `` + // Falls back to the ordinary arrow rather than to nothing: a rig with no pointer at all is worse than one + // with the wrong pointer, and `none` is what the hide button is for. + return `url("data:image/svg+xml,${encodeURIComponent(svg)}") ${centre} ${centre}, default` +} + // Number(null) is 0, so an absent parameter has to be checked for rather than converted — otherwise every // default silently reads as zero. const numberParam = (params: URLSearchParams, key: string, fallback: number) => { @@ -398,7 +423,10 @@ const Rig = ({ compare, measure, leadFrames, capPx, hideCursor, hideReported, hi }, [compare, measure, leadFrames, capPx, hideReported, hidePredicted]) return ( -
+
{[0, 1, 2, 3, 4].map(frames => ( @@ -434,9 +462,10 @@ const Rig = ({ compare, measure, leadFrames, capPx, hideCursor, hideReported, hi CLAMPED - {/* The OS cursor is drawn by the compositor, not by the page, so it is always at the true position - and never late. Hiding it leaves only the two markers, which is what a slow-motion recording - needs to read the lead on its own; showing it is what measures the lag against the truth. */} + {/* The OS pointer — the white dot — is drawn by the compositor, not by the page, so it is always at + the true position and never late. Hiding it leaves only the two markers, which is what a + slow-motion recording needs to read the lead on its own; showing it is what measures the lag + against the truth. */} {/* The legend doubles as the switch for each marker: a marker on its own against the OS cursor is @@ -503,9 +532,10 @@ const Rig = ({ compare, measure, leadFrames, capPx, hideCursor, hideReported, hi

- Move the pointer around. The red square is where the browser says it is; the green circle is where hyperpointer - thinks it will be by the time this frame reaches the screen. They should sit on top of each other when the hand - is still, and the circle should sit on the OS cursor while it moves. + Move the pointer around. The white dot is the OS pointer, drawn by the compositor and never late; the red square + is where the browser says it is; the green circle is where hyperpointer thinks it will be by the time this frame + reaches the screen. All three should sit on top of each other when the hand is still, and the circle should stay + around the dot while it moves.

{measure && capture && (