Skip to content

feat: score trend samples by percentile and sample recent uploads - #71

Merged
devhims merged 3 commits into
fix/trend-lab-empty-topicfrom
feat/trend-metrics-percentile-scoring
Sep 21, 2026
Merged

devhims merged 3 commits into
fix/trend-lab-empty-topicfrom
feat/trend-metrics-percentile-scoring

Conversation

@devhims

@devhims devhims commented Sep 21, 2026

Copy link
Copy Markdown
Owner

Stacked on #67. Base retargets to main once that merges.

Why

The Trend Lab's numbers were internally consistent but answered the wrong question. Three specific failures, all visible in a real Claude Code scan:

  1. Scores came from fitted curves against the sample median (50 + 22 * log2(v / median), exp(-ageHours / 504), engagementRate * 12). The median for that query moved from 276 to 112 views/hour between two scans hours apart, so a score of 58 meant different things each time, and nobody can say why 22.
  2. An estimated video carries 0.55 weight on lifetime views divided by age, so a four month old hit outranked a genuinely accelerating new video and could be banded Breakout on the strength of an average.
  3. Relevance search returns a topic's greatest hits. That sample held one video from the last seven days out of eight, and seven of eight were 20 minutes or longer, which is where the plan's 28 minute length recommendation came from.

What changed

Percentile scoring. Velocity, freshness, channel performance, engagement and acceleration are now percentile ranks inside the sample. Weights are unchanged, so 50 is still the middle, but the number reads as a position and survives the divisor moving. A missing signal scores 50 so it neither helps nor hurts. methodologyVersion goes to 3.0, and each video exposes percentiles, velocityRank and effectiveViewsPerHour.

Breakout requires measurement. It now needs signalSource: observed and at least 5 enriched videos. confidence.reasons says how many videos were withheld and why.

Recency reservation. The candidate pool reads deeper (4 pages or 4x the limit) and up to half the enrichment budget goes to videos published inside a 14 day window. The per-channel cap of 3 is only relaxed to avoid returning a short sample. New window block and summary.medianRecentViewsPerHour / summary.recentVelocityLift; length advice comes from recent uploads when the window holds at least two.

searchRank is now the relevance position rather than the enrichment position, and is absent for a video only the recency pass found.

Dashboard. Charts, leaderboard and scatter were showing lifetime averages even where measured growth existed. They now show the velocity the score used, labelled measured or estimated. The cumulative-views tile is replaced by recent versus established.

Effect on the real sample

Same eight videos, scored both ways:

before after video
58 Rising (listed 1st) 65 Rising (2nd) Anthropic's own "What is Claude Code?", 4 months old
58 Rising (2nd) 69 Rising (1st) Mikey No Code, 3 days old, small channel
28 Steady 26 Steady Nick Saraev, 2.5M views, measured growth of 0/h

The new top row is the one a creator can act on. Breakout count is 0, honestly, because nothing leads on every axis.

Not included

Date-filtered search. SearchFilters declares dateFrom, dateTo and sort: 'date', but packages/all-things-youtube/src/youtube-client.ts implements none of them: only type, channelId, minViews, duration, live, captionsOnly and sort: 'views' do anything. An exact window needs a library release plus a processor lockfile bump, so the window is derived from each candidate's published-age text instead. That cannot see recent uploads ranking too poorly to appear at all, and the report says so in confidence.reasons.

Two follow-ups this exposes, both out of scope here: the agent's search_youtube tool advertises those same unimplemented filters to the model, and a cold scan still sheds videos (This YouTube processor is at capacity.) because enrichment fans out 6 videos, which is 12 concurrent container calls.

Also deferred from the design discussion: topic-level rollups in D1 so scans produce week-over-week deltas (needs a migration), and subscriber tiers for a "can a small channel win here" signal (needs one getChannel per unique channel).

Testing

  • platform: 669 tests pass, 10 in trends.test.ts covering percentile midpoints, the Breakout gate on both estimated momentum and thin samples, the recency reservation against the channel cap, and neutral ranks for missing signals.
  • web: 54 tests pass.
  • tsc --noEmit clean in both packages; npm run docs:check clean after regenerating docs/api-reference/openapi.json.
  • The before/after table above came from running the real API response through buildTrendReport in a throwaway test, since the scoring change cannot be verified against production until this deploys.

…ploads

the report scored every video with fitted curves against the sample median, so a
score moved when the divisor moved and nobody could say why 22 or why a 21 day
decay. each signal is now a percentile rank inside the sample: velocity,
freshness, channel performance, engagement and acceleration. the weights are
unchanged, so 50 still means the middle of the sample, but the number now reads
as a position and stays comparable when the sample's absolute numbers move.
methodologyVersion goes to 3.0.

a lifetime average could also win the ranking. an estimated video carries 0.55
weight on views divided by age, so an old hit outranked a genuinely accelerating
new video, and it could be banded Breakout on that basis. Breakout now requires
measured growth between snapshots and at least five enriched videos, since the
sample is its own reference frame.

relevance search returns a topic's greatest hits, so recent uploads were being
crowded out: the real Claude Code sample held one video from the last seven days
out of eight. the candidate pool now reads deeper and up to half the enrichment
budget is reserved for videos published inside a 14 day window, with the
per-channel cap only relaxed to avoid returning a short sample. the report says
how many recent candidates it saw and how recent uploads compare with the
established ones, and length advice comes from recent uploads when the window
holds enough of them.

searchRank now carries the video's position in the relevance list rather than its
position in the enrichment set, and is absent for a video only the recency pass
found.

the dashboard was showing lifetime averages even where measured growth existed.
charts, the leaderboard and the scatter now show the velocity the score actually
used and label it measured or estimated. the cumulative-views tile is replaced by
recent versus established, which is the actionable one.

not included: date-filtered search. SearchFilters declares dateFrom, dateTo and
sort 'date' but youtube-client implements none of them, so an exact window needs
a library release. the recency window is derived from each candidate's published
age text instead, which cannot see recent uploads that rank too poorly to appear
at all. the report states that limitation in its confidence reasons.
@vercel

vercel Bot commented Sep 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
video2ctx-web Ready Ready Preview Sep 21, 2026 9:54pm UTC

@devhims
devhims added this pull request to stack #72 September 21, 2026 22:00
@devhims
devhims merged commit ff58c84 into main Sep 21, 2026
7 checks passed
@devhims
devhims deleted the feat/trend-metrics-percentile-scoring branch September 21, 2026 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant