Skip to content

fix(analysis): categorise notes by their section and stop early only on looked-up versions - #16

Merged
fleizean merged 1 commit into
developfrom
gsd/quick-261021-analysis-fixes
Oct 21, 2026
Merged

fleizean merged 1 commit into
developfrom
gsd/quick-261021-analysis-fixes

Conversation

@fleizean

@fleizean fleizean commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Summary

Three fixes found by analysing 7 Packagist and 10 Maven upgrades on production:

  • breaking changes written under a heading are now detected;
  • the first comparison of a package no longer stops after 10 versions;
  • more GitHub tag spellings match.

Why

  • No breaking changes anywhere. monolog/monolog 2.9.0 → 3.0.0 and doctrine/orm 2.17.0 → 3.0.0 both list their breaking changes under a heading (Noteworthy BC Breaks:, #### BC Break), yet every one of the 17 comparisons reported 0 breaking changes. AnalyzerService categorised each line on its own: a heading never passed its category to the bullets below it, and "BC break" was not a known phrase. Keep-a-Changelog notes (### Fixed, ### Added) were all "Other" for the same reason.
  • First comparisons cut at 10 versions. A first comparison has no stored notes yet, so every version counted as empty and MaxConsecutiveEmptyVersions (10) stopped the list. phpunit/phpunit 10.5.0 → 11.0.0 showed 10 of 67 versions.
  • Tags that did not match. Several Maven projects tag releases in ways the scraper did not try:
    • Guava releases 33.0.0-jre as v33.0.0;
    • JUnit tags r5.11.0, slf4j v_2.0.17;
    • Gson tags gson-parent-2.11.0, OkHttp parent-4.12.0.

Changes

  • Section headings (AnalyzerService): a heading sets the category of the lines below it until the next heading. Headings are:

    • # lines;
    • lines made only of **bold**;
    • short non-list lines ending in : that name a category.

    A heading that names no category (### 3.0.0) ends the section; a sentence that merely ends in : does not. Section words: breaking / BC break / incompatible, security, deprecat / removed, bug / fixed, feature / added / new, performance, and neutral ones (changed, contributors, thanks, upgrade, ...) so ## New Contributors is not a feature. A line's own prefix (Fixed: ...) still wins over its section. bc break is also a line prefix.

  • Stored categories refresh: categories are stored with each note when it is scraped. A new ReleaseNoteReparseJob runs once at start-up. It compares the new Analyzer.ParserVersion setting (default 1) with ReleaseNoteReparseJob.ParserVersion (2). If they differ, it enqueues one re-parse job per package (one scope each, so no context holds every note) and records the version. Later start-ups are a no-op until the parser version is raised again.

  • Early stop (PackageAnalysisService.ProcessVersionsAsync):

    • Only versions whose notes were looked up and found empty count towards MaxConsecutiveEmptyVersions. Versions never scraped neither count nor reset the run.
    • Registry date lookups for unscraped versions are capped at that same number, because PyPI and npm answer a date with the whole package document. Later dates come from the scrape.
  • Tag candidates (GitHubScraperService): {name}-parent-{v}, parent-{v}, r{v}, v_{v}; the release-suffix fallback also drops -jre / -android.

Notes for reviewers

  • After deploy, the re-parse runs in the background (about 350 small Hangfire jobs). Categories update as each package finishes.
  • Versions that were already scraped empty (Guava, Gson, the missing JUnit/OkHttp tags) are retried by the existing rules: empty notes after 1 day, a package scrape at most every 12 hours. They fill in within a day without any action.
  • Not verified locally: no build or test was run. CI is the check. Known remaining gaps: Apache projects (JIRA notes) and PHPUnit's per-major ChangeLog-11.0.md.

Checklist

  • The pull request targets develop and is about one thing
  • Commit messages follow Conventional Commits (feat(scope): ..., fix(scope): ...)
  • CI is green (backend tests, frontend build and tests, dependency gate)
  • Schema changes are EF Core migrations generated by the tool, not hand-edited
  • A changed backend DTO has its TypeScript twin updated in src/frontend/src/app/core/models
  • New dependencies are justified and their lock files are updated in the same commit
  • No secrets, tokens or personal data in code, logs or this description
  • Code comments are kept to a minimum; the reasoning lives in this description

…on looked-up versions

Headings such as "BC Breaks:", "#### BC Break" or "### Fixed" now give
their category to the lines below them, and a one-off job re-parses the
stored notes. The consecutive-empty cutoff no longer counts versions that
were never scraped, and GitHub tags spelled parent-, r, v_ or without a
-jre/-android variant now match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@fleizean
fleizean merged commit dbd7bbe into develop Oct 21, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant