[jehrr]

The audit's last recommendation asks for a freshness and completeness report — so a consumer can see the date, the walk type, the number of events and the gaps.

The sidecar already carries all four, and checking that claim meant reading a real one rather than building anything:

what the audit asks for	field
date	finished_at
walk type	mode, pagination
number of events	total_events, events_on_page_1
gaps	pages_failed, pages_without_rows, pages_completed/pages_requested

Reading one is how this turned up.

events_on_page_1 was counting the wrong thing

"total_events": 23475,
"events_on_page_1": 3,

on a ten-page walk where page 1 had named 11 events.

It was counted over the merged rows with r.page == 1, and that asks a different question once the walk has merged: --mode events replaces a listing row in place with the deeper read from its event page, and the replacement carries that event page's number. So the field counted the events whose rows happened not to be upgraded — which is an artefact of how far the walk got, not a fact about the listing.

It matters because of what sits beside it. total_events against events_on_page_1 is exactly the comparison a reader makes to judge how much of the catalogue a run saw, and 3-against-23,475 tells them something very wrong about a run that read 11 events in full.

It is correct in --mode markets, where nothing is upgraded, which is why it looked fine.

Verified live after the change

events_on_page_1 = 12
distinct event_slugs in the file = 12
rows still carrying page 1       = 7     <- what the old expression counted

matching the engine's own log line for that run: "this run took 160 market(s) from the 12 event(s) the page renders".

Verification

Fixed in all three engines and controlled against each one separately, because a fix landing in one of three is a failure this repo has already met:

planted fault	caught by
playwright_scraper counts over the merged rows again	...from page 1's own outcome, not the merged rows
selenium_scraper likewise	same, under that engine
puppeteer_scraper likewise	same, under that engine

The check reads the source rather than calling a function, because the sidecar is assembled inline in each engine; that is also why it is worth asserting in all three rather than once.

No new sink, queue or delivery layer

The audit's final item suggests a service layer — queue, schedules, storage, API, webhooks — if the goal is to compete with managed platforms. That is a product decision rather than a defect, and it is not taken here: this repo is a scraper with a file output contract, and the README already says so. Anything of that shape belongs behind one small sink interface in the shared core rather than grown into product_parser.py or the row model, and it would have to preserve two things this contract already guarantees — a failed run writes nothing, and the sidecar reaches the same place as its rows or not at all.

🤖 Generated with Claude Code