Skip to content

SpringerSource: resolve table links against the page URL the page states - #73

Merged
jdkent merged 1 commit into
masterfrom
ace-relative-table-urls
Oct 26, 2026
Merged

jdkent merged 1 commit into
masterfrom
ace-relative-table-urls

Conversation

@jdkent

@jdkent jdkent commented Oct 17, 2026

Copy link
Copy Markdown
Collaborator

SpringerSource joined a table page's root-relative href against the citation_fulltext_html_url meta, and used the href as it was when that meta was missing. A Nature page that has no such meta (article s41380-026-03719-9, PMID 42401715) therefore requested /articles/s41380-026-03719-9/tables/1, got Invalid URL ... No scheme supplied five times, and the article lost its tables.

The page URL now comes from Source._page_url: <base href>, then the canonical link, then og:url, then citation_fulltext_html_url. SpringerSource joins table hrefs against it with urljoin.

Measured on the chain19 extract log: 5 warnings, all for that one URL (the retries of one request, one article). Across the 301,549 saved ACE html files on beast, 9,079 have root-relative /article(s)/.../tables/N links; 9,078 carry citation_fulltext_html_url, and this one page is the only one without it.

Test: test_springer_table_links_resolve_against_page_url runs the real SpringerSource on that saved page (scripts and styles removed) and checks that the first request is the absolute table URL. The table page is a VCR cassette recorded from nature.com (set-cookie headers removed). With content_url = None restored, the test fails. Full suite: 76 passed.

Not changed: _normalize_table_link (DefaultSource) still returns the href unchanged when _get_base_url finds nothing. No log line shows that path failing.

🤖 Generated with Claude Code

A Nature page without citation_fulltext_html_url left its root-relative
table hrefs unjoined, so requests failed with No scheme supplied and the
article lost its tables. The page URL is now <base href>, the canonical
link, og:url, then citation_fulltext_html_url.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdkent
jdkent marked this pull request as ready for review October 17, 2026 09:14
@jdkent jdkent self-assigned this Oct 17, 2026
@jdkent

jdkent commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

LGTM

@jdkent
jdkent merged commit 3af3181 into master Oct 26, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant