Skip to content

Use the standard MD5 for cache ref names - #2092

Merged
xroche merged 3 commits into
masterfrom
fz/B13
Nov 7, 2026
Merged

xroche merged 3 commits into
masterfrom
fz/B13

Conversation

@xroche

@xroche xroche commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Each hts-cache/ref/*.ref file is named by an MD5 of its URL. On little-endian hosts that MD5 was wrong, because url_savename_refname() passed MD5Init() the byte-swapping flag. A new hts_md5_init() now gives the right flag to all three MD5 users, and hts_md5_hex() replaces two long sprintf calls.

A crawl interrupted by an older binary on a little-endian host loses its .ref files once, so a resume downloads those partial files again.

Test 754 checks three names against md5sum. It fails on a build with the old flag, which gives 51924b09... for abc,def instead of d9355c9c....

xroche and others added 3 commits November 7, 2026 18:26
url_savename_refname() passed 0 to MD5Init(), which byte-swaps on a
little-endian host, so the hts-cache/ref names were not the MD5 of
"adr,fil" there. Pass the native-endian flag, as domd5mem() does. Ref
files left by an interrupted crawl from an older binary are no longer
found on resume, and those files are fetched again.

Test 754 pins three names against md5sum vectors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
hts_md5_init() picks the flag that gives the standard digest on this
host, for domd5mem(), the changes report and the ref names.
hts_md5_hex() replaces the two 16-argument sprintf calls. md5.h loses
four prototypes that nothing defines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
@xroche
xroche enabled auto-merge (squash) November 7, 2026 18:47
@xroche
xroche merged commit 610e8d7 into master Nov 7, 2026
39 checks passed
@xroche
xroche deleted the fz/B13 branch November 7, 2026 21:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant