Repository navigation
Keep the cookies of the sites a login went through - #272
Merged
Merged
Conversation
A login that ends on another site lost the cookies of the site the user typed. The engine names the bundle after the address the login landed on. That address alone set the bundle's scope, so every record outside it was dropped before the file was written. Xavier measured it: entering httrack.com and reaching ffii.org through the login wrote a bundle holding only the ffii.org cookies. The crawl needs those records. With --near or a filter it fetches the first site's images and stylesheets through pages on the second. An out-of-scope cookie is dropped when the bundle loads, not when the request goes out. Such a file comes back as a login page or a 403, and lands in the mirror as that. So the window now names the hosts that answered with a cookie, and the engine keeps their records behind a scope line each. A named host never becomes a crawl root. The user asked to copy one site, not the sign-in page it passed through. The engine runs out of room when its jar fills, and it now reports how many records it left out. That count is shown apart from the cookies this app drops for length. The causes differ, and only one of them is something the user can act on. Engine pin moves to 483d5d47 for hts_bundle_from_paste_hosts, engine master. HTTRACK_VERSIONID is still 3.51.0, so versionName stays. Signed-off-by: Xavier Roche <roche@httrack.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The test matched the function name alone, so passing a NULL hosts list read as the fix and did nothing. That mutant survived. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
Every empty-host case was leading, where skipping it and keeping it build the same string. A trailing one is what leaves a stray comma, and the mutant that drops the skip survived without it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
The file uses CRLF, and a text-mode rewrite turned all 303 lines over to LF. Only the added string should show in the diff. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
The engine leaves a named host out when its jar fills or a cookie is too long. Neither can happen from this app, and two existing tests already pin why: MAX_PAIR equals the engine COOKIE_MAX_PAIR and tests the same joint guard, and MAX_BYTES is below COOKIE_MAX_JAR. So the count is always zero and the message was unreachable. That removes a string nobody can see, its branch, the extra, the LoginBundle field and the JNI pass-through. The engine keeps the field for WinHTTrack, whose capture is a whole browser profile. Also drop the empty-host skip in keptHosts, which the one caller cannot reach, and an asserted host order the engine ignores. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A login that ends on another site lost the cookies of the site the user typed. The address the login landed on names the bundle, and that address alone set its scope. Every record outside it was dropped before the file was written. Entering httrack.com and reaching ffii.org through the login wrote a bundle holding only the ffii.org cookies.
The crawl needs those records. With --near or a filter it fetches the first site's images through pages on the second. The images come back as a login page or a 403.
The window now names the hosts that answered with a cookie. A named host never becomes a crawl root.
Engine pin moves to 483d5d47, engine master. HTTRACK_VERSIONID is still 3.51.0, so versionName stays.