Repository navigation
Conversation
A start URL without a scheme was tested with strstr(":/"), so a seed
like host/page?u=http://x got no http:// and was crawled as a relative
link under "primary". The four seed sites (command line, -%L, login
bundles, --why) now share seed_needs_scheme(), built on
link_has_authority(); a "//host" seed keeps its old prefix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
An indented -%L line such as ' http://host/x' found its scheme on
master, because strstr(":/") searched the whole string. The new test
only looked at byte 0 and prefixed http://, so the seed fetched nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
The command line escapes a seed's leading space to %20 before the engine sees it, so " http://host/" still got a second http:// and fetched nothing. Test 752 now crawls such an argv seed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A start URL without a scheme now gets
http://even when its path or query holds:/, for exampleexample.com/go?u=http://x. Before, httrack took such a seed as a relative link under the hostprimaryand fetched nothing. The command line,-%L, login bundles and--whynow share one test built onlink_has_authority, and a//hostseed keeps thehttp://it got before.Test 752 crawls such a seed through each of the four paths and fails on master. Putting the old test back at any one site also makes it fail.