Calibration derives what it needs from a proxy once - #467
Open
bjmeetsfo wants to merge 1 commit into
Open
Conversation
A calibration round walks the proxy groups, and for each one it walked every
proxy three times over: once to collect what is attached to that group, once
to parse every proxy's location and test it against the group's, and once
more -- inside a filter -- to scan the attachments already planned this
round.
None of that varies with the group asking. A proxy's location is the same
string whichever group considers it, and parsing allocated a fresh one every
time. Which group a proxy is attached to is a property of the proxy, known
before the walk begins. And "has this round already promised this proxy" is
a membership question that was being answered by scanning the plan so far,
so the cost grew with each attachment made.
groups x proxies before after
25 x 250 2 324 us 701 us
50 x 500 10 479 us 3 238 us
100 x 1 000 58 020 us 11 772 us
Five times faster at the largest, measured before and after back to back,
and what is left grows with groups times proxies rather than faster than it.
The rule that two groups never claim the same proxy is what the scan was
buying. It has a test, and removing the set that replaced the scan fails it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A calibration round walks the proxy groups, and for each one it walked every
proxy three times over: once to collect what is attached to that group, once
to parse every proxy's location and test it against the group's, and once
more -- inside a filter -- to scan the attachments already planned this
round.
None of that varies with the group asking. A proxy's location is the same
string whichever group considers it, and parsing allocated a fresh one every
time. Which group a proxy is attached to is a property of the proxy, known
before the walk begins. And "has this round already promised this proxy" is
a membership question that was being answered by scanning the plan so far,
so the cost grew with each attachment made.
Five times faster at the largest, measured before and after back to back,
and what is left grows with groups times proxies rather than faster than it.
The rule that two groups never claim the same proxy is what the scan was
buying. It has a test, and removing the set that replaced the scan fails it.