Skip to content

Calibration derives what it needs from a proxy once - #467

Open
bjmeetsfo wants to merge 1 commit into
mainfrom
oss/calibration-derives-per-proxy-facts-once
Open

Calibration derives what it needs from a proxy once#467
bjmeetsfo wants to merge 1 commit into
mainfrom
oss/calibration-derives-per-proxy-facts-once

Conversation

@bjmeetsfo

Copy link
Copy Markdown
Collaborator

A calibration round walks the proxy groups, and for each one it walked every
proxy three times over: once to collect what is attached to that group, once
to parse every proxy's location and test it against the group's, and once
more -- inside a filter -- to scan the attachments already planned this
round.

None of that varies with the group asking. A proxy's location is the same
string whichever group considers it, and parsing allocated a fresh one every
time. Which group a proxy is attached to is a property of the proxy, known
before the walk begins. And "has this round already promised this proxy" is
a membership question that was being answered by scanning the plan so far,
so the cost grew with each attachment made.

groups x proxies      before      after
    25 x    250     2 324 us     701 us
    50 x    500    10 479 us   3 238 us
   100 x  1 000    58 020 us  11 772 us

Five times faster at the largest, measured before and after back to back,
and what is left grows with groups times proxies rather than faster than it.

The rule that two groups never claim the same proxy is what the scan was
buying. It has a test, and removing the set that replaced the scan fails it.

A calibration round walks the proxy groups, and for each one it walked every
proxy three times over: once to collect what is attached to that group, once
to parse every proxy's location and test it against the group's, and once
more -- inside a filter -- to scan the attachments already planned this
round.

None of that varies with the group asking. A proxy's location is the same
string whichever group considers it, and parsing allocated a fresh one every
time. Which group a proxy is attached to is a property of the proxy, known
before the walk begins. And "has this round already promised this proxy" is
a membership question that was being answered by scanning the plan so far,
so the cost grew with each attachment made.

    groups x proxies      before      after
        25 x    250     2 324 us     701 us
        50 x    500    10 479 us   3 238 us
       100 x  1 000    58 020 us  11 772 us

Five times faster at the largest, measured before and after back to back,
and what is left grows with groups times proxies rather than faster than it.

The rule that two groups never claim the same proxy is what the scan was
buying. It has a test, and removing the set that replaced the scan fails it.
@bjmeetsfo
bjmeetsfo requested a review from superhaiou as a code owner August 30, 2026 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants