docs: Proposal for client resiliency - #222
Conversation
jpage-godaddy
changed the title
| - 500 token capacity, starting full | ||
| - 14 tokens per transient retry, 5 per throttling retry | ||
| - Refund the consumed cost when a retry succeeds, plus 1 token on a first-try success | ||
| - When the budget empties, return the error without retrying | ||
|
|
||
| This turns "retry 3 times" from an unconditional behavior into one that yields during a broad failure. | ||
| With 3 max attempts it starts draining at roughly 22% sustained transient failure, so normal operation never notices it. | ||
| It is the item aimed most directly at the failure mode in the GitHub postmortem. |
There was a problem hiding this comment.
I'm not clear on the scope of this. The CLI is typically one or two requests per command, and then it's done. It's not a long-running service making long sequences of requests. If the concern is we're retrying too frequently, wouldn't 500 tokens with a cost of 5 or 14 per retry be far more than the current retry rate we're doing? Is this a persisted token bucket, lasting between CLI invokes? Or are you anticipating that when we add default request timeouts that we'll end up doing larger retry loops than normal so we need a 500 token budget to cover that?
There was a problem hiding this comment.
(ok I see the persist note later on. I like that idea)
| When a 429 or 503 carries `Retry-After`, use the server's value clamped the way AWS clamps it: no lower than the computed backoff, no higher than computed backoff plus 5s, and no jitter applied on top since the server is expected to jitter it. | ||
|
|
||
| Also persist it. | ||
| Write a single "do not call `{host}` before `T`" timestamp per environment under `~/.config/gddy/`. |
There was a problem hiding this comment.
Should we make this per host? One thing that makes it tricky is that everything goes through api.godaddy.com, so it's essentially one host for everything. I suppose of EFD itself is down that makes sense, but if the concern is a single system, we may want to scope to specific paths.
There was a problem hiding this comment.
Good call...I don't know enough about api.godaddy.com and how the routes are configured. Is there some order to the way services underneath are segmented? If so, we could match that logic in the CLI to bifurcate more smoothly.
If we have some signal that the issue truly is in api.godaddy.com, we could specifically return a signal from that front door service and treat it as a global problem.
|
|
||
| ### 5. Honor `Retry-After`, in-process and across invocations | ||
|
|
||
| When a 429 or 503 carries `Retry-After`, use the server's value clamped the way AWS clamps it: no lower than the computed backoff, no higher than computed backoff plus 5s, and no jitter applied on top since the server is expected to jitter it. |
There was a problem hiding this comment.
I would throw 504s in here as well; I think we should treat them like 429s since they usually also reflect overloaded systems that are too exhausted to do a proper 429 response or that aren't configured with load shedding.
|
I like this idea. I had hoped that we could somehow easily use a shared HTTP "agent" across all API clients; that would allow us to do declarative sharing of HTTP behaviors. The trickiest bit is the spec-generated clients. There was a half-baked thing in place to set the user agent globally, but I'd love it if the generated clients could somehow share the code. |
2 participants
Adds
docs/design/client_resiliency.md, a proposal for owning retry, timeout, and rate-limiting policy in the CLI's HTTP layer instead of leaving it to callers.No code changes. This is a policy doc to argue about before any of it gets built.