fix(llm): Stop vague quota wording from making a 429 fatal - #1081
Open
JeanMertz wants to merge 2 commits into
Open
fix(llm): Stop vague quota wording from making a 429 fatal#1081JeanMertz wants to merge 2 commits into
JeanMertz wants to merge 2 commits into
Conversation
A rate limit whose body happens to say "quota exceeded" is no longer reported as exhausted billing. Before, such a response ended the turn with a fatal `Insufficient API quota` and discarded the `Retry-After` the provider sent, instead of waiting out a bucket that refills seconds later. Quota detection runs on the response body, and two of its markers — `quota exceeded` and `resource_exhausted` — are also how providers word an ordinary rate limit. Google reports every 429 as `RESOURCE_EXHAUSTED` whether or not billing is involved, and Cerebras answers an exhausted per-minute token bucket with `token_quota_exceeded`, one character away from matching. A status code is the stronger signal, so on a 429 only a marker that names billing outright now counts: `insufficient_quota`, `billing_error`, `credit balance is too low`, and the credit wording OpenRouter uses. OpenAI reports a drained account as a 429 `insufficient_quota`, so that case stays fatal. Away from a 429 there is no authoritative status to defer to and all markers still classify, so 402 and 403 responses are unaffected. This brings the shared classifier in line with the Google provider, which already checked its 429 first and used a typed `quotaId` rather than prose to separate a spent daily allowance from a refilling one. Signed-off-by: Jean Mertz <git@jeanmertz.com>
A 429 whose body says "insufficient quota" in words is now a retryable rate limit rather than fatal exhausted billing, joining `quota exceeded` and `resource_exhausted` among the markers a status code outranks. The typed `insufficient_quota` code keeps its place among the billing markers and still ends a turn on a 429, which is how OpenAI reports a drained account. What it earns that place with is being a code; spelled out with a space it is the ambiguous word `quota` with a qualifier in front, which describes a refilling bucket as readily as an empty wallet. Every other billing marker names money outright. The phrase has no known producer. It appears exactly once in the tree, is credited to no provider in the documentation that lists the others, and was added alongside the typed code in #402 as a spelling variant rather than an observed response. So it was held to a rule it could not be checked against, in the one code path where guessing wrong ends the turn and discards the provider's own retry timing. Behaviour away from a 429 is unchanged: the permissive check is a superset, so 402 and 403 still classify on the phrase, as do the in-stream error paths in the OpenAI and Google providers, which consult it directly. Signed-off-by: Jean Mertz <git@jeanmertz.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A rate limit whose body happens to say "quota exceeded" is no longer reported as exhausted billing. Before, such a response ended the turn with a fatal
Insufficient API quotaand discarded theRetry-Afterthe provider sent, instead of waiting out a bucket that refills seconds later.Quota detection runs on the response body, and two of its markers —
quota exceededandresource_exhausted— are also how providers word an ordinary rate limit. Google reports every 429 asRESOURCE_EXHAUSTEDwhether or not billing is involved, and Cerebras answers an exhausted per-minute token bucket withtoken_quota_exceeded, one character away from matching. A status code is the stronger signal, so on a 429 only a marker that names billing outright now counts:insufficient_quota,billing_error,credit balance is too low, and the credit wording OpenRouter uses. OpenAI reports a drained account as a 429insufficient_quota, so that case stays fatal.Away from a 429 there is no authoritative status to defer to and all markers still classify, so 402 and 403 responses are unaffected. This brings the shared classifier in line with the Google provider, which already checked its 429 first and used a typed
quotaIdrather than prose to separate a spent daily allowance from a refilling one.