Skip to content

Classify provider refusals once for every chat action - #1753

Merged
kevmo314 merged 1 commit into
mainfrom
claude/eager-dijkstra-hdrup8
Nov 1, 2026
Merged

kevmo314 merged 1 commit into
mainfrom
claude/eager-dijkstra-hdrup8

Conversation

@kevmo314

Copy link
Copy Markdown
Contributor

Problem

Error group CKSyptWhubTOeA (two events about 1s apart on 0cf4831):

HTTP POST /unban returned 500: provider returned HTTP 400

This is a regression from #1752. That PR treated a 400 as a malformed request (and alerted) when Twitch's message mentioned parameter|required|malformed|valid. Twitch's own unban refusal wording mentions the "user_id query parameter", so a normal "this user isn't banned" refusal was reported as an agent bug. I reproduced it with that wording against main: the response was a 500.

The same problem showed up on other endpoints: Twitch deleteMessage/clear and every Kick action (ban, unban, timeout, deleteMessage, send) had no refusal handling. Any 4xx where the provider explained its refusal turned into a 500 alert.

Fix

Refusals are now classified in one place for all chat actions, and the provider's wording is no longer interpreted:

  • clients.HTTPError.Explained records only whether the provider gave a reason (a non-blank JSON message, the field Twitch and Kick use). The text is never stored, so it can't leak into logs or responses.
  • providerRejection in the shared chat wrapper turns an explained 400/404/409/422 into failed-precondition " rejected this request." The existing 429 → resource-exhausted mapping moved into it. This applies to every action and every provider. YouTube keeps its own reason-based mapping.
  • The wording heuristic is removed from writeBan. Known Twitch phrasings ("already banned", "not banned", …) still get their specific messages.
  • Still alerts: 5xx responses, 401/403, a 4xx with no message, an empty or non-JSON body, and transport failures.

The "agent bugs should still alert" guarantee no longer depends on guessing from provider text at runtime. It now comes from test-time contracts: TestTwitchModerationAndRaids and TestKickActions already check the query parameters and body of every request these actions send.

Tests

  • New unban refusal naming its parameter row, using Twitch-style wording. It fails on main with a 500 and passes here.
  • New TestProviderRejectionsAcrossEndpoints covers Twitch delete/clear and Kick ban/unban/delete across 400/404/409/422. It also checks that unexplained, non-JSON, non-string-message and 5xx responses still alert, and that provider text never reaches logs or responses.
  • Removed the four Report unrecognized Twitch ban and timeout refusals as rejections #1752 rows that pinned wording-based "malformed" detection, since that behavior was the bug.

go test ./... passes except the internal/youtube live tests, which need youtube.com (blocked in the sandbox where this was developed).

Known gaps

  • 401/403 still alert on purpose. For moderation that usually means the user isn't a moderator or a scope is missing, which may deserve its own user-facing mapping. That's left out of scope here.

Generated by Claude Code

Twitch refused an unban with HTTP 400 and the agent alerted with a 500:
the refusal's wording mentions the "user_id query parameter", which the
keyword check added in #1752 took for a malformed request. Twitch delete
and clear, and every Kick action, had no refusal handling at all, so any
explained 4xx from them alerted the same way.

HTTPError now records whether the provider explained a rejection (a
non-blank JSON message), never the text. The shared chat wrapper turns
an explained 400, 404, 409 or 422 into a failed-precondition rejection
for every action and provider, next to the existing 429 mapping. The
wording heuristic is gone: the request shapes these actions send are
pinned by the Twitch and Kick action tests instead, so agent bugs are
caught before deploy rather than by guessing from provider text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EzdqAWrbTMdtJutmK41coy
@kevmo314
kevmo314 merged commit 3dc90af into main Nov 1, 2026
8 checks passed
@kevmo314
kevmo314 deleted the claude/eager-dijkstra-hdrup8 branch November 1, 2026 07:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants