Skip to content

fix(bub): skip automatic recall when the prompt has no UTF-8 form - #2666

Open
Fengzdadi wants to merge 1 commit into
oceanbase:masterfrom
Fengzdadi:fix/bub-surrogate-fail-open
Open

Fengzdadi wants to merge 1 commit into
oceanbase:masterfrom
Fengzdadi:fix/bub-surrogate-fail-open

Conversation

@Fengzdadi

Copy link
Copy Markdown
Contributor

Which issue or RFC does this PR close?

Part of #2250 (automatic recall in the host adapters). It applies the fail-open contract from #2647 to the Bub plugin.

Rationale for this change

A prompt can hold an unpaired surrogate, for example a file name decoded with surrogateescape or a JSON \ud800 escape. Such a string has no UTF-8 form. The Bub plugin bounds the recall query with _bounded_recall_query, which encodes the query to UTF-8, and it does this before its client error handling. So before_llm_call raised UnicodeEncodeError out of the plugin.

#2647 fixed the same class of failure for Hermes, LangChain, LangGraph, and Pydantic AI: when the query has no UTF-8 form, automatic recall is skipped and the turn proceeds. Bub is the remaining in-process Python adapter.

What a Bub 0.6.1 user sees on master: Bub's PluginAgentHooks._safe_call_one catches the exception and logs hook.agent_hook_failed hook=before_llm_call adapter=powercontext as a WARNING with a traceback into powercontext_bub/plugin.py. Then Bub continues without injected context. The turn still fails afterwards, but in Bub itself: encoding the model request (httpx2/_content.py) and appending the tape entry (bub/core/store.py) both raise UnicodeEncodeError for the same prompt. This change does not make such a turn succeed on Bub 0.6.1. It removes the plugin's own failure, so the plugin does not depend on Bub's hook guard, and the log no longer blames PowerContext for a prompt that Bub itself cannot send.

What changes are included in this PR?

  • integrations/bub/src/powercontext_bub/plugin.py: _prepare_context bounds the query inside a UnicodeEncodeError handler. For a query without a UTF-8 form it sends no prepare request, returns no context, and writes a capture-log record with event: "context", status: "skipped", and error: "UnicodeEncodeError". Prepare failures keep their existing status: "failed" record. before_llm_call then returns None, so Bub's model request goes on unchanged.
  • The recall-query tail rule (fix(integrations): bound the recall query in Pi, OpenCode, DSH, and Bub #2252) and the recall text taken from the inbound message (fix(bub): recall with the user's text, not Bub's channel frame #2488) are unchanged. A long prompt whose unencodable text falls before the kept tail still recalls with that tail, because the tail path encodes only the kept text.
  • Capture was already safe: render_capture_event serializes with ensure_ascii=True, so the user_prompt event carries the surrogate as a JSON escape and is still captured. save_state only flushes and sends no prompt text.

The new record needs no schema change: CaptureRecord in e2e/bub already accepts any status and an error name, and the evaluation counts only status: "ready" context records.

Are there any user-facing changes?

No API or configuration changes. For a prompt without a UTF-8 form, the plugin no longer raises from before_llm_call, and Bub no longer logs a PowerContext hook failure for it. With capture_log set, the JSONL file gets a context record with status: "skipped". The integration docs do not describe this failure case, so no docs change.

How was this change tested?

  • New tests in e2e/bub/tests/test_bub_recall_query.py call the plugin's before_llm_call hook with capture enabled and a capture log:
    • test_recall_is_skipped_when_the_query_has_no_utf8_form[short] and [in-the-kept-tail]: the hook returns None, no prepare request is sent, the user_prompt event is captured as ASCII content, and the capture log reads user_prompt captured, then context skipped. Both cases fail on master with UnicodeEncodeError.
    • test_recall_keeps_the_tail_when_the_unencodable_text_falls_before_it: the prepare query is the bounded tail. It passes on master too and guards against skipping every prompt that holds a surrogate.
  • e2e/bub/.venv/bin/python -m pytest e2e/bub/tests: 326 passed.
  • A full Bub 0.6.1 turn through BubFramework.process_inbound, with a stub model transport and a surrogate prompt, was run before and after the change (an ad-hoc probe, not committed). Before: hook.agent_hook_failed hook=before_llm_call adapter=powercontext, then Bub's own model request encoding fails. After: no PowerContext hook warning; Bub's model request encoding still fails.
  • make check and make harness-check pass.

Not validated: no real Bub run against a model provider or a running PowerContext Server.

Self-review against REVIEW.md and an independent Opus review of the diff found no blocking defect; their non-blocking notes follow.

Notes (non-blocking)

  • A prompt within 8,192 characters but over 8,192 UTF-8 bytes is bounded by its whole-query encode first, so a surrogate early in it skips recall even though the byte cut would have dropped it (for example "caf\udce9.txt " + "é" * 5000). The shared bounded_recall_query used by the hooks and the fix(integrations): recall with the end of a long prompt in Hermes, LangChain, LangGraph, and Pydantic AI #2647 adapters behaves the same way; this change keeps the Bub copy identical rather than diverging for prompts that already hold invalid text.
  • Two pre-existing encode sites sit outside the prompt path and are not changed here: workspace_binding_key encodes the resolved workspace path in load_state, which Bub does not isolate, so a Linux workspace directory with a non-UTF-8 byte fails the turn; _source_id encodes the session id for capture. Neither comes from prompt text.
  • The model-facing tools (powercontext.search, powercontext.context, powercontext.remember) still raise for a surrogate argument; Bub's tool runner reports that as a tool error, so the turn continues.

AI usage statement

Claude Code (Claude Opus 5.5) wrote this change and its tests, ran the checks listed above, and drafted this description.

A prompt holding an unpaired surrogate, such as a file name decoded with
surrogateescape, made _bounded_recall_query raise UnicodeEncodeError before the
plugin's client error handling, so before_llm_call raised out of the plugin.
Recall is now skipped for such a query: no prepare request is sent, the model
request goes on unchanged, and the capture log records a skipped context. A
long prompt whose unencodable text falls before the kept tail still recalls.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant