Repository navigation
Conversation
hoangvu12
force-pushed
the
plugin-tool-images
branch
from
November 12, 2026 07:09
0552673 to
5311624
Compare
A plugin tool's answer reached the agent as one JSON text block, so a base64 PNG was a long string the model read as characters. Now a tool result with a top-level `imageContent` array goes out as MCP image content, after the rest of the result as text. Each entry must be a PNG, JPEG, WebP or GIF of at most 5 MB, and images stop before the reply reaches the 8 MB cap; one left out is named in a last text block. Such a result goes without `structuredContent`: Codex shows the model only that (no images) when it is there, and Claude Code shows it in place of the texts.
hoangvu12
force-pushed
the
plugin-tool-images
branch
from
November 12, 2026 08:08
5311624 to
9d35358
Compare
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A plugin tool has no way to show the agent a picture. Whatever
runreturns reaches the agent as one JSON text block, so a base64 PNG arrives as a long string the model reads as characters. We're building a design tool as a Luau plugin, and its agent needs to see the screens it draws.Now a tool result with a top-level
imageContentarray goes out as MCP image content, after the rest of the result as text:The change is in the MCP server's
tools/callhandler (crates/mcp/src/jsonrpc.rs), so it applies to any tool. Plugins are the ones that need it, because their answers pass through unchanged. Each entry must be a PNG, JPEG, WebP or GIF of at most 5 MB. An entry that fails this is left out, and a last text block tells the agent which one and why, so one bad image can't make the model's API reject the whole turn. Images also stop before the reply reaches the server's 8 MB cap (MAX_REPLY_BYTES), sincecap_replywould otherwise cut the reply to text and lose every image without saying so.Such a result goes without
structuredContent. When it's there, Codex gives the model onlystructuredContentand drops the images, and Claude Code shows it in place of the text blocks, which would hide the "left out" note. Results without animageContentarray go out exactly as before,structuredContentincluded.I checked this against the agents' own clients with a small MCP server that returns a text block and an image, with and without
structuredContent. Claude Code 2.1.310 and OpenCode 1.18.30 saw the image both ways. Codex 0.128.0 sends a non-nullstructuredContentalone (models.rs,as_function_call_output_payload), so it needs the image result without it. Devin's client passed the image to its model, but its free model has no vision.Documented in
apps/zgx/docs/plugins.md(Tools for agents). The SDK'szeron.luauheader is left alone, since its copies in every plugin must match it byte for byte (sdk_copies_match_canonical).Testing
Four new tests in
jsonrpc.rscall a plugin tool throughtools/callagainst an in-memory engine. They check that images arrive asimagecontent with the rest as text and nostructuredContent, that bad entries (wrong shape, invalid base64, unsupported type, over 5 MB) are left out and named, that images past the 8 MB reply cap are left out and the reply goes whole, and that results without an image array are unchanged.cargo test -p zeron-mcp --lib: 74 passed, 0 failed.rustfmt --checkandcargo clippy -p zeron-mcp --testsshow nothing new. The SDK and its 16 copies inplugins-builtinandplugins-devare unchanged and still identical, assdk_copies_match_canonicalrequires.Open questions
zeron.toolline in the SDK'szeron.luauheader could mentionimageContenttoo, at the next SDK sync.