Skip to content

fix(argent-mcp): return the screenshot-diff diff paths over MCP - #1378

Closed
latekvo wants to merge 5 commits into
mainfrom
fix/screenshot-diff-mcp-diffpath
Closed

latekvo wants to merge 5 commits into
mainfrom
fix/screenshot-diff-mcp-diffpath

Conversation

@latekvo

@latekvo latekvo commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Closes #826

Over MCP, screenshot-diff returned only the downscaled context image and the summary. Both diff files were materialized locally, but their paths were never sent, even though the summary tells the agent to "see diffPath (full size) and contextDiffPath in this result". The adapter now adds diffPath: <local path> and contextDiffPath: <local path> text blocks after the summary. If an artifact could not be downloaded, its line reads (unavailable), as flow-step artifacts already do.

No docs update needed. The tool description already lists both fields among the returned fields.

Verification

The E2E runs the real built tool-server (ARGENT_TOOLS_URL, no device) and the real startMcpServer over stdio with an @modelcontextprotocol/sdk client. It calls screenshot-diff on two generated 400x800 PNGs.

Before (main):

[0] image base64Bytes=1136
[1] text: "Screenshot diff summary ... - diff_images: see diffPath (full size) and contextDiffPath in this result ..."
current-diff.png anywhere: false

After:

[0] image base64Bytes=1136
[1] text: "Screenshot diff summary ..."
[2] text: "diffPath: /var/folders/.../argent-screenshot-diff/d32dc1ea68f7/current-diff.png"
[3] text: "contextDiffPath: /var/folders/.../argent-screenshot-diff/d32dc1ea68f7/current-context-diff.png"
diffPath local exists: true dims=400x800 ; context image dims=120x240

Edge cases, covered by test/content.test.ts against the real materializer:

  • full-size download fails (404): diffPath: (unavailable), and the context image and path still render
  • both fields absent (dimension_mismatch): no path lines
  • remote artifact handles: rewritten to downloaded local paths

Checks: tsc --build, tsc -p tsconfig.test.json, argent-mcp vitest 124/124, eslint, prettier, knip. The new tests fail 3/43 against the previous content.ts. Every mutant of the new lines tried (dropping the (unavailable) fallback, dropping contextDiffPath, dropping the absent-field skip, reading the un-materialized value) fails at least one test.

Out of scope, found while reviewing: #1574 (remote downloads with the same filename overwrite each other).

The MCP adapter built the screenshot-diff result from the context diff
image and the summary only, so the materialized full-size diff never
reached the agent. Add its local path as a text block.

Fixes #826
@coderabbitai

This comment was marked as spam.

@latekvo
latekvo marked this pull request as ready for review October 12, 2026 18:49
…navailable

The summary refers the agent to both diffPath and contextDiffPath; emit
both, and say (unavailable) when an artifact could not be fetched instead
of dropping the line.
@latekvo latekvo changed the title fix(argent-mcp): return the screenshot-diff diffPath over MCP fix(argent-mcp): return the screenshot-diff diff paths over MCP Oct 24, 2026

@latekvo latekvo left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Diplomat, Opus 5.5]: Out of scope, filed separately: #1574. When the tool-server is remote, two downloaded artifacts with the same filename (for example two diffs of a current.png) land at one local path, so a diffPath this PR hands to the agent can later hold a different call's diff. Confirmed by calling materializeArtifacts twice from the built tools-client dist: same path, and the first file's bytes were replaced by the second's.

@latekvo

latekvo commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

Folded into #1753 as cf6f92a, to review together with the other capture fixes.

@latekvo latekvo closed this Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

screenshot-diff over MCP delivers only the downscaled diff; diffPath reaches the agent by no route

1 participant