Skip to content

perf(images): downscale large images, prune old ones from context - #15

Merged
Amanbig merged 1 commit into
mainfrom
feat/image-token-savings
Oct 31, 2026
Merged

Amanbig merged 1 commit into
mainfrom
feat/image-token-savings

Conversation

@Amanbig

@Amanbig Amanbig commented Oct 31, 2026

Copy link
Copy Markdown
Owner

Problem

An attached image lives in the conversation history and is re-sent in full on every model call for the rest of the session (and on every tool-loop iteration). Large screenshots are also sent at full resolution, although providers downscale anything beyond ~1568px, so the extra bytes and tokens are wasted.

Changes

  • prune_old_images(): only the 2 most recent image-bearing messages keep their payload when building the request; older ones become a text note ([1 earlier image(s) omitted to save tokens: shot.png]) so the model knows and can ask for a re-attach. The stored history is not modified. Tunable via Agent.keep_recent_images
  • downscale(): with Pillow, images over 1568px on the long side are resized (JPEG q85 / WebP q85 / optimized PNG), only if the result is actually smaller. Animated GIFs, small images and undecodable files are left untouched. Without Pillow it is a no-op
  • Pillow is the optional extra devorch[images] and part of dev (CI installs .[dev])

Test plan

  • Prune keeps only recent images, doesn't mutate history, no-op within the limit, agent sends pruned history while keeping stored originals
  • Downscale: 4000x2000 -> 1568 long edge with aspect ratio kept and a smaller file; small images/GIFs untouched; garbage input survives
  • Full suite passes (258), ruff check / ruff format clean
  • Token numbers not measured against a live provider

Next

Anthropic prompt caching (system prompt + tools + history prefix) is the other big token lever; separate PR.

Images were re-sent in full on every model call for the rest of a session.
Keep image payloads only for the 2 most recent image messages and replace
older ones with a short text note (stored history is untouched). With
Pillow installed, downscale images above 1568px on the long side before
sending. Pillow is an optional 'images' extra and part of dev.
@Amanbig
Amanbig merged commit 1f0a685 into main Oct 31, 2026
9 of 13 checks passed
@Amanbig
Amanbig deleted the feat/image-token-savings branch October 31, 2026 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant