Repository navigation
perf(headless-runner): don't download files the model can't read - #7418
Merged
Merged
Conversation
A kept file was downloaded from the files bucket and base64-encoded before its type and size were checked, so a 100 MB video cost about 230 MB of memory to produce a one-line note. The same happened for an image over about 3.7 MB, a PDF over 10 MB and an Office file over 12 MB, for both attachments and open_file. The saved-file row already has the name, type and size. SavedFiles.reading now asks unreadableNote first and fetches the bytes only for a file that can be read. readToolFile uses the same check, so the notes are worded exactly as before. Signed-off-by: Jalil <reachjalil@gmail.com>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is this about?
Workbot's runner no longer downloads a kept file when its type or size already says the model can't read it.
What problem does it solve?
Attaching a large video, or opening one again with
open_file, made the runner hold the whole file in memory twice just to tell the model it can't be opened. A 100 MB video cost about 230 MB of memory in a cell that serves all of a person's chats.What was the situation before?
The runner fetched the file from the files bucket, base64-encoded it, and only then looked at its type and size. That happened for audio, video, archives, old Office formats and unknown types, and for an image over about 3.7 MB, a PDF over 10 MB or an Office file over 12 MB.
How is this implemented?
tool-files.ts: the rules for "this can't be read at all" move into one check on name, type and size, exported asunreadableNote.readToolFilecalls the same check, so every note keeps its exact wording.saved-files.ts:SavedFiles.readingasksunreadableNotefrom the saved-file row and fetches the bytes only when the file can be read. Attachments andopen_fileboth go through it.What else changed?
package.json: atest:saved-filesscript for the new test.Release note
none
Evidence
ee/apps/headless-runner/scripts/saved-file-reading.test.ts— before: opening a 100 MB video fetched it from the bucket; after: the same note comes back with no fetch, and a readable file is still fetched and read. No E2E.