Add image generation, and fix non-streaming /v1/responses returning empty output - #117
Open
mazxxy wants to merge 5 commits into
Open
Add image generation, and fix non-streaming /v1/responses returning empty output#117mazxxy wants to merge 5 commits into
mazxxy wants to merge 5 commits into
Conversation
O tool image_generation ja era aceito pelo backend, mas a imagem nunca chegava ao cliente: nao havia rota de imagem, e o caminho nao-streaming descartava o resultado. Descartava porque o response.completed do Codex vem sempre com output vazio -- os itens so existem nos eventos output_item.done. Entao /v1/responses com stream:false respondia vazio para tudo, texto inclusive, com cara de sucesso. Agora os itens sao remontados por output_index. Em cima disso: /v1/images/generations no formato da Images API, e a imagem embutida como data-url no /v1/chat/completions (streaming e nao-streaming). n vira requisicao repetida porque o backend recusa tools[0].n, e size vira instrucao em texto porque o backend ignora o parametro e escolhe a proporcao pelo prompt. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…repo A imagem era emitida como delta de conteudo sem fechar o bloco de raciocinio antes. Resultado: <think>...</think> — em qualquer cliente que esconde o think-tag, e para isso que o modo existe, a imagem sumia junto. O teste anterior contava chunks e nao olhava onde o conteudo caia, entao passava. O fechamento do think estava duplicado em dois pontos do upstream; virou _close_think_tag(), usado agora nos tres. Resto e alinhamento com o codigo que ja existia: comentarios e mensagens de erro em ingles, erro sem o campo "type" que nenhuma outra rota usa, X-Session-Id respeitado como nas demais rotas, e collect_images_from_sse movido para responses_api, que e o modulo que ja le SSE — o que dispensa o import tardio que existia so para driblar o ciclo de imports. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The codex responses endpoint already accepts the image_generation tool and
streams the result back, but nothing in ChatMock could surface it: there was
no image route, and the non-streaming path dropped the result.
It dropped it because response.completed from the backend always carries
"output": [] -- the items only ever exist in the response.output_item.done
events. So aggregate_response_from_sse(), which returned the object from the
final event, made every POST /v1/responses with stream:false answer empty,
plain text included, while still reporting status "completed" with usage
filled in. Items are now collected by output_index and rebuilt when the final
object comes back empty.
On top of that:
- /v1/images/generations, matching the OpenAI Images API shape
- image_generation accepted in responses_tools on both /v1/chat/completions
and /api/chat, with the image embedded in the message content as a data
URL. It is emitted after the think tag is closed, otherwise it lands inside
<think> and is hidden by every client that collapses reasoning
- --image-model / CHATGPT_LOCAL_IMAGE_MODEL for the orchestrating model
Two backend limits shape the implementation: n is refused inside the tool
("Unknown parameter: 'tools[0].n'"), so n>1 repeats the request; and size is
echoed back as "auto" regardless, so it is also written into the instructions
where the model can act on it.
# Conflicts: # tests/test_routes.py
The empty-result error asserted the prompt "usually" hit moderation. Nobody had verified that, and it was not knowable from here: the text the model produces when it answers instead of drawing was collected by the stream reader and then thrown away one function up. Whoever hits this needs the reason, not a guess. The SSE collector now also gathers `message` items, and the route quotes them: The model finished without generating an image. The model answered instead of drawing: "..." No image and no text is a different failure and says so, since it means the request was stopped before the model started working. Two tests cover both branches. Sabotaged by dropping the text again: the first one fails, as it should.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This started as a fork I needed for myself: I wanted image generation out of a Codex account, and there was no way to get one out of ChatMock. While building it I ran into a bug that is worth fixing on its own, so I'm sending both here instead of keeping it private.
The bug underneath
response.completedfrom the codex backend always carries"output": []. The items only ever exist in theresponse.output_item.doneevents. Sinceaggregate_response_from_sse()returns the object from the final event, everyPOST /v1/responseswithstream: falseanswers with an empty output — plain text included, not just images. It looks like a success:status: "completed",usagefilled in, no content.On current
main:Items are now collected by
output_indexand rebuilt into the final object when it comes back empty. Streaming was never affected.Image generation
The endpoint already accepts the
image_generationtool; only ChatMock had no way to receive the result. What the backend actually does, measured rather than assumed:gpt-image-2-codex, chosen server-sidetool_choice: {"type": "image_generation"}works and forces the calltool_usage.image_gen, so it comes off the ChatGPT plan quotaninside the tool is refused (Unknown parameter: 'tools[0].n'), son > 1repeats the request, capped at 4 and stated in the error messagesizeandqualityare accepted but echoed back as"auto"; the resolution follows the prompt, so the requested size is written into the instructions as well. Asking for 1536x1024 landed exactly; asking for 1024x1024 returned 1254x1254What that turns into:
/v1/images/generationsin the OpenAI Images shape, so existing clients work unchanged.response_format: "url"is rejected, mirroringgpt-image-1image_generationallowed inresponses_toolson/v1/chat/completionsand/api/chat, with the image embedded as a data URL in the message content--image-model/CHATGPT_LOCAL_IMAGE_MODELfor the model that orchestrates the callOne detail worth calling out: in streaming chat the image is emitted only after the think tag is closed. Without that it lands inside
<think>and disappears in any client that collapses reasoning, which is the whole point of that mode. The close-tag block was duplicated twice insse_translate_chat(); it is now a single_close_think_tag()helper used in all three places.How to try locally
Every path above was run against the live backend, streaming and non-streaming, on both the OpenAI and Ollama routes.
Checklist notes
main. No issue reference — opening this cold; happy to file one first if you prefer that order.Disclosure
AI was used to write this patch (Claude Code). Everything it claims was verified against the live backend and the test suite rather than taken on faith, but review it as you would any patch from a stranger. Happy to rework anything, split the
aggregate_response_from_ssefix into its own PR since it stands alone, or close this if it's not a direction you want the project to go.