Skip to content

Add image generation, and fix non-streaming /v1/responses returning empty output - #117

Open
mazxxy wants to merge 5 commits into
RayBytes:mainfrom
mazxxy:image-generation
Open

Add image generation, and fix non-streaming /v1/responses returning empty output#117
mazxxy wants to merge 5 commits into
RayBytes:mainfrom
mazxxy:image-generation

Conversation

@mazxxy

@mazxxy mazxxy commented Aug 15, 2026

Copy link
Copy Markdown

This started as a fork I needed for myself: I wanted image generation out of a Codex account, and there was no way to get one out of ChatMock. While building it I ran into a bug that is worth fixing on its own, so I'm sending both here instead of keeping it private.

The bug underneath

response.completed from the codex backend always carries "output": []. The items only ever exist in the response.output_item.done events. Since aggregate_response_from_sse() returns the object from the final event, every POST /v1/responses with stream: false answers with an empty output — plain text included, not just images. It looks like a success: status: "completed", usage filled in, no content.

On current main:

curl http://127.0.0.1:8000/v1/responses -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.4-mini","input":"say ok","stream":false}'
# "output": []

Items are now collected by output_index and rebuilt into the final object when it comes back empty. Streaming was never affected.

Image generation

The endpoint already accepts the image_generation tool; only ChatMock had no way to receive the result. What the backend actually does, measured rather than assumed:

  • the drawing model is gpt-image-2-codex, chosen server-side
  • tool_choice: {"type": "image_generation"} works and forces the call
  • consumption shows up in tool_usage.image_gen, so it comes off the ChatGPT plan quota
  • n inside the tool is refused (Unknown parameter: 'tools[0].n'), so n > 1 repeats the request, capped at 4 and stated in the error message
  • size and quality are accepted but echoed back as "auto"; the resolution follows the prompt, so the requested size is written into the instructions as well. Asking for 1536x1024 landed exactly; asking for 1024x1024 returned 1254x1254

What that turns into:

  • /v1/images/generations in the OpenAI Images shape, so existing clients work unchanged. response_format: "url" is rejected, mirroring gpt-image-1
  • image_generation allowed in responses_tools on /v1/chat/completions and /api/chat, with the image embedded as a data URL in the message content
  • --image-model / CHATGPT_LOCAL_IMAGE_MODEL for the model that orchestrates the call
  • 9 tests added, 27 passing

One detail worth calling out: in streaming chat the image is emitted only after the think tag is closed. Without that it lands inside <think> and disappears in any client that collapses reasoning, which is the whole point of that mode. The close-tag block was duplicated twice in sse_translate_chat(); it is now a single _close_think_tag() helper used in all three places.

How to try locally

python chatmock.py serve --port 8000

# the fix: this returned an empty output before
curl http://127.0.0.1:8000/v1/responses -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.4-mini","input":"say ok","stream":false}'

# image, OpenAI Images shape
curl http://127.0.0.1:8000/v1/images/generations -H "Content-Type: application/json" \
  -d '{"prompt":"a minimalist geometric fox logo","size":"1024x1024"}'

# image inside a chat request
curl http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.4-mini","messages":[{"role":"user","content":"draw a blue cube"}],"responses_tools":[{"type":"image_generation"}]}'

python -m pytest tests -q

Every path above was run against the live backend, streaming and non-streaming, on both the OpenAI and Ollama routes.

Checklist notes

  • Rebased on latest main. No issue reference — opening this cold; happy to file one first if you prefer that order.
  • README.md and DOCKER.md updated (new flag, new env var, usage section).
  • Defaults unchanged: the image path only activates when a request asks for the tool. The aggregator fix does change existing behaviour, but only from "empty" to "the content that was already paid for".
  • Ollama routes updated alongside the OpenAI ones, per CONTRIBUTING.
  • No entry points moved and no existing parameter names or payload shapes changed.

Disclosure

AI was used to write this patch (Claude Code). Everything it claims was verified against the live backend and the test suite rather than taken on faith, but review it as you would any patch from a stranger. Happy to rework anything, split the aggregate_response_from_sse fix into its own PR since it stands alone, or close this if it's not a direction you want the project to go.

Vitor - Obliq Studios and others added 5 commits August 15, 2026 12:13
O tool image_generation ja era aceito pelo backend, mas a imagem nunca
chegava ao cliente: nao havia rota de imagem, e o caminho nao-streaming
descartava o resultado.

Descartava porque o response.completed do Codex vem sempre com output
vazio -- os itens so existem nos eventos output_item.done. Entao /v1/responses
com stream:false respondia vazio para tudo, texto inclusive, com cara de
sucesso. Agora os itens sao remontados por output_index.

Em cima disso: /v1/images/generations no formato da Images API, e a imagem
embutida como data-url no /v1/chat/completions (streaming e nao-streaming).

n vira requisicao repetida porque o backend recusa tools[0].n, e size vira
instrucao em texto porque o backend ignora o parametro e escolhe a proporcao
pelo prompt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…repo

A imagem era emitida como delta de conteudo sem fechar o bloco de
raciocinio antes. Resultado: <think>...![img](data:...)</think> — em
qualquer cliente que esconde o think-tag, e para isso que o modo existe,
a imagem sumia junto. O teste anterior contava chunks e nao olhava onde
o conteudo caia, entao passava.

O fechamento do think estava duplicado em dois pontos do upstream; virou
_close_think_tag(), usado agora nos tres.

Resto e alinhamento com o codigo que ja existia: comentarios e mensagens
de erro em ingles, erro sem o campo "type" que nenhuma outra rota usa,
X-Session-Id respeitado como nas demais rotas, e collect_images_from_sse
movido para responses_api, que e o modulo que ja le SSE — o que dispensa
o import tardio que existia so para driblar o ciclo de imports.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The codex responses endpoint already accepts the image_generation tool and
streams the result back, but nothing in ChatMock could surface it: there was
no image route, and the non-streaming path dropped the result.

It dropped it because response.completed from the backend always carries
"output": [] -- the items only ever exist in the response.output_item.done
events. So aggregate_response_from_sse(), which returned the object from the
final event, made every POST /v1/responses with stream:false answer empty,
plain text included, while still reporting status "completed" with usage
filled in. Items are now collected by output_index and rebuilt when the final
object comes back empty.

On top of that:

- /v1/images/generations, matching the OpenAI Images API shape
- image_generation accepted in responses_tools on both /v1/chat/completions
  and /api/chat, with the image embedded in the message content as a data
  URL. It is emitted after the think tag is closed, otherwise it lands inside
  <think> and is hidden by every client that collapses reasoning
- --image-model / CHATGPT_LOCAL_IMAGE_MODEL for the orchestrating model

Two backend limits shape the implementation: n is refused inside the tool
("Unknown parameter: 'tools[0].n'"), so n>1 repeats the request; and size is
echoed back as "auto" regardless, so it is also written into the instructions
where the model can act on it.
# Conflicts:
#	tests/test_routes.py
The empty-result error asserted the prompt "usually" hit moderation. Nobody had
verified that, and it was not knowable from here: the text the model produces
when it answers instead of drawing was collected by the stream reader and then
thrown away one function up. Whoever hits this needs the reason, not a guess.

The SSE collector now also gathers `message` items, and the route quotes them:

  The model finished without generating an image. The model answered instead of
  drawing: "..."

No image and no text is a different failure and says so, since it means the
request was stopped before the model started working.

Two tests cover both branches. Sabotaged by dropping the text again: the first
one fails, as it should.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant