Support structured outputs on the chat routes

[EmBista]

ChatMock reads response_format off the request and then drops it. Upstream never sees a schema, answers 200, and the model returns whatever shape it likes — so a caller asking for structured output gets valid JSON that isn't the JSON they asked for:

// asked for
{ "city": "string", "temperature": "integer" }

// came back
{ "location": "Paris, France", "temp_c": 18, "conditions": "cloudy" }

Nothing errors and nothing warns, so it looks like it worked until something reads a field that isn't there. Clients that validate the reply against the schema — the Vercel AI SDK's generateObject, say — fail at the parse step instead, with no hint that the schema was never sent.

Why it happened

Chat Completions calls this response_format; the Responses API carries it at text.format. The chat routes build the upstream payload field by field, and there was no mapping between the two spellings, so the field had nowhere to go. Nothing was missing upstream: /v1/responses already forwards text untouched, and posting a Responses payload through ChatMock's own authenticated client confirms the backend accepts text.format json_schema.

There is a second half. Once a schema does reach the model the content is supposed to be JSON, but the default think-tags mode prepends <think>…</think> to it — a prefix no JSON parser accepts.

Changes

• a json_schema response_format → text.format, accepting both the nested Chat Completions spelling and the flat Responses one
• Ollama's format → the same, when it holds a schema
• when the caller asks for JSON, reasoning-compat falls back to legacy so the content stays parseable. This is a precedence call and worth your say-so: think-tags is a deliberate setting, and it is being overridden per-request. My reasoning is that the two cannot both be satisfied — no client wants content that is at once valid JSON and prefixed with <think> — so the request-specific ask beats the server default, and reasoning is relocated rather than lost (message.reasoning_summary). Note the collision already exists on main: any client that prompt-injects a schema gets think tags glued to its JSON today, silently. What changes here is that a declared response_format finally tells ChatMock the content is a machine contract. Happy to put it behind a flag instead. It applies only in think-tags mode: the others already keep reasoning out of content, as upstream does by itself, and forcing legacy over o3 would change the type of message.reasoning
• 10 tests added, 28 passing

json_object, and Ollama's bare format: "json", are deliberately not forwarded. Upstream refuses that format unless the input happens to mention "json", so sending it would turn requests that pass today into 400s for a reason the caller cannot see. They still get the compat fallback, so ollama.chat(..., format="json") now returns content a JSON parser accepts — the override keys off what the caller asked for, not off what was forwarded.

How to try locally

python chatmock.py serve --port 8000

# on main: content is "<think>…</think>" + JSON in a shape of the model's choosing.
# with this: exactly {"city": …, "temperature": …}, reasoning moved to reasoning_summary.
curl -s http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model":"gpt-5.6-sol",
  "messages":[{"role":"user","content":"Invent plausible weather for Paris."}],
  "response_format":{"type":"json_schema","json_schema":{"name":"weather","strict":true,
    "schema":{"type":"object","additionalProperties":false,"required":["city","temperature"],
      "properties":{"city":{"type":"string"},"temperature":{"type":"integer"}}}}}}'

# same expectation on the Ollama route
curl -s http://127.0.0.1:8000/api/chat -H 'Content-Type: application/json' -d '{
  "model":"gpt-5.6-sol","stream":false,
  "messages":[{"role":"user","content":"Invent plausible weather for Paris."}],
  "format":{"type":"object","required":["city"],"properties":{"city":{"type":"string"}}}}'

python -m unittest tests.test_routes

Run against the live backend on gpt-5.6-sol, gpt-5.6-luna, gpt-5.6-terra and gpt-daybreak-blue-latest, streaming and non-streaming, on both routes, and across all four --reasoning-compat modes.

Checklist notes

• Rebased on latest main. No issue reference — opening this cold; happy to file one first if you prefer that order.
• README.md updated (Features list). DOCKER.md not required: no new flags, env vars or ports, since the capability is per-request.
• Defaults unchanged: a request sending neither field adds no text to the upstream payload and still gets its reasoning in think tags. Three tests pin that.
• Ollama routes updated alongside the OpenAI ones, per CONTRIBUTING. One related edit there: the two reasoning-compat reads went straight to current_app.config, so the handler's own local could never affect them; they now read the local, same value unless a request overrides it.
• No entry points moved and no existing parameter names or payload shapes changed.
• Tests kept deliberately thin, and happy to thin them further or drop them entirely if you would rather own that. The one thing I would keep is the wire-level class: route tests mock start_upstream_request, so they pass whether or not the schema ever reaches the payload — I had exactly that, a green suite over a feature that did nothing. Those tests patch requests.post and assert the real outbound payload. Each hunk was checked by reverting it and confirming a test fails.
• Two behaviour changes worth knowing: a caller sending strict: true with a schema whose properties are not all required now gets upstream's 400 rather than a silent 200 — note the Vercel AI SDK sets that flag by default once structured outputs are enabled. And on the Ollama route reasoning is dropped rather than relocated when JSON is requested; that is pre-existing legacy behaviour which a schema request now reaches.

Disclosure

AI was used to write this patch (Claude Code). Everything it claims was verified against the live backend and the test suite rather than taken on faith, but review it as you would any patch from a stranger. Happy to rework anything, drop the strict forwarding if you would rather this PR changed no status codes at all, or close it if it is not a direction you want the project to go.