feat: assistant-prefill for prompt-based tools, structured output + KV-cache opt-ins - #122
Merged
Merged
Conversation
…files, prefill Adds the full design for removing the legacy champ.* settings layer in favour of a single guided setup wizard, plus runtime context-window truth, task-aware decode profiles, and prefill/structured-output workstreams. Ticket map: #118-121.
…d of a hardcoded 8192 Adds a context-window resolver that prioritises the runtime num_ctx from /api/show parameters and modelfile over model_info defaults, caps the effective window with providers.<id>.contextWindow from YAML, and threads that cap through ProviderFactory into every provider config. Tests: num_ctx precedence, cap semantics, degenerate-value guards. Closes #119
Adds per-task decode defaults (deterministic tool calls at 0.2, tighter coding turns, fixed autocomplete seed) with an effective-parameter merge: explicit request options win, then the YAML providers.<id>.options block, then the task profile, then built-in defaults. Backend field mapping: Ollama native options.* (temperature, top_k, min_p, repeat_penalty, seed, stop), OpenAI-compatible camelCase fields on the wire. Config schema now validates options.* types and bounds and rejects a non-integer contextWindow at load time. Tests: resolver precedence, per-backend request-body mapping, schema bounds. Closes #120
Closes #118 The configure-provider command walks provider -> endpoint -> (401?) API key capture -> live model discovery and merges the result into the effective .champ/config.yaml via the new merge-preserving writer, then hot-reloads the provider. Cloud providers with a stored key keep their current model; self-hosted providers only prompt for a key when the endpoint probe actually returns 401/403. configSource is gone: YAML is the single source of truth. The legacy workspace-settings layer, its champ.* flat keys, and the createFromConfig() factory path are deleted so there is nowhere left to fall back to.
…V-cache opt-ins
Prompt-based local models often narrate ('Let me check that file…')
instead of emitting a tool call, wasting a round-trip and polluting
history with fake results. On tool-continuation turns we now append a
trailing assistant seed (the <tool_call> opener plus <name>) so the
backend begins its next output inside the XML format. If the model
answers with prose instead of completing a call, the seed is stripped
before it reaches history or the UI.
Also adds two per-provider YAML opt-ins:
- structuredOutput: pins response_format/format json on chat bodies,
never on tool-call turns (the XML tool prompt needs free text)
- cachePrompt: sends options.cache_prompt so the backend keeps the
processed prompt across turns instead of recomputing the full prefix
Closes #121
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: assistant-prefill for prompt-based tools, structured output + KV-cache opt-ins
Prompt-based local models often narrate ('Let me check that file…')
instead of emitting a tool call, wasting a round-trip and polluting
history with fake results. On tool-continuation turns we now append a
trailing assistant seed (the <tool_call> opener plus ) so the
backend begins its next output inside the XML format. If the model
answers with prose instead of completing a call, the seed is stripped
before it reaches history or the UI.
Also adds two per-provider YAML opt-ins:
never on tool-call turns (the XML tool prompt needs free text)
processed prompt across turns instead of recomputing the full prefix
Closes #121
Tests