Skip to content

Add first-class local Flint provider - #1905

Open
justrach wants to merge 1 commit into
mainfrom
feat/native-flint-provider
Open

justrach wants to merge 1 commit into
mainfrom
feat/native-flint-provider

Conversation

@justrach

Copy link
Copy Markdown
Owner

What changed

  • Registers nanoattention/flint as a built-in OpenAI-compatible local provider, with chat and model discovery sharing one validated loopback endpoint.
  • Supports the GRAFF_NANOATTENTION_URL endpoint override and explicit GRAFF_CONTEXT; adds a conservative 32K default context row.
  • Adds routing, endpoint, context, request-shape, and missing-credential coverage; updates Lean conformance and generated SDKs.
  • Documents setup and the external-runtime boundary.

Why

Previously, using this local runtime required a project router entry. That becomes invalid once nanoattention is built in because router configuration correctly rejects collisions with reserved provider IDs. Reusing Codegraff's existing OpenAI-compatible local-provider path adds native routing without inventing a new protocol.

The inference runtime and model checkpoint remain separately installed prerequisites. This PR does not vendor engine code, bundle or download weights, modify global settings, or fall back to a cloud provider. Endpoint overrides are restricted to loopback so prompts cannot be redirected to a remote host through this setting.

Expose nanoattention/flint through the existing OpenAI-compatible local-provider path, with a validated loopback endpoint shared by chat and model discovery. This replaces manual router wiring that conflicts with reserved built-in IDs while keeping inference in the separately installed runtime. The provider fails closed without local credentials; no weights are downloaded or engine code vendored.

Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant