Skip to content

docs: design of record for the local engine API - #44

Open
its-janghoon wants to merge 1 commit into
developfrom
feature/local-engine-openai-api
Open

its-janghoon wants to merge 1 commit into
developfrom
feature/local-engine-openai-api

Conversation

@its-janghoon

Copy link
Copy Markdown
Contributor

Three products are written against a local engine endpoint that has never
existed. redrob-browser POSTs to /experimental/external/chat/completions and
streams the reply; no such route is served, and the string "external" does not
appear under packages/redrob/src/server. redrob-reblend ranks a LOCAL transport
its own sidecar cannot use. redrob-office compiles this engine at release time,
ships it inside its installer behind a packaging guard, and never calls it.

The products that drive the engine successfully drive its session API. The ones
that tried to give it an OpenAI face are the broken ones. So this serves that
face for real, at one stable path.

Decisions recorded here:

  • POST /v1/chat/completions is the core path, with no alias. It sits on
    packages/llm and the provider registry, not on Session: a completion is
    stateless and a session is not, and local inference then falls out as just
    another resolved provider.
  • Tool ownership is decided by the request, not configured. tools present
    means the CALLER owns them and the engine returns tool_calls; absent means the
    engine runs its own. Office edits a document held in its renderer and browser
    drives a live tab, so for them caller ownership is the only correct answer.
  • Inbound auth is the server's existing Basic. Outbound is the engine's OWN
    Console credential, which is what makes one login serve every product.
  • 401 carries code "engine_not_authenticated", and that code alone may trigger a
    sign-in prompt. Branching on message text instead is how a network timeout
    shipped as "please log in".
  • Every /experimental route is promoted in one breaking rename. No product calls
    any of them; the only callers are this repo's generated SDKs, which
    httpapi-codegen regenerates. No compatibility window is owed to callers that
    do not exist.
  • Products stop bundling the binary. One shared daemon, recorded at
    ~/.redrob/run/server.json (0600), resolved via REDROB_CODE_BIN then
    $HOME/.redrob/bin then PATH. Per-product processes were rejected: with local
    inference that loads the same weights N times.
  • Nothing pins an engine version. /capabilities gains a chatCompletions entry
    and products declare a MINIMUM capability, so auto-update cannot break them
    silently. Compatibility is kept by rules plus a contract test that fails CI
    when a v1 field is removed, not by a promise.

No code changes. The route, the promotion, the contract test, and the per-product
migration follow as their own pull requests.

Three products are written against a local engine endpoint that has never
existed. redrob-browser POSTs to /experimental/external/chat/completions and
streams the reply; no such route is served, and the string "external" does not
appear under packages/redrob/src/server. redrob-reblend ranks a LOCAL transport
its own sidecar cannot use. redrob-office compiles this engine at release time,
ships it inside its installer behind a packaging guard, and never calls it.

The products that drive the engine successfully drive its session API. The ones
that tried to give it an OpenAI face are the broken ones. So this serves that
face for real, at one stable path.

Decisions recorded here:

- POST /v1/chat/completions is the core path, with no alias. It sits on
  packages/llm and the provider registry, not on Session: a completion is
  stateless and a session is not, and local inference then falls out as just
  another resolved provider.
- Tool ownership is decided by the request, not configured. `tools` present
  means the CALLER owns them and the engine returns tool_calls; absent means the
  engine runs its own. Office edits a document held in its renderer and browser
  drives a live tab, so for them caller ownership is the only correct answer.
- Inbound auth is the server's existing Basic. Outbound is the engine's OWN
  Console credential, which is what makes one login serve every product.
- 401 carries code "engine_not_authenticated", and that code alone may trigger a
  sign-in prompt. Branching on message text instead is how a network timeout
  shipped as "please log in".
- Every /experimental route is promoted in one breaking rename. No product calls
  any of them; the only callers are this repo's generated SDKs, which
  httpapi-codegen regenerates. No compatibility window is owed to callers that
  do not exist.
- Products stop bundling the binary. One shared daemon, recorded at
  ~/.redrob/run/server.json (0600), resolved via REDROB_CODE_BIN then
  $HOME/.redrob/bin then PATH. Per-product processes were rejected: with local
  inference that loads the same weights N times.
- Nothing pins an engine version. /capabilities gains a chatCompletions entry
  and products declare a MINIMUM capability, so auto-update cannot break them
  silently. Compatibility is kept by rules plus a contract test that fails CI
  when a v1 field is removed, not by a promise.

No code changes. The route, the promotion, the contract test, and the per-product
migration follow as their own pull requests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant