Skip to content

feat: add OrcaRouter as a named LLM provider - #998

Open
JinhaoSong322 wants to merge 1 commit into
dataelement:mainfrom
JinhaoSong322:add-orcarouter-provider
Open

feat: add OrcaRouter as a named LLM provider#998
JinhaoSong322 wants to merge 1 commit into
dataelement:mainfrom
JinhaoSong322:add-orcarouter-provider

Conversation

@JinhaoSong322

Copy link
Copy Markdown

Summary

Add OrcaRouter as a first-class, named LLM provider.

OrcaRouter is an OpenAI-compatible LLM gateway (https://api.orcarouter.ai/v1, keys prefixed sk-orca-) that routes to leading models behind one endpoint. It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

This change mirrors the existing openrouter wiring:

  • backend/app/services/llm/client.py — register orcarouter in PROVIDER_REGISTRY (openai-compatible protocol, default_base_url=https://api.orcarouter.ai/v1, supports_tool_choice=True, default_max_tokens=16384).
  • frontend/src/pages/enterprise-settings/tabs/LlmTab.tsx — add orcarouter to FALLBACK_LLM_PROVIDERS so it appears in the Enterprise → LLM provider picker (the live /enterprise/llm-providers endpoint serves the same registry).
  • backend/tests/test_orcarouter_provider.py — new tests pinning the registry entry, manifest round-trip, base-url resolution, and client creation.
  • README.md — document the OrcaRouter provider.

No database migration is needed: provider ids are stored as strings and the registry is the single source of truth for base URL / protocol.

Checklist

  • Tested locally
  • No unrelated changes included

Verification

  • pytest tests/test_orcarouter_provider.py → 4 passed; full suite → 2177 passed, 2 pre-existing html_to_pdf env failures (missing libgobject-2.0-0) + 1 pre-existing test_sso_toggle collection error (missing tests.test_auth), both present on clean main.
  • Frontend tsc --noEmit → clean; existing 8 test failures are pre-existing on main.
  • Live call through the repo's own create_llm_client(provider="orcarouter", ...) against https://api.orcarouter.ai/v1/chat/completions → HTTP 200, model orcarouter/autoqwen3.6-flash, returned ORCA-OK; streaming SSE path also returns 200.

Disclosure: I'm an engineer on the OrcaRouter team.

Register OrcaRouter in the backend provider registry and the frontend
fallback provider list, pointing at https://api.orcarouter.ai/v1 with an
openai-compatible protocol. Add registry tests.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: JinhaoSong322 <jinhao.song@myflashcloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant