Add Anthropic Messages API Routing

[Ratherrayees]

Summary

Added support for routing requests to the Anthropic Messages API via PAIR's unified nvpair-proxy service whether using Ollama or LM Studio.

Since the POST /v1/messages endpoint is an inference route, it makes use of PAIR's existing model selection, routing, and failover path without protocol translation.

It enables compatible clients such as Claude Code to make Messages API requests via PAIR and at the same time keep the upstream request/response format.

Fixes #16

Changes

Include the POST request to the /v1/messages endpoint in the shared inference route set that is used by both the Ollama and LM Studio facades.
Extend the route-role tests for both engine profiles.
Extend the failover coverage of unified-proxy for requests to Anthropic Messages.
Extend the test of the cross-process strict model-routing to include requests to Ollama and LM Studio via the /v1/messages endpoint.
Update the README file for the unified nvpair-proxy to include information about the Anthropic Messages route.

• Leave the existing client-facing ollama-proxy: and lmstudio-proxy: namespaces unchanged.
  We do not make any changes to services/versions.json or to CHANGELOG.md; the release metadata is provided via the required PR release-intent block listed below.

Design

The implementation employs passthrough rather than protocol translation; PAIR simply adds the /v1/messages endpoint to the current inference-routing interface, after which the chosen engine receives the original request and returns a response in the original format.

The same system components that are used by the other inference paths—that is, the model eligibility mechanism, the candidate selection process, the scheduler reservation, the first-byte commit, the retry function, and the failover mechanism—are also responsible for handling this route.

There is no Anthropic-specific routing subsystem or separate proxy process.

Validation

Automated

• cd services/nvpair-proxy && go test -count=1 ./... . PASS.
  cd services/tests && go test -count=1 -run 'TestStrictModelRoutingAcrossProcesses' -v . yielded a pass in the cases involving Ollama, Ollama Anthropic, LM Studio, and LM Studio Anthropic.
  cd services/tests && go test ./... resulted in one failure in the TestBrokerProxySetPortRebinds test (in the file broker_supervision_test.go at line 551). This test also fails when run against the current origin/develop baseline, showing the same port/settings resolution error.
  The script node scripts/spdx-headers.mjs has passed; out of 1014 files checked, 0 were missing and 97 were skipped.
• git diff --check passes.

End-to-end

I verified that a genuine Claude Code session had been established via PAIR using Ollama. Through the routed session, Claude Code produced the output with the required content, thus confirming that there had actually been an interaction between an Anthropic client and the backend via PAIR.

Notes

The implementation is deliberately restricted so that the Anthropic Messages endpoint will make use of PAIR's current routing and failover facilities and will not carry out Anthropic-specific request/response translation.

Changelog title

Route Anthropic Messages API requests

Changelog body

The Ollama and LM Studio endpoints offered by PAIR now accept requests for the Anthropic Messages API via POST to the /v1/messages endpoint and direct them to the owner of the requested model, just as happens with the other inference methods.

Bumps

• services: minor
• nvpair-cluster-manager: none
• nvpair-engine-manager: none
• nvpair-errors: none
• nvpair-job-scheduler: none
• nvpair-manual-nodes: none
• nvpair-node-info: none
• nvpair-node-scanner: none
• nvpair-node-settings: none
• nvpair-proxy: minor
• nvpair-tui: none
• nvpair-ui-broker: none
• nvpair-workload-manager: none

[Noah-Tervalon-Nvidia]

Glad to have you here contributing to the project! This lgtm - I am currently working on some major redesigns for the proxies to unify them. I want to wait on merging in changes for the proxies until we get that out to avoid extra work in merging into the new version so I expect this will need a minor rework once we get that out and we'll wait on merging it until then.

[Ratherrayees]

Thanks for taking the to review it and heads-up on the proxy rework. Sounds good - I’ll hold off until the proxies are redesigned and then make the necessary adjustments once it’s ready. I’m looking forward to being able to contribute more to the project!

[kjlubick]

Notes: https://lmstudio.ai/docs/developer/anthropic-compat and https://docs.ollama.com/integrations/claude-code are where the currently supported engines document their Anthropic pass-through support (which this PR adds).

[kjlubick]

Thanks for adding this test. I'm going to upload a follow-on commit that reduces the duplication here with a table-based test (that seemed easier than me describing what I had in mind).

[kjlubick]

One more review comment, could you add something in the description about improving support for Claude Code (I'm guessing that's the motivation) or whatever problem you bumped into that inspired this change? It's nice to include the "how does this benefit the user" a bit more explicitly in changes. Thanks again for your contribution!

[Ratherrayees]

Thanks Kaylee! I’ve improved the PR description to better describe the user benefit and Claude Code compatibility. Also really appreciate you adding the test refactor based on tables, it's a much cleaner way to express the test coverage.

[Ratherrayees]

I really enjoy working with PAIR and look forward to doing more for the project.

Fingers crossed it gets merged!

[kjlubick]

Just to touch base here, we are working on getting our final batch of internal changes out into the public repo. Once those are in, we'll get this rebased (if necessary) and landed. Thanks again for your contribution!

[Noah-Tervalon-Nvidia]

Heads up — develop has just taken a change that merges the Ollama and LM Studio
proxies into a single nvpair-proxy process, hosting one facade per enabled
engine. services/ollama-proxy/ and services/lmstudio-proxy/ are gone.

This PR edits files in those directories, so it will need updating before it can
merge. The good news is that the client-facing ollama-proxy: and
lmstudio-proxy: namespaces are unchanged — only the process behind them moved —
so the change should port across to services/nvpair-proxy/ fairly directly.

Apologies for the churn, and thanks for the contribution. Happy to help work out
where it lands if it isn't obvious.

[Ratherrayees]

Hi Noah, thanks again for the kind words and for the heads-up about the proxy redesign.

I’ve now rebased the PR onto the current develop and ported the Anthropic Messages API support to the new unified nvpair-proxy architecture.

The implementation now:

• Adds POST /v1/messages to the shared inference route set used by both Ollama and LM Studio facades.
• Keeps the existing ollama-proxy: and lmstudio-proxy: client-facing namespaces unchanged.
• Reuses the existing model selection, eligibility, reservation, retry, and failover flow without introducing Anthropic-specific routing or protocol translation.
• Updates the unified proxy tests, cross-process strict model-routing coverage, and README.
• Incorporates the table-based test structure from the earlier review.

For validation:

• Unified nvpair-proxy tests pass.
• Anthropic routing/failover cases pass for both Ollama and LM Studio.
• The full services/tests suite has one failure in TestBrokerProxySetPortRebinds; I reproduced the same failure on a clean current origin/develop baseline with the same port/settings resolution error.
• SPDX and git diff --check pass.
• I also verified an end-to-end Claude Code session through PAIR using Ollama.

The updated branch is pushed at 7749b54. Thanks again for pointing me in the right direction with the redesign.

[kjlubick]

You and I were apparently trying to rebase at the same time. You got in first, I pushed a few other changes on top of yours (e.g. documentation and cleanup tests). I think we should be good to go

[Ratherrayees]

Hehe, looks like we were racing each other there 😄 Glad we got it sorted. Thanks for jumping in and cleaning things up!