Opt-in, API-key-authenticated LAN access to the compatibility endpoints

[CryptoJones]

Description

Closes #28.

PAIR's Ollama- and OpenAI-compatible proxies refuse every plaintext request that does not arrive from loopback (403 loopback-only). That is the right default, but it leaves no supported way for an ordinary OpenAI-compatible client on a trusted LAN — an automation host, a container, a Kubernetes workload, a Windows workstation's SDK — to use a node without running PAIR itself. #28 asks for an explicit, documented opt-in; the relay described in its comments (#31) shows people are already working around the gate with something that does no authentication at all.

This adds an opt-in, API-key-gated path for non-loopback plaintext callers to the unified nvpair-proxy, enforced identically by every engine facade it hosts (Ollama and LM Studio):

• Default unchanged. With no key configured, a LAN caller still gets exactly the same 403 loopback-only. TestHandlePlainRejectsNonLoopback is untouched.
• Opt-in by configuring a key: a key file (NVPAIR_PROXY_API_KEYS_FILE, default proxy-api-keys in the PAIR data directory) and/or NVPAIR_PROXY_API_KEYS inline. One key per line, # comments; a key must be ≥ 32 characters from the RFC 6750 b64token alphabet (letters, digits, - . _ ~ + / =), generated randomly.
• With a key configured, a non-loopback caller presenting it as Authorization: Bearer <key> (or X-Api-Key: <key>, the Anthropic SDK convention — relevant to [Feature]: Expose the Anthropic Messages API (POST /v1/messages) on the PAIR proxy #16/Add Anthropic Messages API Routing #27; either header may carry it) is routed through the same local router a loopback client uses. Missing key → 401 unauthorized with WWW-Authenticate: Bearer realm="nvpair-proxy"; a presented but unknown key adds error="invalid_token" (RFC 6750 §3.1).
• Optional source allowlist NVPAIR_PROXY_ALLOWED_CIDRS; a caller outside it gets 403 source-not-allowed before its key is even examined.
• Loopback callers are never asked for a key — the desktop app, TUI, and local tools are unaffected. A loopback client's own Authorization header is forwarded untouched, as today.
• The key is stripped from an admitted request before it is forwarded, so PAIR's credential never reaches an engine or a peer.
• Fails closed everywhere. A key file that other users can read or that another user owns (Unix checks, made on the opened handle), contains a malformed entry, is a directory, or cannot be read contributes no keys; judged per source, so a refused file does not disable inline keys, and with no source yielding a key the LAN stays closed. Group read is refused even for a group the proxy belongs to, because the gate cannot tell a pod fsGroup from a shared group such as macOS staff; on Kubernetes the key goes in through NVPAIR_PROXY_API_KEYS from a Secret via valueFrom. The reason is logged once per distinct problem. A malformed inline key or CIDR disables the gate for the process — a mistyped allowlist must not silently leave the allowlist off.
• Hot reload. The key file is re-read per non-loopback request and swapped in when its content hash changes (a size/mtime stamp cannot see a same-length rewrite within the filesystem's timestamp granularity), so keys can be added, rotated, or revoked without a restart — matching how mesh.Refresh() already handles cluster membership. The file is a few short lines; the read is cheap.
• Secret hygiene. Keys are held in memory only as SHA-256 digests; comparison is crypto/subtle.ConstantTimeCompare over every configured digest with no early exit; a rejection logs the caller's address and the first 8 hex digits of the presented key's digest, never the key. Forwarding headers (X-Forwarded-For etc.) are never consulted. Enabling the gate logs a WARN at startup and on every key-set change.
• OPTIONS preflight gets no exemption. A keyless preflight from another machine is refused 401 like any keyless request, before any body is read or any engine or peer is contacted. A browser cannot send a key on a preflight, so a web page on another machine is not a supported client; it never was, since before this PR every LAN caller was refused. Loopback preflights are unchanged.

This does not reopen the amplification concern that keeps the mTLS ingress terminal: an authenticated LAN caller takes the same one-hop path a loopback client takes today, and a peer hop still lands on the mTLS ingress, which never routes onward.

Scope

In: nvpair-shared/ingressauth (new package, following the nvpair-shared/cors precedent); the gate, held once on Proxy and called from handlePlain of every facade; tests at both layers; documentation.

Out (deliberately): rate limiting / lockout (a ≥ 32-character random key makes online guessing infeasible; can be added later without changing the contract); TLS on the plaintext personality (terminate TLS in front if the network is not trusted — documented); Settings-tab / JSON-RPC wiring for the key (the key file is the operator surface for this PR — no JSON-RPC method or payload changes, so no broker relay, desktop bridge, or service-contracts changes); per-key scopes.

Validation

Ported onto develop after the two proxies were unified into nvpair-proxy (base 6abb9ec). Go on macOS (arm64):

Gate	Result
gofmt -l / go vet ./... on shared/ingressauth and nvpair-proxy; GOOS=linux and GOOS=windows vet of ingressauth	clean
go test ./... services/shared (incl. ingressauth and the owner/mode matrix in owner_unix_test.go)	ok
go test -race ./... services/nvpair-proxy (incl. ingress_auth_test.go)	ok
make check (SPDX, build-script verify, lint, typecheck, service contracts, desktop unit tests)	ok
npm run dead-code:check	ok
make test (desktop unit + every Go module)	all ok except one pre-existing failure¹

¹ nvpair-engine-manager TestUninstallTerminatesRunningInstance fails identically on pristine develop (6abb9ec, 5/5 runs) on this host; this PR does not touch that module.

Kubernetes (from review, thanks @DustinTrap): verified on single-node OpenShift 4.22 with the key from a Secret via valueFrom: every gate row held, including 403 source-not-allowed for a LAN host outside a pod-network NVPAIR_PROXY_ALLOWED_CIDRS. The key-file path failed closed under a pod fsGroup (root:<fsGroup> 0440). Accepting that shape was tried and withdrawn: the review below showed the same rule accepts root:staff, root:everyone and root:admin on macOS. The getting-started guide now directs Kubernetes to valueFrom and the pod CIDR for the allowlist, and says why a mounted Secret file is refused.

Live LAN check (before the port, against the then-separate ollama-proxy), built from this branch on a spare port (11499) on one node, probed with curl from another node on the LAN, key file 0600 holding one openssl rand -hex 32 key:

Probe	Result
startup log	WARN authenticated LAN ingress ENABLED keys=1 …
loopback, no key	routed (the standalone proxy's own 503 model inventory unavailable, since it knows no nodes; the gate is what is under test)
LAN, no key	401, Www-Authenticate: Bearer realm="nvpair-proxy", Access-Control-Allow-Origin: *, body {"code":"unauthorized",…}; log key_fp=none
LAN, wrong key	401; log key_fp=108f5470
LAN, right key as Authorization: Bearer	routed; log key_fp=7cf2fd42 = sha256(key)[:8]
LAN, right key as X-Api-Key	routed
chmod 644 key file	next LAN request → 403 loopback-only; log ERROR … permissions 0644 allow other users to read it; chmod 600, then INFO … disabled
chmod 600 again	next LAN request routed, no restart; log WARN … ENABLED
NVPAIR_PROXY_ALLOWED_CIDRS=10.99.0.0/24, LAN caller outside it, right key	403 source-not-allowed; log key_fp=none (key never examined)
key text anywhere in the proxy log	0 occurrences

Independent review. The diff was reviewed by a panel of twelve independently trained models (Anthropic, OpenAI, Google, DeepSeek, Mistral, Z.AI, MiniMax, NVIDIA Nemotron, poolside), one at a time, each blind to the others. No lane found a bypass. Adopted into the final commit: the key file is opened and checked through the opened handle and re-read by content hash (closes a stat/open TOCTOU and a same-size, same-mtime rewrite that a stat stamp cannot see); a single Authorize call reports enablement and judges the request from one view (no window between "is the gate on" and "is this key good"); the documentation no longer suggests a same-host TLS-terminating proxy, which would present every external client as a loopback caller; RFC 6750 error="invalid_token" on a rejected key; either presented header may carry the key; an sshd-style owner check on the key file; the key alphabet restricted to b64token characters and bounded at 512 bytes (a presented credential longer than that is not hashed); the CIDR allowlist applied before the preflight; a 1 s re-check floor so a node that never opted in does not open() the key file per LAN request; key rotation logged even when the key count is unchanged; successful LAN authentications logged at Info; fuzz targets and a concurrent-rotation test; and SECURITY.md statements about cluster-wide reach of a key, multi-user hosts, same-host reverse proxies, same-user processes, and Windows ACLs.

Review of the port (2026-09-22). The ported branch (f61aa4b) went to eleven independently trained models, one at a time, each blind to the others; four ran with tool access against a disposable worktree and wrote probe tests. They found two real problems, both since fixed in e65ea8a:

• The keyless-preflight exemption let a LAN caller with no key reach handleHTTP, which reads the whole body before recognizing a preflight (a 256 MiB probe was read in full), fans the OPTIONS out to every cluster candidate over this node's mTLS identity, and with one candidate relays the engine's raw reply for any path. Proved independently by three lanes. Fixed by removing the exemption.
• The group-read allowance accepted any root-owned key file whose group the proxy belongs to, including shared login groups; ten of eleven lanes flagged it and three proved it on macOS. Fixed by reverting to owner-only.

A verification round on e65ea8a (six lanes, four with tools) found both closed and no regression to loopback CORS, keyed requests, the out-of-CIDR 403 ordering, the mTLS path or the Windows build; its remaining findings (OpenAPI credential scoped to the proxy operations, a keyed-preflight test, wording) are in a76b22b. Not adopted, as a product decision for maintainers rather than a fix: answering LAN preflights locally from an operator-configured origin allowlist, which would restore browser access without reopening the first problem. Pre-existing on develop and left alone: Go's default OPTIONS * handler answers before handlePlain (no outbound request, body capped at 4 KB), and the OpenAPI Error schema says message where the ingress writes error.

Risk

• Security: widens the proxy's exposure only when an operator configures a key; nothing changes otherwise. The plaintext personality remains plaintext when enabled — documented prominently in SECURITY.md, the getting-started guide, and the startup WARN. A holder of a key can do everything a local application can, including routing to peers; documented.
• Compatibility: additive. No JSON-RPC, payload, port, or default behavior changes. The desktop/ tree is untouched.
• Pre-existing, out of scope, named so it is not mistaken for new: the loopback exemption is address-based, so a page that DNS-rebinds to 127.0.0.1 in a LAN user's browser reaches the proxy as a loopback caller today, before and after this change. fix(proxy): cap request bodies and gate browser origins #108's deny-by-default NVPAIR_PROXY_ALLOWED_ORIGINS allowlist is the fix path for that; this PR does not touch it.
• Overlap with fix(proxy): cap request bodies and gate browser origins #108 (maintainers): fix(proxy): cap request bodies and gate browser origins #108 also edits services/nvpair-proxy/ingress.go and proxy.go (a 32 MiB request-body cap and an Origin allowlist ahead of forwarding). This PR's change is a self-contained block at the top of handlePlain for non-loopback callers only, so a rebase in either order should be mechanical. The two compose: fix(proxy): cap request bodies and gate browser origins #108 bounds and origin-gates what an admitted caller can send, and this gate decides whether a non-loopback caller is admitted at all. Whichever lands second, I will rebase this one.

Release intent

Changelog title

Opt-in authenticated LAN access to the compatibility endpoints

Changelog body

The Ollama- and OpenAI-compatible endpoints can now accept plaintext requests from other machines when the operator configures an API key (key file or NVPAIR_PROXY_API_KEYS), optionally restricted by NVPAIR_PROXY_ALLOWED_CIDRS. With no key configured, behavior is unchanged: non-loopback plaintext is refused.

Bumps

• services: minor
• nvpair-cluster-manager: none
• nvpair-engine-manager: none
• nvpair-errors: none
• nvpair-job-scheduler: none
• nvpair-manual-nodes: none
• nvpair-node-info: none
• nvpair-node-scanner: none
• nvpair-node-settings: none
• nvpair-proxy: minor
• nvpair-tui: none
• nvpair-ui-broker: none
• nvpair-workload-manager: none

nvpair-proxy is MINOR: new, additive, HTTP-visible behavior. nvpair-shared/ingressauth is compiled only into nvpair-proxy.

AI disclosure

This change was written with AI assistance and reviewed by a panel of AI models, under the direction of the human author, who tested it on his own hardware and takes responsibility for it.

• Authored: Claude (Anthropic, Fable 5.1) running in Claude Code, operated by Aaron K. Clark (CryptoJones), who set the design decisions (opt-in via env/key file, loopback exempt, CIDR allowlist, full-router routing) and ran the live LAN checks on his cluster.
• Quality review: the diff and design were put to twelve independently trained models, one at a time, each blind to the others' answers; every finding was verified against the code before being acted on, and the adopted ones are listed under Independent review above.

Reviewer	Model
Anthropic	Claude Opus (opus), Claude Sonnet (sonnet), Claude Haiku (haiku), Claude Fable 5.1 (fable) — via the claude CLI; the Opus lane also ran probes against the working tree
OpenAI	openai/gpt-oss-120b
Google	gemini-3.6-flash-medium (via agy)
DeepSeek	deepseek/deepseek-v4-flash
Mistral	mistralai/mistral-large-2512
Z.AI	z-ai/glm-5.3-flash
MiniMax	minimax/minimax-m2.7
NVIDIA	nvidia/nemotron-3-super-120b-a12b
poolside	poolside/laguna-s-2.1

No model found a bypass of the key check.

Review of the port (2026-09-22), same method, run by Claude (Anthropic, Opus 5.5) in Claude Code:

Reviewer	Model	Round
Anthropic	Claude Fable (fable), Claude Opus (opus) — via the claude CLI, with tools	port + verification
OpenAI	via the codex CLI, with tools (configured gpt-5.6-sol; the lane self-reported openai/gpt-oss-120b)	port + verification
Google	gemini-3.1-pro-high (via agy, with tools)	port + verification
xAI	x-ai/grok-4.6	port + verification
NVIDIA	nvidia/nemotron-3-ultra-550b-a55b, nvidia/nemotron-3-super-120b-a12b	port; super also verification
DeepSeek	deepseek/deepseek-v4-flash	port
Z.AI	z-ai/glm-5.3-flash	port
Moonshot	moonshotai/kimi-k2.6	port
Nous	nousresearch/hermes-4-405b	port
Amazon	amazon/nova-pro-v1	port

Two configured lanes did not answer and are not counted: a local Qwen lane (host unreachable) and Mistral (model no longer served). Transcripts of every lane's answer are retained by the author and can be shared with maintainers on request.

Checklist

• I have read the Contributing Guidelines.
• Every commit is signed off (git commit -s), certifying the Developer Certificate of Origin.
• New or existing tests cover the change.
• Relevant documentation is updated.
• I checked the diff, changed filenames, and commit messages for credentials, private data, internal URLs, internal issue identifiers, and generated artifacts.
• I recorded the validation commands and results above.
• I declared the affected component bumps in the release-intent block, and described user-visible changes above so they reach the release notes.

---

Proudly Made in Nebraska. Go Big Red! 🌽 https://xkcd.com/2347/

🤖 Generated with Claude Code

https://claude.ai/code/session_01BwxtwuoRxP75PdR6NAmMS3
https://claude.ai/code/session_01BSXRLJk5knQd4rNnn7XH1S

[DustinTrap]

Tried it on the OpenShift setup from #31/#32, as asked on #28. It does what the description says, and it is the better answer for a Kubernetes workload than the relay. Two findings for Kubernetes users, one of which is a fail-closed path that will catch people.

Environment. ollama-proxy 0.27.0 and lmstudio-proxy 0.17.0 built from a4ca630 (linux/amd64, Go 1.26, CGO_ENABLED=0), every other service the 0.1.1 release, running as a hostNetwork pod on single-node OpenShift 4.22 (OVN-Kubernetes) under an arbitrary UID, paired with a 0.1.1 Linux node holding the GPU. Callers: a pod on the cluster network, and a macOS host on the LAN. Key: 64 hex characters from openssl rand, held in a Kubernetes Secret. NVPAIR_PROXY_ALLOWED_CIDRS=10.128.0.0/14 (the cluster's pod network). The env vars reach the proxies unchanged: nvpair-tui, the broker, and the proxy launches all inherit the environment.

Inline key (NVPAIR_PROXY_API_KEYS from the Secret via valueFrom), callers hitting the node's own port 1234:

Caller	Request	Result
pod, no key	GET /v1/models	401, Www-Authenticate: Bearer realm="nvpair-proxy", {"code":"unauthorized",…}
pod, right key, Authorization: Bearer	GET /v1/models	200
pod, right key, X-Api-Key	GET /v1/models	200
pod, wrong key	GET /v1/models	401, Www-Authenticate: … error="invalid_token"
pod, right key	GET :11434/api/tags (Ollama proxy)	200
pod, right key	POST /v1/chat/completions, 27B model	200, routed to the GPU node over cluster mTLS, "pong", usage reported
LAN host, right key	GET /v1/models	403 {"code":"source-not-allowed","error":"the caller's address is outside NVPAIR_PROXY_ALLOWED_CIDRS"}; log code=source-not-allowed, key never examined
LAN host, no key	GET /v1/models	403 source-not-allowed
loopback (the relay from #31, no key)	GET /v1/models	200, unchanged

Finding 1: the key-file path fails closed on Kubernetes, by design, and the reason is invisible to most operators. A Secret mounted as a file under a pod fsGroup (which every OpenShift SCC sets, and many vanilla clusters do) arrives as -r--r----- root:<fsGroup>, 0440. ownedByProcessUser accepts uid 0, but the perm check rejects the group bit, so the gate stays closed: with only NVPAIR_PROXY_API_KEYS_FILE set and the mount at defaultMode: 0400, a LAN caller with the right key still got 403 loopback-only. Two possible answers, either is fine: accept group-readable when the file's gid is the process's own primary or supplementary gid (that is exactly the fsGroup contract, and the group has nobody else in it), or say in the docs that on Kubernetes the key goes in through NVPAIR_PROXY_API_KEYS from a Secret via valueFrom, and the file path is for hosts. I would lean to the docs line plus the gid check.

Finding 2, a good one: a bad key file does not poison inline keys. With the 0440 file still configured and the inline var added, the gate enabled and every row above held. So "contributes no keys" is per source, not per process; worth one sentence in the README since the description's "the LAN stays closed" reads as global.

Source address on OVN-Kubernetes, for the ALLOWED_CIDRS advice: a pod-network caller reaching a host-network pod on the same node is seen with its pod IP (10.128.0.x), not the node's management address, so the pod CIDR is the right allowlist entry.

Not captured: the startup WARN … ENABLED and the once-per-problem key-file error. The headless TUI's Logs tab had scrolled past them by the time I looked, and the proxies do not write a file; the behavior above is the evidence. If you want the lines, a --log-level debug run is easy to repeat.

For #31: once this merges, the manifests there can point the Service at port 1234 with the key from a Secret and drop the relay entirely, which also removes the trust-boundary paragraph I had to correct today.

[Noah-Tervalon-Nvidia]

This is a great idea, thank you for bringing it up. We're having some discussions internally about the security implications and how we want to navigate that and will get back to you on this.

[Noah-Tervalon-Nvidia]

Heads up — develop has just taken a change that merges the Ollama and LM Studio
proxies into a single nvpair-proxy process, hosting one facade per enabled
engine. services/ollama-proxy/ and services/lmstudio-proxy/ are gone.

This PR edits files in those directories, so it will need updating before it can
merge. The good news is that the client-facing ollama-proxy: and
lmstudio-proxy: namespaces are unchanged — only the process behind them moved —
so the change should port across to services/nvpair-proxy/ fairly directly.

Apologies for the churn, and thanks for the contribution. Happy to help work out
where it lands if it isn't obvious.

[CryptoJones]

Heads up — develop has just taken a change that merges the Ollama and LM Studio proxies into a single nvpair-proxy process, hosting one facade per enabled engine. services/ollama-proxy/ and services/lmstudio-proxy/ are gone.

This PR edits files in those directories, so it will need updating before it can merge. The good news is that the client-facing ollama-proxy: and lmstudio-proxy: namespaces are unchanged — only the process behind them moved — so the change should port across to services/nvpair-proxy/ fairly directly.

Apologies for the churn, and thanks for the contribution. Happy to help work out where it lands if it isn't obvious.

No apologies needed. Such is the life of a SWE. =)

Please give me some time to implement and test.

CJ

[CryptoJones]

@DustinTrap thank you for the OpenShift run. It was exactly the evidence this needed. The branch is now ported onto develop (the unified nvpair-proxy) and is at a76b22b. Could you repeat the run against it when you have time?

What changed since you tested, and what to expect:

• Finding 1 (key file under fsGroup). I first took your gid-check suggestion. A cross-model review then showed the same rule also accepts root:staff, root:everyone and root:admin on macOS, and docker or users on Linux, because the proxy can't tell a pod fsGroup from a shared login group. So group read stays refused. A Secret mounted as a file will still fail closed with 403 loopback-only, now with a log line that names group access and points at NVPAIR_PROXY_API_KEYS. The getting-started guide now says to supply the key through valueFrom.secretKeyRef on Kubernetes, which is the path you already verified works.
• Finding 2 (per-source failure). This is documented now: a refused key file leaves inline keys working. The reverse is also documented: a malformed inline key or CIDR entry turns the gate off entirely.
• Your OVN-Kubernetes source-address note is in the guide: use the pod CIDR as the allowlist.
• New and worth checking: a keyless OPTIONS preflight from another machine is now refused 401 like any other keyless request. Before, it went to the engine's CORS policy, which let an unauthenticated caller push an unbounded body into memory and fan requests out to cluster peers. Loopback preflights are unchanged.

The rows I'd most like re-confirmed on your setup:

1. The pod, with the key via valueFrom: Bearer, X-Api-Key, and a chat completion routed to the GPU node.
2. The pod with no key, and with a wrong key: 401.
3. The LAN host outside NVPAIR_PROXY_ALLOWED_CIDRS: 403 source-not-allowed.
4. Loopback (your relay): 200, unchanged.
5. The mounted-file path at 0440: expect 403 loopback-only, plus the new log line if you can catch it with --log-level debug.