Skip to content

Harden proxy request handling: 32 MiB body cap, loopback backend gate, fail-closed peer TLS - #79

Open
mkalkere wants to merge 2 commits into
NVIDIA:developfrom
mkalkere:fix/proxy-request-hardening
Open

Harden proxy request handling: 32 MiB body cap, loopback backend gate, fail-closed peer TLS#79
mkalkere wants to merge 2 commits into
NVIDIA:developfrom
mkalkere:fix/proxy-request-hardening

Conversation

@mkalkere

@mkalkere mkalkere commented Sep 14, 2026 •
edited
Loading

Copy link
Copy Markdown

Changelog title

Proxy request hardening: 32 MiB body cap, loopback-only engine addresses, fail-closed peer TLS

Changelog body

  • The proxy now rejects inference request bodies over 32 MiB with 413 before routing, closing an unauthenticated OOM vector on the loopback ingress.
  • Engine addresses supplied via set-local-backend must be loopback; anything else is refused at store time.
  • A missing peer certificate pin now fails closed (every dial errors) instead of falling back to an unpinned transport.

Bumps

  • services: patch
  • nvpair-cluster-manager: none
  • nvpair-engine-manager: none
  • nvpair-errors: none
  • nvpair-job-scheduler: none
  • nvpair-manual-nodes: none
  • nvpair-node-info: none
  • nvpair-node-scanner: none
  • nvpair-node-settings: none
  • nvpair-proxy: patch
  • nvpair-tui: none
  • nvpair-ui-broker: none
  • nvpair-workload-manager: none

Summary

Rebased onto the unified services/nvpair-proxy (the per-engine
services/ollama-proxy / services/lmstudio-proxy directories are gone).
The three hardening fixes are ported across:

  1. 32 MiB request-body cap — bufferBodyAndModel reads through
    io.LimitReader; over-cap bodies get a 413 JSON error before candidate
    selection or any engine work.
  2. Loopback-only engine addresses — the unified ingress already refuses
    non-loopback hosts in setLocalBackend() at store time; this adds
    regression coverage and documents the invariant.
  3. Fail-closed unpinned peers — a missing certificate pin returns a
    transport whose dials always fail with errPeerUnpinned instead of an
    unpinned transport.

Details in docs/proxy-request-hardening.mdx.

Test plan

  • go build ./..., go vet ./... — clean
  • gofmt -l — clean
  • go test -race -count=1 . — full nvpair-proxy suite green
  • proxy_hardening_test.go: TestBufferBodyAndModelRejectsOversizedBody,
    TestHandleHTTPRejectsOversizedBody (asserts the exact 413 JSON payload),
    TestHandleHTTPAcceptsNormalBody,
    TestSetLocalBackendRejectsNonLoopback,
    TestPeerHTTPTransportFailsClosedWithoutPin

@Noah-Tervalon-Nvidia

Noah-Tervalon-Nvidia commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

Heads up — develop has just taken a change that merges the Ollama and LM Studio
proxies into a single nvpair-proxy process, hosting one facade per enabled
engine. services/ollama-proxy/ and services/lmstudio-proxy/ are gone.

This PR edits files in those directories, so it will need updating before it can
merge. The good news is that the client-facing ollama-proxy: and
lmstudio-proxy: namespaces are unchanged — only the process behind them moved —
so the change should port across to services/nvpair-proxy/ fairly directly.

Apologies for the churn, and thanks for the contribution. Happy to help work out
where it lands if it isn't obvious.

@Noah-Tervalon-Nvidia
Noah-Tervalon-Nvidia changed the base branch from main to develop September 21, 2026 21:58
@MALLIKH
Rebase proxy request hardening onto unified nvpair-proxy
229769d 
Ports the request-hardening changes onto the unified proxy process
(services/nvpair-proxy), one facade per engine:

- 32 MiB cap on proxied request bodies (413 before routing)
- Fail-closed peer transport when no certificate pin exists
- Loopback-only local backend is now enforced at store time by
  setLocalBackend, which the unification made strictly stronger
  than the original read-time gate

No versions.json change: release automation owns version bumps.

Signed-off-by: Mallikh Kaula <mallikh@users.noreply.github.com>
@MALLIKH
Verify the 413 body as JSON and use engine terminology in the docs
7d1b4ad 
The oversized-body test now unmarshals the 413 response and asserts
the exact error payload instead of substring-matching, so the
response can never silently stop being valid JSON.

docs/proxy-request-hardening.mdx uses engine where the prose means
the engine address rather than the set-local-backend wire name, and
the sequence diagram shows the 413 payload as the JSON it is.

Signed-off-by: Mallikh Kaula <mallikh@users.noreply.github.com>
@mkalkere
mkalkere force-pushed the fix/proxy-request-hardening branch from c3244d8 to 7d1b4ad Compare September 21, 2026 22:27
@mkalkere

mkalkere commented Sep 21, 2026

Copy link
Copy Markdown
Author

Rebased onto the unified layout and pushed. The three hardening fixes are ported across to services/nvpair-proxy/:

  • 32 MiB request-body cap, answered 413 with a JSON error before routing.
  • Loopback-only engine addresses, enforced at store time in setLocalBackend(), with regression tests.
  • Fail-closed peer transport when no certificate pin exists.

Details in docs/proxy-request-hardening.mdx.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Noah-Tervalon-Nvidia Noah-Tervalon-Nvidia Awaiting requested review from Noah-Tervalon-Nvidia

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@mkalkere @Noah-Tervalon-Nvidia @MALLIKH