Conversation
|
Heads up — This PR edits files in those directories, so it will need updating before it can Apologies for the churn, and thanks for the contribution. Happy to help work out |
The 120 s ResponseHeaderTimeout on the upstream transport is now overridable: --response-header-timeout flag first, then the NVPAIR_PROXY_RESPONSE_HEADER_TIMEOUT environment variable, then the 120 s default. A missing, unparseable, or non-positive value logs a warning and keeps the default, so a bad setting can never silently disable the timeout. firstBodyTimeout tracks the same resolved value because the two bound the same wait: how long an engine may take to start its work. Ported onto the unified nvpair-proxy after the engine proxy unification; the per-proxy copies are gone, so the flag, the resolver, and the test exist exactly once. Tests: TestResolveResponseHeaderTimeout (flag > env > default precedence, invalid/zero/negative fall back), and TestProxyTransportUsesConfiguredHeaderTimeout (the configured value reaches transports built after startup). Signed-off-by: Mallikh Kaula <mallikh@users.noreply.github.com>
ff0650b to
014b7b1
Compare
|
Rebased onto the unified layout and pushed. The configurable header timeout is ported across to
Details in |
Changelog title
Proxy upstream response-header timeout is now configurable
Changelog body
nvpair-proxyaccepts--response-header-timeout(or theNVPAIR_PROXY_RESPONSE_HEADER_TIMEOUTenvironment variable) to raise the 120 s default when engines queue requests or load models slowly. Invalid values log a warning and keep the default.Bumps
Summary
Rebased onto the unified
services/nvpair-proxy. The 120 sResponseHeaderTimeouton the upstream transport is now configurable:--response-header-timeoutflag first, thenNVPAIR_PROXY_RESPONSE_HEADER_TIMEOUT, then the 120 s default. A missing,unparseable, or non-positive value logs a warning and keeps the default, so
a bad setting can never silently disable the timeout.
firstBodyTimeouttracks the same resolved value, since both bound the samewait — how long an engine may take to start its work. Documented in
docs/proxy-response-header-timeout.mdx(new) andservices/nvpair-proxy/spec.md; troubleshooting entry added for the502-after-120-seconds symptom.
Test plan
go build ./...,go vet ./...— cleangofmt -l— cleango test -race -count=1 .— fullnvpair-proxysuite greenheader_timeout_test.go:TestResolveResponseHeaderTimeout(flag >env > default precedence; invalid, zero, and negative values fall back),
TestProxyTransportUsesConfiguredHeaderTimeout(the configured valuereaches transports built after startup)