[SDK] Migrate deployment log reading to logs_v4 and add pod listing - #148
Open
V2arK wants to merge 2 commits into
Open
[SDK] Migrate deployment log reading to logs_v4 and add pod listing#148V2arK wants to merge 2 commits into
V2arK wants to merge 2 commits into
Conversation
V2arK
force-pushed
the
honglin/logs-v4-sdk
branch
3 times, most recently
from
August 14, 2026 18:18
065a23a to
27c180e
Compare
Signed-off-by: Honglin Cao <hocao@nvidia.com>
Signed-off-by: Honglin Cao <hocao@nvidia.com>
V2arK
force-pushed
the
honglin/logs-v4-sdk
branch
from
August 14, 2026 19:38
27c180e to
b692578
Compare
V2arK
marked this pull request as ready for review
August 14, 2026 20:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The SDK reads deployment logs through the CloudWatch-backed
logs_v3endpoint, which is being retired in favor of the Loki-backedlogs_v4read API (platform #4182, merged 2026-08-13).logs_v4is per-pod and cursor-less: there is no server page token, pagination is driven by an exclusive epoch-millisecondtimestampboundary minted from the events themselves, and fetch-newer requests re-deliver a ~15s look-behind window that consumers must deduplicate by eventid. The SDK also had no way to discover which pods a revision has logged from.Change
get_deployment_pods(deployment_id, revision_number) -> List[str], exposingGET /deployments/pods/{deployment_id}/{revision_number}(includes terminated pods within log retention; empty list is normal for a fresh deployment).get_deployment_logsonlogs_v4as a stateless page fetch anchored on events from a previous call:get_deployment_logs(deployment_id, revision_number, pod, before=None, after=None, max_lines=100).before=<events you hold>: the page strictly older than the oldest of them; an empty result means the beginning of history — prepend and repeat to reassemble full history.after=<events you hold>: only lines strictly newer than the newest of them; an empty result means nothing new yet — poll again to tail. The SDK drops the server's ~15s look-behind re-deliveries by matching eventids against the passed events, so callers get no duplicates while still receiving genuinely late-arriving lines.after=[]reads from the head of the log window (oldest page).beforeandafterraisesValueError.logs_v3-shaped parameters (start_time,end_time,line_count,start_from_head, token handling) are removed with the endpoint — a breaking SDK-surface change that should ride a minor version bump.DeploymentLogSession(factory:cclient.deployment_log_session(deployment_id, revision_number, pod, events=None)), a stateful reader that anchors every request on the window it has already fetched, so pages can never overlap or leave gaps inside it:fetch_older()prepends history pages (empty = beginning reached),fetch_newer()merges only new lines and returns the delta (empty = nothing new; rare late arrivals are sorted into place by id),.eventsis the merged ordered window (a copy). A first call on an empty session fetches the tail page in either direction. The optionalevents=seed resumes a session across processes from previously fetched logs (seed is deduplicated by id and sorted; interior gaps in seeded data are undetectable in principle — log lines carry no sequence numbers). Precedent for a stateful protocol wrapper inside the SDK:centml/sdk/shell/session.py.examples/sdk/get_deployment_logs.py(pod discovery, session backfill + tail as the primary flow, the stateless anchors shown as the low-level alternative) and add a README section for the flow (mirrors the Dynamo example section from [SDK] Add Dynamo deployment support; bump platform-api-python-client to 4.23.1 #146).requirements.txtnow pinsplatform-api-python-client==4.25.0, the version the next platform release will publish: the generated client is versioned by the platform release tag (sync_client.yml), and the latest release v4.24.0 (2026-08-07) predates thelogs_v4merge, so the next release is the first client to carry these endpoints. Until it lands on PyPI,pip install -r requirements.txt(and this PR's CI) will fail to resolve — merge after that release is cut. Development and all testing below used the generated client from platformmain(post-#4182) installed locally.Test plan
Unit tests (TDD) cover: session first-fetch tail unification, backfill prepending, delta merge with late-arrival ordering, empty-delta stability, seed canonicalization and anchoring,
.eventscopy semantics, per-callmax_lines; and for the stateless layer: tail request shape,beforeanchoring on the oldest held timestamp, emptybeforepage as begin-of-history,afteranchoring on the newest held timestamp, look-behind dedup that keeps late arrivals,after=[]head read, emptyafterpage as nothing-new,max_linespass-through, mutual-exclusionValueError, and a generated-client contract check (hasattron the two new endpoint methods, mirroring the Dynamo contract test).Live validation ran the real SDK code against the dev API (kubectl port-forward to
svc/api-service, platform-team test org): created 2-replica log-pump inference_v3 deployments (ids 8704 and 8706, cluster 1036) emittingSEQ=<n>lines, verified, then deleted them.cd tests && pytest --sanity./scripts/format.sh --diff --check./scripts/lint.sh./scripts/typecheck.shget_deployment_podsbeforemax_lines=7afterafter=[]after=history[:50]fetch_older()loopfetch_newer()polls.eventsfetch_newer()/fetch_older()Dev smoke total: 13/13 passed; the test deployment was deleted afterwards. The heavy
requirements-dev.txtextras (torch) were not installed;pytest --sanityskips the torch-importing test files by design, everything else mirrors the CI recipes exactly.