Skip to content

[SDK] Migrate deployment log reading to logs_v4 and add pod listing - #148

Open
V2arK wants to merge 2 commits into
mainfrom
honglin/logs-v4-sdk
Open

[SDK] Migrate deployment log reading to logs_v4 and add pod listing#148
V2arK wants to merge 2 commits into
mainfrom
honglin/logs-v4-sdk

Conversation

@V2arK

@V2arK V2arK commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Problem

The SDK reads deployment logs through the CloudWatch-backed logs_v3 endpoint, which is being retired in favor of the Loki-backed logs_v4 read API (platform #4182, merged 2026-08-13). logs_v4 is per-pod and cursor-less: there is no server page token, pagination is driven by an exclusive epoch-millisecond timestamp boundary minted from the events themselves, and fetch-newer requests re-deliver a ~15s look-behind window that consumers must deduplicate by event id. The SDK also had no way to discover which pods a revision has logged from.

Change

  • Add get_deployment_pods(deployment_id, revision_number) -> List[str], exposing GET /deployments/pods/{deployment_id}/{revision_number} (includes terminated pods within log retention; empty list is normal for a fresh deployment).
  • Reimplement get_deployment_logs on logs_v4 as a stateless page fetch anchored on events from a previous call: get_deployment_logs(deployment_id, revision_number, pod, before=None, after=None, max_lines=100).
    • No anchor: the newest page (tail). Events are always oldest-first within a page.
    • before=<events you hold>: the page strictly older than the oldest of them; an empty result means the beginning of history — prepend and repeat to reassemble full history.
    • after=<events you hold>: only lines strictly newer than the newest of them; an empty result means nothing new yet — poll again to tail. The SDK drops the server's ~15s look-behind re-deliveries by matching event ids against the passed events, so callers get no duplicates while still receiving genuinely late-arriving lines. after=[] reads from the head of the log window (oldest page).
    • Passing both before and after raises ValueError.
    • The logs_v3-shaped parameters (start_time, end_time, line_count, start_from_head, token handling) are removed with the endpoint — a breaking SDK-surface change that should ride a minor version bump.
  • Add DeploymentLogSession (factory: cclient.deployment_log_session(deployment_id, revision_number, pod, events=None)), a stateful reader that anchors every request on the window it has already fetched, so pages can never overlap or leave gaps inside it: fetch_older() prepends history pages (empty = beginning reached), fetch_newer() merges only new lines and returns the delta (empty = nothing new; rare late arrivals are sorted into place by id), .events is the merged ordered window (a copy). A first call on an empty session fetches the tail page in either direction. The optional events= seed resumes a session across processes from previously fetched logs (seed is deduplicated by id and sorted; interior gaps in seeded data are undetectable in principle — log lines carry no sequence numbers). Precedent for a stateful protocol wrapper inside the SDK: centml/sdk/shell/session.py.
  • Rewrite examples/sdk/get_deployment_logs.py (pod discovery, session backfill + tail as the primary flow, the stateless anchors shown as the low-level alternative) and add a README section for the flow (mirrors the Dynamo example section from [SDK] Add Dynamo deployment support; bump platform-api-python-client to 4.23.1 #146).

requirements.txt now pins platform-api-python-client==4.25.0, the version the next platform release will publish: the generated client is versioned by the platform release tag (sync_client.yml), and the latest release v4.24.0 (2026-08-07) predates the logs_v4 merge, so the next release is the first client to carry these endpoints. Until it lands on PyPI, pip install -r requirements.txt (and this PR's CI) will fail to resolve — merge after that release is cut. Development and all testing below used the generated client from platform main (post-#4182) installed locally.

Test plan

Unit tests (TDD) cover: session first-fetch tail unification, backfill prepending, delta merge with late-arrival ordering, empty-delta stability, seed canonicalization and anchoring, .events copy semantics, per-call max_lines; and for the stateless layer: tail request shape, before anchoring on the oldest held timestamp, empty before page as begin-of-history, after anchoring on the newest held timestamp, look-behind dedup that keeps late arrivals, after=[] head read, empty after page as nothing-new, max_lines pass-through, mutual-exclusion ValueError, and a generated-client contract check (hasattr on the two new endpoint methods, mirroring the Dynamo contract test).

Live validation ran the real SDK code against the dev API (kubectl port-forward to svc/api-service, platform-team test org): created 2-replica log-pump inference_v3 deployments (ids 8704 and 8706, cluster 1036) emitting SEQ=<n> lines, verified, then deleted them.

Check Command / probe Result
Unit tests cd tests && pytest --sanity 81 passed
Format ./scripts/format.sh --diff --check 24 files unchanged
Lint ./scripts/lint.sh 10.00/10
Types ./scripts/typecheck.sh Success, 0 issues
Pods poll get_deployment_pods both pods at +20s
Tail page no anchor 99 events, ascending
History via before prepend until empty page SEQ 0..250 contiguous, unique ids, 3 pages
Small-page equivalence same walk at max_lines=7 earlier snapshot is an exact prefix; walk gap-free
Incremental after 5 polls against held events +174 lines, zero duplicates, combined SEQ contiguous (425 lines)
Head read after=[] exact oldest prefix
Mid-history resume after=history[:50] continues exactly at SEQ 50
Session backfill fetch_older() loop SEQ 0..249 gapless, unique ids
Session tail 5 fetch_newer() polls +75 lines, deltas disjoint, window stays gapless and id-ordered
Session seed resume new session seeded with saved .events catches up +25 lines, zero duplicates, contiguous
Unknown pod session fetch_newer()/fetch_older() empty lists, no error

Dev smoke total: 13/13 passed; the test deployment was deleted afterwards. The heavy requirements-dev.txt extras (torch) were not installed; pytest --sanity skips the torch-importing test files by design, everything else mirrors the CI recipes exactly.

@V2arK V2arK self-assigned this Aug 14, 2026
@V2arK
V2arK requested a review from michaelshin August 14, 2026 17:36
@V2arK
V2arK force-pushed the honglin/logs-v4-sdk branch 3 times, most recently from 065a23a to 27c180e Compare August 14, 2026 18:18
V2arK added 2 commits August 14, 2026 15:38
Signed-off-by: Honglin Cao <hocao@nvidia.com>
Signed-off-by: Honglin Cao <hocao@nvidia.com>
@V2arK
V2arK force-pushed the honglin/logs-v4-sdk branch from 27c180e to b692578 Compare August 14, 2026 19:38
@V2arK
V2arK marked this pull request as ready for review August 14, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant