Repository navigation
feat: v1alpha1 protobuf contract + native gRPC - #251
Draft
krisztianfekete wants to merge 9 commits into
Draft
krisztianfekete wants to merge 9 commits into
krisztianfekete wants to merge 9 commits into
Conversation
…on and TypeScript code
…valuation, plus redaction and storage fixes
…o module on release, and document the API
…tion, self tracing, watchers and Postgres repos
…ignore rule, and fail proto-check on ignored generated files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Typed API: v1alpha1 protobuf contract + native gRPC
Adds a typed contract for agentevals (
proto/agentevals/api/v1alpha1) and serves it over native gRPC on port 8002, next to the existing REST API. The server now builds session and turn views itself (tool calls paired with their results, token counts only when recorded, coded gaps), and the UI live view and annotation queue just render them.What's in it:
gen/go, taggedgen/go/vX.Y.Zon release), Python and TypeScript code is committed. CI runs buf lint, breaking checks and a drift check.runningforever.grpc-apiService port,grpcApi.{enabled,reflection,grpcReadinessProbe},EXPOSE 8002. New docs indocs/grpc-api.md.AGENTEVALS_TEST_DATABASE_URL./stream/ui-updatesnow sends protojsonWatchSessionsResponseframes: every current session, thensynced, then upserts and removals. The old per event messages (span_received,user_input,tool_call,token_update, ...) are gone.GET /api/streaming/sessionsno longer returnsinvocations. UseGetSessionorWatchSessionswith turns.--host(0.0.0.0unless set). It has no authentication. Turn it off with--api-grpc-port 0/AGENTEVALS_API_GRPC_PORT=0, orgrpcApi.enabled=falsein the chart.GET /api/runs*, cancel and the 409 body no longer return header values, secret like keys, full sink URLs or credentials inbase_url. Stored runs keep the real values for the worker.trace_idmust be 16 or 32 hex characters, otherwise you get a 422.Usagetoken fields areNonewhen not recorded (they used to be0).Turn.warningsis a list ofGap(code, reason)instead of strings.RoutingPolicyimplementations must addis_own_span.LiveManagerbroadcast and SSE client API is replaced bysubscribe()/unsubscribe()watchers.grpcio>=1.78,protobuf>=6.33.5,<7,grpcio-health-checking,grpcio-reflection,grpcio-status,opentelemetry-instrumentation-grpc.Tested
Unit and integration suites (also on Python 3.11), the live Postgres tests, a Helm install on kind with the gRPC readiness probe on and off, and live sessions from openai-agents and langchain + some manual testing.