Repository navigation
test(core): add unit tests for auth, instructions, tool callbacks, live runs, and usage metadata - #1105
Conversation
…ve runs, and usage metadata ## Summary This change ports the remaining unit tests from `adk-python` `v0.5.0..v1.0.0` across `auth`, `flows`, and `models` to `adk-js`. Upstream `adk-js` already implements the runtime behavior from this range. This pull request adds 20 unit test cases covering `generateAuthEvent` preprocessing, instruction provider state-injection bypass, tool callback chains, `SequentialAgent.runLive`, and streaming `usageMetadata`. ## Files, insertions, and deletions 5 files changed, 779 insertions(+), 2 deletions(-). | File | Insertions | Deletions | Notes | | --- | --- | --- | --- | | `core/test/auth/auth_preprocessor_test.ts` | 93 | 0 | Tests `AUTH_PREPROCESSOR` with `generateAuthEvent` output. | | `core/test/agents/processors/instructions_llm_request_processor_test.ts` | 320 | 0 | Ports `test_instructions.py` state-injection and provider tests. | | `core/test/agents/functions_test.ts` | 92 | 2 | Lines 2423-2424 widen `invokeToolWithCallbacks` to `BeforeToolCallback` and `AfterToolCallback`. | | `core/test/agents/sequential_agent_test.ts` | 181 | 0 | Tests `SequentialAgent.runLive` tool injection and execution order. | | `core/test/models/google_llm_test.ts` | 93 | 0 | Tests streaming `usageMetadata` preservation on `Gemini`. | ## Ported behavior - `fe5a5494e`: `AUTH_PREPROCESSOR` processes `adk_request_credential` events produced by `generateAuthEvent` with `snake_case` argument keys. - `9e9dfa747` and `4c4cfb74a`: `INSTRUCTIONS_LLM_REQUEST_PROCESSOR` injects session state into string instructions and skips state injection for sync and async `InstructionProvider` functions. - `2cbbf8813`: `handleFunctionCallsAsync` executes `beforeToolCallback` and `afterToolCallback` chains in order and short-circuits when a callback returns a response. - `4188673b0`: `SequentialAgent.runLive` registers `task_completed` and appends completion instructions on `LlmAgent` sub-agents before running them sequentially. - `509db3f9f`: `Gemini.generateContentAsync` preserves `usageMetadata` on flushed and final aggregated streaming `LlmResponse` objects. ## Interface discrepancies | Member | adk-python | adk-js | Why | | --- | --- | --- | --- | | `generateAuthEvent` args | Emits `camelCase` via `model_dump(by_alias=True)` | Emits `function_call_id` and `auth_config`; `AUTH_PREPROCESSOR` accepts both casings | Existing `adk-js` tests pin `snake_case` keys on `generateAuthEvent`. | | `BuiltInCodeExecutionTool` | Removed in `v1.0.0` (`5115474f2`) | Kept in `core/src/tools/built_in_code_execution_tool.ts` alongside `BuiltInCodeExecutor` | Strictly additive rule preserves existing exports. | | `GeminiCodeExecutor` | Added as deprecated alias in `993f9971b` | Not added; `adk-js` provides `BuiltInCodeExecutor` | `adk-python` later removed `GeminiCodeExecutor` on `main`. | | Live audio transcription | `SingleFlow` runs client-side `AudioTranscriber` on transfer (`39f78dc28`) | `Runner.runLive` enables native model-side `inputAudioTranscription` and `outputAudioTranscription` | `AudioTranscriber` in `adk-js` requires an explicit `SpeechClientLike` instance. | | `AnthropicLlm` / `LiteLlm` | Streaming `usage_metadata` updated (`509db3f9f`) | Omitted from `core` | `adk-js` `core` does not bundle Anthropic or LiteLLM model wrappers. | ## Reference test accounting | Reference file (`adk-python` `v1.0.0`) | Reference tests in range | Ported in `adk-js` | Status | | --- | --- | --- | --- | | `tests/unittests/flows/llm_flows/test_instructions.py` | 7 | 7 + 1 global provider state test | Ported in `instructions_llm_request_processor_test.ts`. | | `tests/unittests/flows/llm_flows/test_async_tool_callbacks.py` | 8 (2 single + 6 parameterized) | 8 (2 existing + 6 added) | Ported in `functions_test.ts`. | | `tests/unittests/models/test_google_llm.py` | 1 streaming test | 2 streaming `usageMetadata` tests | Ported in `google_llm_test.ts`. | | `tests/unittests/models/test_litellm.py` | 1 streaming test | 0 | Not portable: `LiteLlm` is a Python-only wrapper not in `adk-js`. | | `tests/unittests/flows/llm_flows/test_functions.py` | 1 (`test_WorkFlowAgent`) | Covered by 3 `runLive` tests in `sequential_agent_test.ts` | Ported for `SequentialAgent.runLive`. | | `tests/unittests/artifacts/test_artifact_service.py` | 3 | Already in `in_memory_artifact_service_test.ts` and `gcs_artifact_service_test.ts` | Covered upstream. | | `tests/unittests/events/test_event.py` | 9 | Already in `event_test.ts` | Covered upstream. | | `tests/unittests/tools/test_langchain_tool.py` | 2 | Already in `langchain_tool_test.ts` | Covered upstream. | Mutation testing confirmed the new tests fail when production logic is broken (12 failed across all 5 test files when mutating `auth_preprocessor.ts`, `instructions_llm_request_processor.ts`, `functions.ts`, `sequential_agent.ts`, and `streaming_utils.ts`). ## Verification - `npx vitest run --project unit:core core/test/auth/auth_preprocessor_test.ts core/test/agents/processors/instructions_llm_request_processor_test.ts core/test/agents/functions_test.ts core/test/agents/sequential_agent_test.ts core/test/models/google_llm_test.ts`: 5 test files passed, 174 tests passed. - `npm run build`: passed. - `npm run ts:check`: passed (0 errors). - `npm run lint`: passed (0 errors). - `npm run format:check`: passed. - `npm run docs:check`: passed. ## What the reviewer must decide - Keep `BuiltInCodeExecutionTool` exported alongside `BuiltInCodeExecutor` for backward compatibility. - Keep `AnthropicLlm` and `LiteLlm` out of `core`. - Keep `generateAuthEvent` emitting `function_call_id` and `auth_config` while `AUTH_PREPROCESSOR` normalizes both `snake_case` and `camelCase` keys. ## Manual testing Executed the unit test suite, mutation checks, build, type checker, linter, formatter check, and TypeDoc check locally. ## Checklist - [x] I have formatted my code using `npm run format` - [x] I have linted my code using `npm run lint` - [x] I have added unit tests for my changes - [ ] I have updated the documentation (if applicable) - [x] I have tested my changes manually Produced by Foundry (task 45509488-c2b3-4c92-8dcf-e9542b114287). Reviewed by an automated reviewer before upload; still needs human review.
|
Scored 2026-10-17 01:59 UTC at acd07af, round 1. Earlier runs are in this comment's edit history. Foundry merge critic: APPROVE This change adds unit tests for credential preprocessing, instruction state injection, tool callback chains, live sequential agents, and streaming usage metadata. Rubric — the default is approve; each rule is a reason a human should read this change first. [no ] public_api — Touches only test files under core/test/; no exported symbol or production file changes. Posted by Foundry's merge critic as evidence rather than a request, so nothing here is waiting on you. Advisory only: Foundry has given no approval. |
Summary
This change ports the remaining unit tests from
adk-pythonv0.5.0..v1.0.0acrossauth,flows, andmodelstoadk-js. Upstreamadk-jsalready implements the runtime behavior from this range. This pull request adds 20 unit test cases coveringgenerateAuthEventpreprocessing, instruction provider state-injection bypass, tool callback chains,SequentialAgent.runLive, and streamingusageMetadata.Files, insertions, and deletions
5 files changed, 779 insertions(+), 2 deletions(-).
core/test/auth/auth_preprocessor_test.tsAUTH_PREPROCESSORwithgenerateAuthEventoutput.core/test/agents/processors/instructions_llm_request_processor_test.tstest_instructions.pystate-injection and provider tests.core/test/agents/functions_test.tsinvokeToolWithCallbackstoBeforeToolCallbackandAfterToolCallback.core/test/agents/sequential_agent_test.tsSequentialAgent.runLivetool injection and execution order.core/test/models/google_llm_test.tsusageMetadatapreservation onGemini.Ported behavior
fe5a5494e:AUTH_PREPROCESSORprocessesadk_request_credentialevents produced bygenerateAuthEventwithsnake_caseargument keys.9e9dfa747and4c4cfb74a:INSTRUCTIONS_LLM_REQUEST_PROCESSORinjects session state into string instructions and skips state injection for sync and asyncInstructionProviderfunctions.2cbbf8813:handleFunctionCallsAsyncexecutesbeforeToolCallbackandafterToolCallbackchains in order and short-circuits when a callback returns a response.4188673b0:SequentialAgent.runLiveregisterstask_completedand appends completion instructions onLlmAgentsub-agents before running them sequentially.509db3f9f:Gemini.generateContentAsyncpreservesusageMetadataon flushed and final aggregated streamingLlmResponseobjects.Interface discrepancies
generateAuthEventargscamelCaseviamodel_dump(by_alias=True)function_call_idandauth_config;AUTH_PREPROCESSORaccepts both casingsadk-jstests pinsnake_casekeys ongenerateAuthEvent.BuiltInCodeExecutionToolv1.0.0(5115474f2)core/src/tools/built_in_code_execution_tool.tsalongsideBuiltInCodeExecutorGeminiCodeExecutor993f9971badk-jsprovidesBuiltInCodeExecutoradk-pythonlater removedGeminiCodeExecutoronmain.SingleFlowruns client-sideAudioTranscriberon transfer (39f78dc28)Runner.runLiveenables native model-sideinputAudioTranscriptionandoutputAudioTranscriptionAudioTranscriberinadk-jsrequires an explicitSpeechClientLikeinstance.AnthropicLlm/LiteLlmusage_metadataupdated (509db3f9f)coreadk-jscoredoes not bundle Anthropic or LiteLLM model wrappers.Reference test accounting
adk-pythonv1.0.0)adk-jstests/unittests/flows/llm_flows/test_instructions.pyinstructions_llm_request_processor_test.ts.tests/unittests/flows/llm_flows/test_async_tool_callbacks.pyfunctions_test.ts.tests/unittests/models/test_google_llm.pyusageMetadatatestsgoogle_llm_test.ts.tests/unittests/models/test_litellm.pyLiteLlmis a Python-only wrapper not inadk-js.tests/unittests/flows/llm_flows/test_functions.pytest_WorkFlowAgent)runLivetests insequential_agent_test.tsSequentialAgent.runLive.tests/unittests/artifacts/test_artifact_service.pyin_memory_artifact_service_test.tsandgcs_artifact_service_test.tstests/unittests/events/test_event.pyevent_test.tstests/unittests/tools/test_langchain_tool.pylangchain_tool_test.tsMutation testing confirmed the new tests fail when production logic is broken (12 failed across all 5 test files when mutating
auth_preprocessor.ts,instructions_llm_request_processor.ts,functions.ts,sequential_agent.ts, andstreaming_utils.ts).Verification
npx vitest run --project unit:core core/test/auth/auth_preprocessor_test.ts core/test/agents/processors/instructions_llm_request_processor_test.ts core/test/agents/functions_test.ts core/test/agents/sequential_agent_test.ts core/test/models/google_llm_test.ts: 5 test files passed, 174 tests passed.npm run build: passed.npm run ts:check: passed (0 errors).npm run lint: passed (0 errors).npm run format:check: passed.npm run docs:check: passed.What the reviewer must decide
BuiltInCodeExecutionToolexported alongsideBuiltInCodeExecutorfor backward compatibility.AnthropicLlmandLiteLlmout ofcore.generateAuthEventemittingfunction_call_idandauth_configwhileAUTH_PREPROCESSORnormalizes bothsnake_caseandcamelCasekeys.Manual testing
Executed the unit test suite, mutation checks, build, type checker, linter, formatter check, and TypeDoc check locally.
Checklist
npm run formatnpm run lintProduced by Foundry (task 45509488-c2b3-4c92-8dcf-e9542b114287). Reviewed by an automated
reviewer before upload; still needs human review.