Skip to content

test(core): add unit tests for auth, instructions, tool callbacks, live runs, and usage metadata - #1105

Open
ScottMansfield wants to merge 1 commit into
foundry/671469f3-9119-4e23-83a5-32be6b655726from
foundry/45509488-c2b3-4c92-8dcf-e9542b114287
Open

ScottMansfield wants to merge 1 commit into
foundry/671469f3-9119-4e23-83a5-32be6b655726from
foundry/45509488-c2b3-4c92-8dcf-e9542b114287

Conversation

@ScottMansfield

Copy link
Copy Markdown
Member

Summary

This change ports the remaining unit tests from adk-python v0.5.0..v1.0.0 across auth, flows, and models to adk-js. Upstream adk-js already implements the runtime behavior from this range. This pull request adds 20 unit test cases covering generateAuthEvent preprocessing, instruction provider state-injection bypass, tool callback chains, SequentialAgent.runLive, and streaming usageMetadata.

Files, insertions, and deletions

5 files changed, 779 insertions(+), 2 deletions(-).

File Insertions Deletions Notes
core/test/auth/auth_preprocessor_test.ts 93 0 Tests AUTH_PREPROCESSOR with generateAuthEvent output.
core/test/agents/processors/instructions_llm_request_processor_test.ts 320 0 Ports test_instructions.py state-injection and provider tests.
core/test/agents/functions_test.ts 92 2 Lines 2423-2424 widen invokeToolWithCallbacks to BeforeToolCallback and AfterToolCallback.
core/test/agents/sequential_agent_test.ts 181 0 Tests SequentialAgent.runLive tool injection and execution order.
core/test/models/google_llm_test.ts 93 0 Tests streaming usageMetadata preservation on Gemini.

Ported behavior

  • fe5a5494e: AUTH_PREPROCESSOR processes adk_request_credential events produced by generateAuthEvent with snake_case argument keys.
  • 9e9dfa747 and 4c4cfb74a: INSTRUCTIONS_LLM_REQUEST_PROCESSOR injects session state into string instructions and skips state injection for sync and async InstructionProvider functions.
  • 2cbbf8813: handleFunctionCallsAsync executes beforeToolCallback and afterToolCallback chains in order and short-circuits when a callback returns a response.
  • 4188673b0: SequentialAgent.runLive registers task_completed and appends completion instructions on LlmAgent sub-agents before running them sequentially.
  • 509db3f9f: Gemini.generateContentAsync preserves usageMetadata on flushed and final aggregated streaming LlmResponse objects.

Interface discrepancies

Member adk-python adk-js Why
generateAuthEvent args Emits camelCase via model_dump(by_alias=True) Emits function_call_id and auth_config; AUTH_PREPROCESSOR accepts both casings Existing adk-js tests pin snake_case keys on generateAuthEvent.
BuiltInCodeExecutionTool Removed in v1.0.0 (5115474f2) Kept in core/src/tools/built_in_code_execution_tool.ts alongside BuiltInCodeExecutor Strictly additive rule preserves existing exports.
GeminiCodeExecutor Added as deprecated alias in 993f9971b Not added; adk-js provides BuiltInCodeExecutor adk-python later removed GeminiCodeExecutor on main.
Live audio transcription SingleFlow runs client-side AudioTranscriber on transfer (39f78dc28) Runner.runLive enables native model-side inputAudioTranscription and outputAudioTranscription AudioTranscriber in adk-js requires an explicit SpeechClientLike instance.
AnthropicLlm / LiteLlm Streaming usage_metadata updated (509db3f9f) Omitted from core adk-js core does not bundle Anthropic or LiteLLM model wrappers.

Reference test accounting

Reference file (adk-python v1.0.0) Reference tests in range Ported in adk-js Status
tests/unittests/flows/llm_flows/test_instructions.py 7 7 + 1 global provider state test Ported in instructions_llm_request_processor_test.ts.
tests/unittests/flows/llm_flows/test_async_tool_callbacks.py 8 (2 single + 6 parameterized) 8 (2 existing + 6 added) Ported in functions_test.ts.
tests/unittests/models/test_google_llm.py 1 streaming test 2 streaming usageMetadata tests Ported in google_llm_test.ts.
tests/unittests/models/test_litellm.py 1 streaming test 0 Not portable: LiteLlm is a Python-only wrapper not in adk-js.
tests/unittests/flows/llm_flows/test_functions.py 1 (test_WorkFlowAgent) Covered by 3 runLive tests in sequential_agent_test.ts Ported for SequentialAgent.runLive.
tests/unittests/artifacts/test_artifact_service.py 3 Already in in_memory_artifact_service_test.ts and gcs_artifact_service_test.ts Covered upstream.
tests/unittests/events/test_event.py 9 Already in event_test.ts Covered upstream.
tests/unittests/tools/test_langchain_tool.py 2 Already in langchain_tool_test.ts Covered upstream.

Mutation testing confirmed the new tests fail when production logic is broken (12 failed across all 5 test files when mutating auth_preprocessor.ts, instructions_llm_request_processor.ts, functions.ts, sequential_agent.ts, and streaming_utils.ts).

Verification

  • npx vitest run --project unit:core core/test/auth/auth_preprocessor_test.ts core/test/agents/processors/instructions_llm_request_processor_test.ts core/test/agents/functions_test.ts core/test/agents/sequential_agent_test.ts core/test/models/google_llm_test.ts: 5 test files passed, 174 tests passed.
  • npm run build: passed.
  • npm run ts:check: passed (0 errors).
  • npm run lint: passed (0 errors).
  • npm run format:check: passed.
  • npm run docs:check: passed.

What the reviewer must decide

  • Keep BuiltInCodeExecutionTool exported alongside BuiltInCodeExecutor for backward compatibility.
  • Keep AnthropicLlm and LiteLlm out of core.
  • Keep generateAuthEvent emitting function_call_id and auth_config while AUTH_PREPROCESSOR normalizes both snake_case and camelCase keys.

Manual testing

Executed the unit test suite, mutation checks, build, type checker, linter, formatter check, and TypeDoc check locally.

Checklist

  • I have formatted my code using npm run format
  • I have linted my code using npm run lint
  • I have added unit tests for my changes
  • I have updated the documentation (if applicable)
  • I have tested my changes manually

Produced by Foundry (task 45509488-c2b3-4c92-8dcf-e9542b114287). Reviewed by an automated
reviewer before upload; still needs human review.

…ve runs, and usage metadata

## Summary

This change ports the remaining unit tests from `adk-python` `v0.5.0..v1.0.0` across `auth`, `flows`, and `models` to `adk-js`. Upstream `adk-js` already implements the runtime behavior from this range. This pull request adds 20 unit test cases covering `generateAuthEvent` preprocessing, instruction provider state-injection bypass, tool callback chains, `SequentialAgent.runLive`, and streaming `usageMetadata`.

## Files, insertions, and deletions

5 files changed, 779 insertions(+), 2 deletions(-).

| File | Insertions | Deletions | Notes |
| --- | --- | --- | --- |
| `core/test/auth/auth_preprocessor_test.ts` | 93 | 0 | Tests `AUTH_PREPROCESSOR` with `generateAuthEvent` output. |
| `core/test/agents/processors/instructions_llm_request_processor_test.ts` | 320 | 0 | Ports `test_instructions.py` state-injection and provider tests. |
| `core/test/agents/functions_test.ts` | 92 | 2 | Lines 2423-2424 widen `invokeToolWithCallbacks` to `BeforeToolCallback` and `AfterToolCallback`. |
| `core/test/agents/sequential_agent_test.ts` | 181 | 0 | Tests `SequentialAgent.runLive` tool injection and execution order. |
| `core/test/models/google_llm_test.ts` | 93 | 0 | Tests streaming `usageMetadata` preservation on `Gemini`. |

## Ported behavior

- `fe5a5494e`: `AUTH_PREPROCESSOR` processes `adk_request_credential` events produced by `generateAuthEvent` with `snake_case` argument keys.
- `9e9dfa747` and `4c4cfb74a`: `INSTRUCTIONS_LLM_REQUEST_PROCESSOR` injects session state into string instructions and skips state injection for sync and async `InstructionProvider` functions.
- `2cbbf8813`: `handleFunctionCallsAsync` executes `beforeToolCallback` and `afterToolCallback` chains in order and short-circuits when a callback returns a response.
- `4188673b0`: `SequentialAgent.runLive` registers `task_completed` and appends completion instructions on `LlmAgent` sub-agents before running them sequentially.
- `509db3f9f`: `Gemini.generateContentAsync` preserves `usageMetadata` on flushed and final aggregated streaming `LlmResponse` objects.

## Interface discrepancies

| Member | adk-python | adk-js | Why |
| --- | --- | --- | --- |
| `generateAuthEvent` args | Emits `camelCase` via `model_dump(by_alias=True)` | Emits `function_call_id` and `auth_config`; `AUTH_PREPROCESSOR` accepts both casings | Existing `adk-js` tests pin `snake_case` keys on `generateAuthEvent`. |
| `BuiltInCodeExecutionTool` | Removed in `v1.0.0` (`5115474f2`) | Kept in `core/src/tools/built_in_code_execution_tool.ts` alongside `BuiltInCodeExecutor` | Strictly additive rule preserves existing exports. |
| `GeminiCodeExecutor` | Added as deprecated alias in `993f9971b` | Not added; `adk-js` provides `BuiltInCodeExecutor` | `adk-python` later removed `GeminiCodeExecutor` on `main`. |
| Live audio transcription | `SingleFlow` runs client-side `AudioTranscriber` on transfer (`39f78dc28`) | `Runner.runLive` enables native model-side `inputAudioTranscription` and `outputAudioTranscription` | `AudioTranscriber` in `adk-js` requires an explicit `SpeechClientLike` instance. |
| `AnthropicLlm` / `LiteLlm` | Streaming `usage_metadata` updated (`509db3f9f`) | Omitted from `core` | `adk-js` `core` does not bundle Anthropic or LiteLLM model wrappers. |

## Reference test accounting

| Reference file (`adk-python` `v1.0.0`) | Reference tests in range | Ported in `adk-js` | Status |
| --- | --- | --- | --- |
| `tests/unittests/flows/llm_flows/test_instructions.py` | 7 | 7 + 1 global provider state test | Ported in `instructions_llm_request_processor_test.ts`. |
| `tests/unittests/flows/llm_flows/test_async_tool_callbacks.py` | 8 (2 single + 6 parameterized) | 8 (2 existing + 6 added) | Ported in `functions_test.ts`. |
| `tests/unittests/models/test_google_llm.py` | 1 streaming test | 2 streaming `usageMetadata` tests | Ported in `google_llm_test.ts`. |
| `tests/unittests/models/test_litellm.py` | 1 streaming test | 0 | Not portable: `LiteLlm` is a Python-only wrapper not in `adk-js`. |
| `tests/unittests/flows/llm_flows/test_functions.py` | 1 (`test_WorkFlowAgent`) | Covered by 3 `runLive` tests in `sequential_agent_test.ts` | Ported for `SequentialAgent.runLive`. |
| `tests/unittests/artifacts/test_artifact_service.py` | 3 | Already in `in_memory_artifact_service_test.ts` and `gcs_artifact_service_test.ts` | Covered upstream. |
| `tests/unittests/events/test_event.py` | 9 | Already in `event_test.ts` | Covered upstream. |
| `tests/unittests/tools/test_langchain_tool.py` | 2 | Already in `langchain_tool_test.ts` | Covered upstream. |

Mutation testing confirmed the new tests fail when production logic is broken (12 failed across all 5 test files when mutating `auth_preprocessor.ts`, `instructions_llm_request_processor.ts`, `functions.ts`, `sequential_agent.ts`, and `streaming_utils.ts`).

## Verification

- `npx vitest run --project unit:core core/test/auth/auth_preprocessor_test.ts core/test/agents/processors/instructions_llm_request_processor_test.ts core/test/agents/functions_test.ts core/test/agents/sequential_agent_test.ts core/test/models/google_llm_test.ts`: 5 test files passed, 174 tests passed.
- `npm run build`: passed.
- `npm run ts:check`: passed (0 errors).
- `npm run lint`: passed (0 errors).
- `npm run format:check`: passed.
- `npm run docs:check`: passed.

## What the reviewer must decide

- Keep `BuiltInCodeExecutionTool` exported alongside `BuiltInCodeExecutor` for backward compatibility.
- Keep `AnthropicLlm` and `LiteLlm` out of `core`.
- Keep `generateAuthEvent` emitting `function_call_id` and `auth_config` while `AUTH_PREPROCESSOR` normalizes both `snake_case` and `camelCase` keys.

## Manual testing

Executed the unit test suite, mutation checks, build, type checker, linter, formatter check, and TypeDoc check locally.

## Checklist

- [x] I have formatted my code using `npm run format`
- [x] I have linted my code using `npm run lint`
- [x] I have added unit tests for my changes
- [ ] I have updated the documentation (if applicable)
- [x] I have tested my changes manually

Produced by Foundry (task 45509488-c2b3-4c92-8dcf-e9542b114287). Reviewed by an automated
reviewer before upload; still needs human review.
@ScottMansfield ScottMansfield added foundry parity Requested by the Foundry task that opened this pull request parity-v1.0 Requested by the Foundry task that opened this pull request labels Oct 17, 2026
@ScottMansfield

Copy link
Copy Markdown
Member Author

Scored 2026-10-17 01:59 UTC at acd07af, round 1. Earlier runs are in this comment's edit history.

Foundry merge critic: APPROVE

This change adds unit tests for credential preprocessing, instruction state injection, tool callback chains, live sequential agents, and streaming usage metadata.

Rubric — the default is approve; each rule is a reason a human should read this change first.

[no ] public_api — Touches only test files under core/test/; no exported symbol or production file changes.
[no ] weakened_guarantee — Adds test cases and widens the local invokeToolWithCallbacks helper in core/test/agents/functions_test.ts to accept callback arrays.
[no ] sensitive_surface — Adds a unit test in core/test/auth/auth_preprocessor_test.ts; no production auth or credential code changes.
[no ] release_surface — Modifies no CI workflow, package manifest, lockfile, or build configuration.
[no ] test_removed — Adds 20 new test cases across five test files and leaves all existing tests and assertions intact.
[no ] inexplicable — All five modified files add unit tests for existing core runtime behavior.

Posted by Foundry's merge critic as evidence rather than a request, so nothing here is waiting on you. Advisory only: Foundry has given no approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

foundry parity Requested by the Foundry task that opened this pull request parity-v1.0 Requested by the Foundry task that opened this pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants