Skip to content

feat(core): add opt-in retries for transient LLM failures - #1413

Open
014-code wants to merge 1 commit into
google:mainfrom
014-code:feature/llm-retry-policy
Open

feat(core): add opt-in retries for transient LLM failures#1413
014-code wants to merge 1 commit into
google:mainfrom
014-code:feature/llm-retry-policy

Conversation

@014-code

@014-code 014-code commented Aug 8, 2026

Copy link
Copy Markdown

Summary

Adds an opt-in retry policy for transient LLM provider failures.

The retry boundary is placed around BaseLlm.generateContent(...) inside
BaseLlmFlow, so failed provider attempts can be retried without replaying
Runner-level side effects such as user events, session updates, or tool calls.

Changes

  • Add RetryConfig to RunConfig.
  • Keep retries disabled by default for backward compatibility.
  • Support bounded exponential backoff with optional jitter.
  • Support configurable retryable HTTP status codes.
  • Add provider exception classification through BaseLlm.
  • Count every provider attempt against RunConfig.maxLlmCalls().
  • Do not retry after a streaming response has emitted its first response.
  • Add retry logging and OpenTelemetry attributes.
  • Traverse exception cause chains safely.

Tests

Added coverage for:

  • Disabled retries by default.
  • Successful recovery after a transient failure.
  • Configured HTTP status-code retries.
  • Immediate failure for non-retryable errors.
  • Exhausted retry attempts.
  • LLM call-budget exhaustion.
  • Streaming failures after a response has been emitted.
  • RetryConfig defaults, copying, and validation.

google-java-format, git diff --check, and an independent javac
compilation of RetryConfig passed.

The full Maven test suite could not be executed in this environment because
the local Maven cache is missing a large number of project dependencies.

Fixes #1397

@google-cla

google-cla Bot commented Aug 8, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@014-code
014-code force-pushed the feature/llm-retry-policy branch from bb7af9d to 38e2ef4 Compare August 8, 2026 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Add opt-in retry policy for transient LLM failures

1 participant