Skip to content

fix(models): treat a part-less candidate as an error, not a success - #1417

Open
svetanis wants to merge 1 commit into
google:mainfrom
svetanis:fix/partless-candidate
Open

fix(models): treat a part-less candidate as an error, not a success#1417
svetanis wants to merge 1 commit into
google:mainfrom
svetanis:fix/partless-candidate

Conversation

@svetanis

@svetanis svetanis commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

2. Or, if no issue exists, describe the change:

Problem:

LlmResponse.Builder.response(GenerateContentResponse) decides success by testing whether the
candidate's content object is present, never whether it holds anything. A candidate cut off
mid-generation arrives with content present and part-less ("content": {} on the wire), so
candidate.content().isPresent() sends it down the success branch: errorCode and errorMessage
stay empty, and finishMessage is discarded. The caller receives an empty answer that reads exactly
like a model with nothing to say.

errorCode is documented as "Error code if the response is an error", yet the result carries
finishReason=MAX_TOKENS with errorCode empty.

Solution:

One predicate. Decide on whether the candidate produced parts, and treat a STOP with nothing to
say as a legitimate empty turn:

Candidate candidate = candidatesOpt.get().get(0);
this.finishReason(candidate.finishReason().orElse(null));
boolean hasParts =
    candidate.content().flatMap(Content::parts).map(parts -> !parts.isEmpty()).orElse(false);
boolean stopped =
    candidate.finishReason().map(reason -> reason.knownEnum() == FinishReason.Known.STOP).orElse(false);
if (hasParts || stopped) {
  this.content(candidate.content().orElse(null));
  this.groundingMetadata(candidate.groundingMetadata().orElse(null));
} else {
  candidate.finishReason().ifPresent(this::errorCode);
  candidate.finishMessage().ifPresent(this::errorMessage);
}

Two rows of behavior change, both in this branch:

Candidate Before After
content present, no parts, non-STOP reason success, empty content, reason dropped errorCode + errorMessage from the candidate
no content, finishReason=STOP errorCode=STOP — a normal turn marked as an error plain success

Everything else is untouched: parts present is still a success whatever the reason, a part-less
STOP still succeeds, a blocked prompt still becomes errorCode from blockReason, and the
no-candidates path is unchanged.

File Change
core/src/main/java/…/models/LlmResponse.java the predicate above
core/src/test/java/…/models/LlmResponseTest.java 8 tests

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Eight in LlmResponseTest. The four marked as failing were confirmed failing on unmodified main
before the change, by reverting LlmResponse.java alone and re-running:

Test Candidate Asserts On main
testCreate_partlessCandidateOutOfTokens_isReportedAsError part-less, MAX_TOKENS errorCode from the finish reason, no content ❌ fails
testCreate_partlessCandidateBlocked_reportsFinishMessageAsError part-less, SAFETY errorCode + errorMessage from finishMessage ❌ fails
testCreate_candidateWithEmptyPartsList_isReportedAsError empty parts list, MAX_TOKENS an empty list is treated the same as no list ❌ fails
testCreate_contentlessCandidateStoppedNormally_isReportedAsSuccess no content, STOP success — a normal turn is not an error ❌ fails
testCreate_candidateWithEmptyTextPart_isReportedAsSuccess one empty-text part, MAX_TOKENS success — an empty part is still a part ✅ passes
testCreate_partlessCandidateStoppedNormally_isReportedAsSuccess part-less, STOP success, content preserved ✅ passes
testCreate_partlessCandidateWithoutFinishReason_reportsNeitherContentNorError part-less, no finish reason no error code — nothing to raise, and no content carried ✅ passes
testCreate_truncatedCandidateWithText_isReportedAsSuccess parts with text, MAX_TOKENS success, content preserved ✅ passes

The four that already pass are the guards. They describe behavior this change must leave alone, so
they only break if the classification is inverted rather than tightened.

Manual End-to-End (E2E) Tests:

  • A real LlmAgent on gemini-3.1-flash-lite with maxOutputTokens=24 and
    thinkingBudget=2048 returns a part-less MAX_TOKENS candidate on demand. Before the change it is
    reported as a success and the caller receives an empty answer; after it, the same response carries
    errorCode=MAX_TOKENS. A control arm on an ordinary budget answers normally in both runs, and a
    truncated-but-non-empty answer stays a success in both.

Checklist

  • I have read the CONTRIBUTING.md document.
  • My pull request contains a single commit.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

@hemasekhar-p hemasekhar-p self-assigned this Aug 10, 2026
@hemasekhar-p
hemasekhar-p force-pushed the fix/partless-candidate branch from efd53fa to e0e9425 Compare August 10, 2026 10:51
@hemasekhar-p

Copy link
Copy Markdown
Contributor

Hi @svetanis, thank you for your contribution! We appreciate you taking the time to submit this pull request. Currently this PR is under review by our team, we will keep you posted if any additional information is required. thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] LlmResponse reports a candidate that produced nothing as a success

2 participants