fix: make visual OCR review text-first - #52
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
layoutSafeRegionnull/[]when geometry is unavailable instead of contradicting each other about omitted fieldsgray image+Return OK) with a real image-ingestion challenge: a random six-character code exists only inside the generated image and the model must read it back correctlyWhy
Real-device testing shows that local OCR works, while visual recognition has repeatedly ended with no usable text. The previous block-review path already knew the crop geometry but still required the vision model to return precise
textBox/bubbleBoxdata; a model could read the text correctly and still have the entire result discarded for missing/malformed geometry.A second issue was found during transport review: the settings-page vision test did not prove vision at all. It sent a blank/gray image and asked for
OK, so a text-only model or gateway that ignored the image could still be reported as a successful vision connection.A third contract issue existed in full-page recognition: the prompt told the model to omit some unknown geometry while strict structured output required those same keys. The contract now consistently uses
nullfor unavailable rectangle fields and empty arrays for unavailable polygons.Transport review
The app-side image payloads are protocol-specific:
image_urlinput_imageimageThe remaining provider/model uncertainty is therefore now tested with the image-only challenge rather than an HTTP-only success check.
Regression coverage
无法返回文本is rejectedlayoutSafeRegionand normalized text geometrynull/[]optional-geometry representationOK, refusal text and wrong codesValidation boundary
CI proves the app-side request/response contract, parser behavior and challenge verifier. The real external provider/model still has to pass the in-app vision connection probe, because CI has no user API key. After that succeeds, retest the same manga page. If the probe fails, its returned excerpt now distinguishes “endpoint reachable” from “model actually read the image” and should be used to correct the selected visual model/API protocol before further OCR tuning.