[mkaic]

Note

Code and PR description are LLM-written. If this PR has been marked Ready For Review, code has
been reviewed and tested in staging by Kai

Diff breakdown

Approximate added and removed lines.

Category	Lines
Genuinely new or altered logic	179
Moving or rearranging existing logic	0
Tests	367
Documentation	20
Generated file / lockfile churn	0

Description

Reuse OWLv2HF, the existing OWLv2 request adapter, and architecture owlv2 for uploaded full weights. Saved labels and prompts enable ordinary image-only detection requests. Explicit text queries and reference-example requests retain their existing paths. The model manager permits omitted classes only for OWLv2, and the model rejects requests without either explicit or saved prompts. Confidence filtering runs once before NMS for both scalar and per-class thresholds.

Companion PRs: exporter #981, SDK #569, platform #18540. No separate custom model class or model type remains.

Type of change

• New feature

How has this change been tested?

At a865ac8, T4 reruns passed for direct inference, batches, inline and saved Workflows, class filters, and the Auto Label request format. All 18 comparisons against b43d3db matched exactly in normal and tensor modes.

The few-shot matrix matched pre-PR 9d75f3b in all 37 cases: 35 successful results, including 28 nonempty cases, and two matching preexisting errors. Box and confidence differences were zero. Encoded images, typed/dictionary references, batches, cold/warm caches, confidence boundaries, and detection limits were covered. A separate authenticated manager load and the in-process v2 parser passed.

Negative-only requests still fail, and negative examples did not suppress detections in these cases. Both behaviors predate this PR. The earlier two-file staging upload preserved all 418 tensors. Hosted Auto Label jobs, annotation writeback, and customer-model accuracy remain unverified.

Any specific deployment considerations

Kai handles releases. Release #3294 prepares a new unpublished package version and synchronized runtime pins. In GitHub Actions, publish that version through Publish Inference Models Wheels to PyPi, then build both Build and Push CPU Container and Build and Push GPU Container from kai/owlv2-staging-release. Use the same new custom_tag and enable force_push.

The v2 serving stack also needs the model-manager registry change released with that stack. The regular image builds alone do not update it. Coordinate with the active rf-serving staging branch before deployment. Nothing was published or deployed by the agent.

[github-actions]

👋 Thanks for the pull request! Here is how automated Claude review works here, so you spend credits (and reviewer time) wisely.

🚧 Right now this is a draft, so automated Claude review is paused — nothing is being spent yet. Mark it Ready for review to trigger it.

Warning

💸 The Claude reviewer bills in credits, not vibes

Automated review spins up a real agent that reads real code and spends real credits on every pass. It is glad to help — but it is not a rubber duck, a linter you poke in a loop, or a substitute for reading the contributing guide. Treat it like an expensive senior reviewer whose time you booked, and show up prepared.

Draft when unsure, Ready when you mean it:

• 🌱 Not sure the PR is in good shape yet? Keep it (or set it back) as a draft — drafts pause review, so you can push and iterate without burning credits on a moving target.
• 💪 Feel strong about the contents? Mark it Ready for review and the reviewer will take a look.

However you get there, arrive prepared:

• 🧱 Bring a SOLID, thorough PR. Point your local agent at our skills/ to tune it to our guidelines first — or, if you are one of those fabled carbon-based contributors, read them yourself. A half-baked diff costs exactly the same to review as a finished one.
• ✅ Resolve every comment before you re-request review. Re-requesting with threads still open means paying twice for the same conversation.
• 🔁 Do not use CI review as an inner loop for a local agent. The reviewer is not a step-by-step debugger — do the unfolding locally and arrive with the answer, not the search.
• 🙋 If something looks off, ask a human. One question to a maintainer is cheaper and faster than three rounds of agent re-review chasing a misread.

Reviews are not free. A draft costs nothing to review; a Ready PR is a promise that it is worth reviewing.

• Prefer to skip automated review entirely? Add the skip-claude-review label.

[github-actions]

🤖 Claude review started at commit 7c83453df09b1c22f1e5ae0e1c70f17efed6b2c2.

New commits are not auto-reviewed. Add the claude-review label to request a re-review — the label is consumed when the review starts, so just add it again next time.

[github-actions]

Medium: the tensor-native workflow path still fails because this adapter has no class_names attribute.

With ENABLE_TENSOR_DATA_REPRESENTATION=True, the object detection tensor blocks (workflows/roboflow_workflows/core_steps/models/roboflow/object_detection/v{1,2,3}_tensor.py) call run_tensor_native_inference and then immediately call self._model_manager.get_class_names(model_id) (v3_tensor.py, right after the inference call). That resolves to ModelManager.get_class_names (inference/core/managers/base.py:705-706), which returns model.class_names. InferenceModelsOwlV2Adapter never sets class_names, and Model/BaseInference don't define it either.

So a workflow that runs an uploaded OWLv2 model through the OD block in tensor mode now gets past inference and then fails with AttributeError. Before this PR it failed earlier with NotImplementedError. This method looks like it was added for exactly that path, so the path should work end to end.

Suggested fix: in __init__, expose the saved labels, e.g. self.class_names = list(self._model._class_names or []) (ideally through a public property on OWLv2HF). Also add a test that drives the tensor block, or at least get_class_names, for an OWLv2 package with saved prompts.

[github-actions]

Medium: string confidence modes crash the explicit-classes open-vocabulary path.

When class_names is None (the base OWLv2 model or a package without saved prompts, called with explicit classes), threshold stays as the raw confidence. Line 359 then evaluates scores > score_thresholds. For confidence="best" or "default", that is a tensor-vs-string comparison and raises TypeError.

This is newly reachable on the v2 stack. The new OWLv2HF infer action in inference_model_manager/.../registry_defaults.py uses _K_OD, which declares confidence as type "confidence" (string modes allowed). The fallback it replaces, OpenVocabularyObjectDetectionModel, used _P_FLOAT_CONFIDENCE. So a request like {"images": ..., "classes": ["dog"], "confidence": "best"} against owlv2/owlv2-large-patch14-ensemble used to be rejected by validation and now reaches the model and fails as a server error.

Suggested fix: always resolve through ConfidenceFilter, using the prompts as names when there are no saved labels:

threshold = ConfidenceFilter(
    confidence=confidence,
    recommended_parameters=self.recommended_parameters,
    default_confidence=INFERENCE_MODELS_OWLV2_DEFAULT_CONFIDENCE,
).get_threshold(class_names or [])

(get_threshold returns a scalar when there is no per-class map.) Alternatively, keep _P_FLOAT_CONFIDENCE on the action when no saved prompts exist. Add a unit test for confidence="best" / "default" with explicit classes and no saved prompts.

[github-actions]

Claude review summary

Status: blocked. 3 Medium findings must be fixed before sign-off.

Skills: review-inference-models-pkg, review-legacy-models-registries, review-topic-legacy-new-stack-parity, review-topic-prediction-integrity, review-topic-backward-compat-and-versioning, review-topic-test-hygiene

No dedicated surface skill covers inference_model_manager/**, so that part got generic review plus topic skills only.

Findings

1. Medium: the tensor-native workflow path for uploaded OWLv2 models fails on get_class_names, because InferenceModelsOwlV2Adapter has no class_names attribute. See the inline comment on inference/models/owlv2/owlv2_inference_models.py:238.
2. Medium: confidence="best" or "default" with explicit classes and no saved prompts raises TypeError in post_process (scores > "best"). The new v2-stack OWLv2HF.infer action now accepts string confidence modes (_K_OD); the open-vocabulary fallback it replaces accepted floats only. See the inline comment on inference_models/inference_models/models/owlv2/owlv2_hf.py:333.
3. Medium: no changelog entry. This is a functional inference_models change: a new (owlv2, object-detection, hf) registry entry, saved-prompt packages, a new CorruptedModelPackageError case, per-class confidence, and a fix for batched text queries. It needs an entry under ## Unreleased in inference_models/docs/changelog.md.

Tests to add

• inference_models unit: OWLv2HF.infer(image, classes=[...], confidence="best") and confidence="default" with no saved prompts should return detections, not raise.
• Workflow execution / legacy unit: an OWLv2 adapter loaded from a package with saved prompts should expose class_names that ModelManager.get_class_names can read (tensor OD block path).

Legacy→new-stack parity: 1 legacy hunk checked (inference/models/owlv2/owlv2_inference_models.py; its twin is inference_models); mirrored: inference_model_manager/inference_model_manager/registry_defaults.py (tested in inference_model_manager/tests/unit_tests/test_registry_defaults.py); already present: none; not ported (documented): none; MISSING: none.

Minor doubts (non-blocking):

• Routing in the legacy server depends on the platform returning ("object-detection", "owlv2") for uploaded weights (companion platform PR); I can't verify that from this repo.
• The legacy generic path silently ignores class_filter for OWLv2.

Commands that informed this review: gh pr diff 3288, gh pr view 3288, the paginated gh api reads of issue comments, review comments, reviews and commits, plus reading review-source/ (OWLv2 adapter, owlv2_hf.py, ConfidenceFilter, registry_defaults.py, serializers_typed.py, object_detection/v3_tensor.py, managers/base.py).

New commits are not auto-reviewed. Add the claude-review label to request a re-review.

Reviewed at HEAD: 7c83453

[github-actions]

Release coordination

• @mkaic: add a user-facing entry under ## Unreleased in inference_models/docs/changelog.md covering saved-prompt OWLv2 packages served as object detectors, per-class confidence, and the batched text-query fix. Don't select or bump a version.
• Maintainers: inference-models needs a version bump for release. The inference_model_manager registry change has to ship with the v2 serving stack release.

Reviewed at HEAD: 7c83453

[github-actions]

🤖 Claude review started at commit 0d06c1f0f8ac207ec1e2c71062b5a3c96738e9b7.

New commits are not auto-reviewed. Add the claude-review label to request a re-review — the label is consumed when the review starts, so just add it again next time.

[github-actions]

Claude review summary

Status: all earlier findings are fixed in code. No new blocking findings in the delta (e6e99f2, 0d06c1f).

Skills: review-inference-models-pkg, review-legacy-models-registries, review-topic-legacy-new-stack-parity, review-topic-prediction-integrity, review-topic-backward-compat-and-versioning, review-topic-test-hygiene

No dedicated surface skill covers inference_model_manager/**, so that part got generic review plus topic skills only.

Earlier findings

1. Fixed: tensor-native get_class_names. InferenceModelsOwlV2Adapter.__init__ now sets self.class_names = self._model.saved_class_names. That is a public copy of the saved labels, or [] when the package has none. ModelManager.get_class_names can now read it. The test test_model_manager_reads_saved_owlv2_labels checks both cases. On the tensor path, string confidence modes from v3_tensor.py reach post_process with class_names set, so ConfidenceFilter resolves them.
2. Fixed: string confidence on the explicit-classes path. The OWLv2HF infer action in registry_defaults.py declares _P_FLOAT_CONFIDENCE again, so the v2 stack rejects "best" and "default" at validation, as it did before this PR. Without saved prompts, the legacy image-only path raises ModelInputError before it reaches post-processing.
3. Fixed: changelog. Entries are now under ## Unreleased in inference_models/docs/changelog.md.

Legacy→new-stack parity: 1 legacy hunk checked (inference/models/owlv2/owlv2_inference_models.py; its twin is inference_models). Mirrored: inference_model_manager/inference_model_manager/registry_defaults.py, tested in inference_model_manager/tests/unit_tests/test_registry_defaults.py. Already present: none. Not ported (documented): none. MISSING: none.

Minor doubts (non-blocking):

• Legacy routing for uploaded weights depends on the platform returning ("object-detection", "owlv2") (companion platform PR). I can't verify that from this repo.
• class_filter and max_candidates from the tensor OD block are silently ignored by OWLv2HF.

Commands that informed this review: gh pr diff 3288, gh pr view 3288, the paginated gh api reads of issue comments, review comments, reviews and commits, plus reading review-source/ (owlv2_inference_models.py, owlv2_hf.py, ConfidenceFilter, images_to_pillow, object_detection/v3_tensor.py, registry_defaults.py).

Reviewed at HEAD: 0d06c1f

[github-actions]

Release coordination (updated)

• @mkaic: the inference_models/docs/changelog.md entry is now present. Nothing further needed, and don't bump a version.
• Maintainers: inference-models needs a version bump for release. The inference_model_manager registry change has to ship with the v2 serving stack release.

Reviewed at HEAD: 0d06c1f

[github-actions]

😎 PR passes the vibe-check and trust-me-bro verification.

[github-actions]

Maintainer review requested.

Final approval and merge remain in GitHub.

[MTaraszewski]

Independent review of b43d3dbdcd8cc762caa5fce04a4f461ffe3398c0: three reproducible findings below.

Validation: 56 existing focused tests passed across saved-prompt behavior, reference-example coercion, embedding caches, the legacy adapter, and registry defaults. Separate behavioral probes reproduced all three findings. The reference-example and integer-confidence controls passed against the OWLv2 implementation from base revision 9d75f3baf45647a7d164a595af5fba2666b4f26c.

Verification used CPU and a tiny synthetic OWLv2 checkpoint. The class-filter and confidence probes replaced only the neural-network forward pass with known scores while preserving preprocessing, thresholding, NMS, and response handling. GPU and hosted staging were not exercised.

[MTaraszewski]

[P1] Preserve the v2 reference-example action

Adding input_color_format to pre_process makes inference_model_manager.marshalling.with_decoded_color_format inject that keyword into reference-example requests too: its capability check inspects both the selected method and pre_process. infer_with_reference_examples does not accept the keyword, so the existing v2 serving path now raises TypeError: OWLv2HF.infer_with_reference_examples() got an unexpected keyword argument 'input_color_format' before inference.

Reproduced through ModelManager.process(..., action="infer_with_reference_examples", wire_marshalling=True) with encoded PNG input. The same request succeeds with the base OWLv2 code. The probe substitutes model acquisition and uses OpenCV to supply the production decoder's BGR uint8 representation; model-manager marshalling and action dispatch are real.

Please make the reference-example path correctly handle the color-format argument, or restrict injection to entry points that support it, and add a regression test through the serving boundary.

[MTaraszewski]

[P2] Honor class_filter on image-only detection requests

The new image-only path forwards class_filter through **kwargs into OWLv2HF, whose preprocessing/postprocessing ignores it. make_response then returns every detected class, so ordinary object-detection requests violate their filtering contract.

Reproduced with a real ObjectDetectionInferenceRequest containing a base64 PNG, confidence=0.1, and class_filter=["label-b"]: adapter.infer_from_request(request) returns both label-a and label-b. Only neural-network forward was substituted with two known detections; request handling, preprocessing, NMS, and response serialization remained real.

Please apply the filter while constructing the legacy response, following the generic object-detection adapter, and cover it with a request-level regression test.

[MTaraszewski]

[P2] Preserve integer confidence overrides

The new ConfidenceFilter path silently ignores integer thresholds: _resolve_fallback_threshold recognizes only float values as numeric overrides, so an integer falls through to the recommended/default confidence.

For a package with saved labels and recommended confidence 0.5, I reproduced model.infer(image, classes=["a", "b"], confidence=1) returning detections scored 0.9 and 0.8, while the otherwise identical call with confidence=1.0 returns none. The base OWLv2 implementation treats both numeric forms consistently.

Please normalize numeric thresholds before passing them to ConfidenceFilter, or extend the helper's numeric handling, and test integer/float equivalence on the saved-label path.

[mkaic]

Comments should be addressed in a865ac8

[mkaic]

Marking as a draft as I clarify more about the intended usecase for this feature

[mkaic]

will target next week's release instead of this one.