You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TensorRT support is implemented in draft PR #4: #4.
It adds a local ONNX-to-TensorRT engine builder, a standalone relation-head API, and --backend tensorrt for the demo. Dynamic predicate vocabulary, calibrated scores and spatial/semantic graph decoding are preserved.
The initial path is TensorRT 10, FP32 with TF32 disabled. On an RTX 3080 Laptop GPU, the released ViT-S/16+ passed 45 comparisons against ONNX Runtime, including empty/single-region inputs and vocabulary changes: identical valid pair sets, maximum raw-logit difference 0.000201225. All 88 tests pass locally, including native CUDA integration tests.
The PR includes installation and inference commands in deploy/README.md. Engines are built on the target GPU rather than redistributed. Jetson, the other two towers and reduced precision still need validation. This is ready for review in the draft PR; it is not merged into the release yet.
Activity
Maelic commentedon Sep 22, 2026
Hi, thank you for your question. Yes I am planning to add TensorRT support, but I can't give a timeline for now, stay tuned!
Maelic commentedon Sep 22, 2026
TensorRT support is implemented in draft PR #4: #4.
It adds a local ONNX-to-TensorRT engine builder, a standalone relation-head API, and
--backend tensorrtfor the demo. Dynamic predicate vocabulary, calibrated scores and spatial/semantic graph decoding are preserved.The initial path is TensorRT 10, FP32 with TF32 disabled. On an RTX 3080 Laptop GPU, the released ViT-S/16+ passed 45 comparisons against ONNX Runtime, including empty/single-region inputs and vocabulary changes: identical valid pair sets, maximum raw-logit difference 0.000201225. All 88 tests pass locally, including native CUDA integration tests.
The PR includes installation and inference commands in
deploy/README.md. Engines are built on the target GPU rather than redistributed. Jetson, the other two towers and reduced precision still need validation. This is ready for review in the draft PR; it is not merged into the release yet.