Hi!
Thank you for a good implementation, but there is a missing API point.
For models like Parakeet TDT, Qwen3-ASR I use to obtain token log probabilities from the raw decoder logits. This is useful for detecting uncertain words, and UI highlighting.
Can you add an API endpoint or parameter similar to the other execution providers, such as Llama.cpp and OONX?
This is a relatively small non-BC breaking addition to the current API, cause logprobs are used internally.
I can provide a PR, if you'd like, I just did not want to overstep with an API design suggestions.
Ideally:
token_id, token_text, logprob
with probability = exp(logprob) left to the caller.
Grigori
Hi!
Thank you for a good implementation, but there is a missing API point.
For models like Parakeet TDT, Qwen3-ASR I use to obtain token log probabilities from the raw decoder logits. This is useful for detecting uncertain words, and UI highlighting.
Can you add an API endpoint or parameter similar to the other execution providers, such as Llama.cpp and OONX?
This is a relatively small non-BC breaking addition to the current API, cause logprobs are used internally.
I can provide a PR, if you'd like, I just did not want to overstep with an API design suggestions.
Ideally:
token_id, token_text, logprob
with probability = exp(logprob) left to the caller.
Grigori