You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
bug(web): selecting ការ in predictive text then pressing space gives suggestions starting with ការ #16587
See also #16585 (comment); I believe the underlying cause may be the same, as missing whitespace spans would likely be "invisible" in cached tokenized context, thus the start of a new word would alias to the end of the previous word.
That said, I think I did previously enhance the khmer_angkor model somewhat, so I should probably inspect a bit deeper. We'd definitely see this comment's noted affects for mon_anonta, at the least.
I'm also working on a smaller-scale fix to address cases where a context-final empty token isn't properly emitted; that part is kinda simple to detect. Interestingly, there are some edge cases in our current wordbreakers where it's missing, so I should address those too.
Activity
mcdurdin commentedon Sep 17, 2026
Screen_Recording_20260917_152939_Keyman-local.mp4
jahorton commentedon Sep 18, 2026
See also #16585 (comment); I believe the underlying cause may be the same, as missing whitespace spans would likely be "invisible" in cached tokenized context, thus the start of a new word would alias to the end of the previous word.
That said, I think I did previously enhance the
khmer_angkormodel somewhat, so I should probably inspect a bit deeper. We'd definitely see this comment's noted affects formon_anonta, at the least.jahorton commentedon Sep 18, 2026
OK, I've confirmed that the
sil.km.gccmodel would be suspect to the same issues - it uses a similar, but enhanced, .split().jahorton commentedon Oct 27, 2026
Took me a while to get back to this; see the following:
I'm also working on a smaller-scale fix to address cases where a context-final empty token isn't properly emitted; that part is kinda simple to detect. Interestingly, there are some edge cases in our current wordbreakers where it's missing, so I should address those too.