Unbundle tokenizer.json from the wheel — load from converted-checkpoint dir - #129

Unbundle tokenizer.json from the wheel — load from converted-checkpoint dir#129

Andrewxu313 wants to merge 8 commits into

mainbatchgen-project/batchgen:mainfrom

lausannel/unbundle-tokenizer-json-from-the-wheel-lbatchgen-project/batchgen:lausannel/unbundle-tokenizer-json-from-the-wheel-lCopy head branch name to clipboard