Show HN: UBPE – a universal BPE tokenizer, optimized and rethought

https://github.com/Scurrra/ubpe
Tokenize everything meaningful and efficient.

Comments