H‑Net++ : Hierarchical Dynamic Chunking for Tokenizer‑Free Language Modelling in Morphologically‑Rich Languages
Mehrdad Zakershahrak, Samira Ghodratnama · 2025
Read H‑Net++ : Hierarchical Dynamic Chunking for Tokenizer‑Free Language Modelling in Morphologically‑Rich Languages by Mehrdad Zakershahrak, Samira Ghodratnama (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-05628.txt.
Description
Mehrdad Zakershahrak and Samira Ghodratnama present H-Net++, a language model that works on raw bytes and learns its own chunking, with no tokenizer. Their 2025 paper targets languages rich in word building. In such languages words span many bytes, and fixed tokenizers break down. Persian is the test case, with its own script quirks handled in a special way.
A small transformer mixes context across chunks, and a two level prior keeps documents consistent. Training grows in stages of length. On a 1.4 billion token Persian corpus the model won on several fronts. It beat a BPE based rival on compression, gained on a language benchmark, and proved far more robust to corruption of a Persian joining character. Its learned chunks matched true word parts well.
Catalog record
| CREATOR | Mehrdad Zakershahrak, Samira Ghodratnama |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-05628