H‑Net++ : Hierarchical Dynamic Chunking for Tokenizer‑Free Language Modelling in Morphologically‑Rich Languages

Mehrdad Zakershahrak, Samira Ghodratnama · 2025

Read H‑Net++ : Hierarchical Dynamic Chunking for Tokenizer‑Free Language Modelling in Morphologically‑Rich Languages by Mehrdad Zakershahrak, Samira Ghodratnama (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-05628.txt.

Description

Mehrdad Zakershahrak and Samira Ghodratnama present H-Net++, a language model that works on raw bytes and learns its own chunking, with no tokenizer. Their 2025 paper targets languages rich in word building. In such languages words span many bytes, and fixed tokenizers break down. Persian is the test case, with its own script quirks handled in a special way.

A small transformer mixes context across chunks, and a two level prior keeps documents consistent. Training grows in stages of length. On a 1.4 billion token Persian corpus the model won on several fronts. It beat a BPE based rival on compression, gained on a language benchmark, and proved far more robust to corruption of a Persian joining character. Its learned chunks matched true word parts well.

Catalog record

CREATORMehrdad Zakershahrak, Samira Ghodratnama
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-05628

Browse the full library · About the Tower of Babel