Self-Training Doesn’t Flatten Language — It Restructures It: Surface Markers Amplify While Deep Syntax Dies
Ming Liu · 2026
Read Self-Training Doesn’t Flatten Language — It Restructures It: Surface Markers Amplify While Deep Syntax Dies by Ming Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20602.txt.
Description
Ming Liu challenges the usual story that self-training simply flattens a language model's output. Across eleven generations on five models, surface markers such as connectives and hedges rise, while deeper syntax such as questions, parentheticals, and passives collapses. He formalizes this as the Structural Depth Hypothesis, linking decay rate mainly to how many nested dependencies a feature needs. Pooled data give a Spearman correlation of 0.540 for that link.
Catalog record
| CREATOR | Ming Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70459 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.20602