Self-Training Doesn’t Flatten Language — It Restructures It: Surface Markers Amplify While Deep Syntax Dies

Ming Liu · 2026

Read Self-Training Doesn’t Flatten Language — It Restructures It: Surface Markers Amplify While Deep Syntax Dies by Ming Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20602.txt.

Description

Ming Liu challenges the usual story that self-training simply flattens a language model's output. Across eleven generations on five models, surface markers such as connectives and hedges rise, while deeper syntax such as questions, parentheticals, and passives collapses. He formalizes this as the Structural Depth Hypothesis, linking decay rate mainly to how many nested dependencies a feature needs. Pooled data give a Spearman correlation of 0.540 for that link.

Catalog record

CREATORMing Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70459
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20602

Browse the full library · About the Tower of Babel