From Linear Input to Hierarchical Structure: Function Words as Statistical Cues for Language Learning
Xiulin Yang, Heidi Getz, Ethan Gotlieb Wilcox · 2026
Read From Linear Input to Hierarchical Structure: Function Words as Statistical Cues for Language Learning by Xiulin Yang, Heidi Getz, Ethan Gotlieb Wilcox (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21191.txt.
Introduction
Xiulin Yang, Heidi Getz, and Ethan Gotlieb Wilcox ask what lets learners find grammar's hidden tree in a flat stream of words. Their 2026 paper studies statistical cues for language learning. Language arrives in a line, one word after another. Its structure is a hierarchy of nested phrases. The authors focus on the spread of function words. Words like the, of, and to are function words. They have long been argued to play a key role in learning.
Function words are frequent and tied to syntactic structure with care. They also align with phrase bounds. Those traits could cue a learner to where phrases start and end. The paper asks which statistical conditions support learning hierarchy from such cues. The answer bears on how children learn language so fast. It also informs how machines should be taught grammar.
Catalog record
| CREATOR | Xiulin Yang, Heidi Getz, Ethan Gotlieb Wilcox |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-21191
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Linear Input to Hierarchical Structure: Function Words as Statistical Cues for Language Learning Xiulin Yang, Heidi Getz, Ethan Gotlieb Wilcox Abstract What statistical conditions support learning hierarchical structure from linear input? In this paper, we address this question by focusing on the statistical distribution of function words. Function words have long been argued to play a crucial role in language acquisition due to their distinctive distributional properties, including high frequency, reliable association with syntactic structure, and alignment with phrase boundaries. We use cross-linguistic corpus analysis to first establish that all three properties are present across 186 studied languages. Next, we use a combination of counterfactual language modeling and ablation experiments to show that language variants preserving all three properties are more easily acquired by neural learners, with frequency and structural association contributing more strongly than boundary alignment. Follow-up probing and ablation analyses further reveal that different learning conditions lead to systematically different reliance on function words, indicating that similar performance can arise from distinct internal mechanisms. 1 1 1 Code: https://github.com/xiulinyang/function_word ; models: https://huggingface.co/collections/xiulinyang/function-words . 1 Introduction A central puzzle in language acquisition is how learners abstract grammatical structure from linear input. Extensive research on statistical learning argues that learners make use of distributional information in their input, such as transition probability ( 52 ) and prosodic grouping ( 41 ; 46 ) . Within this literature, function words, such as determiners, auxiliaries, and prepositions, have been argued to play a critical role ( 23 ; 25 , e.g.,) . Three properties of function words have been argued to facilitate acquisition: (i) high lexical frequency, (ii) reliable association with particular syntactic structures, and (iii) consistent position at phrase boundaries. Because of these properties, function words might serve as anchor points for learning, highlighting particular word sequences for analysis (e.g., the Anchoring Hypothesis; 53 ; see also 41 on boundary-aligned cues) or giving learners a small set of high-frequency items to track (e.g., the Marker Hypothesis; 23 , with related evidence from 21 ; 59 ; 40 ). This literature suggests that function words contribute in important ways to statistical learning, yet two major gaps remain. First, existing studies draw conclusions about the properties of function words primarily from English or a small set of languages ( 23 ; 34 ; 11 ; 48 , e.g.,) . If these properties play a systematic role in learning, it is important to examine whether they are robust across languages (RQ1). Second, because these properties are often examined in isolation in simplified artificial language settings, it remains unclear whether their findings can be extended to more complex natural texts (RQ2.1) and which properties most influence learning (RQ2.2). Furthermore, if these statistical properties shape learning, do they necessarily result in the same strategy for deploying function words during inference? (RQ3) Addressing this question would clarify whether there are multiple routes to successful acquisition and whether certain statistical cues are redundant. We answer RQ1 by conducting a cross-linguistic analysis using data from the Universal Dependencies (UD) project ( 13 ) , testing whether the three distributional properties attributed to function words are universal across 186 languages in our sample. To address RQ2, in line with a growing body of work that uses neural language models as tools for theory specification and evaluation ( 42 ; 35 ) , we take transformer models as domain-general and weakly-biased learners ( 56 ) and train them on counterfactual variants of natural text in which each distributional property of function words is systematically manipulated. To address RQ3, we conduct attention probing and function-word-related ablation experiments to examine how models internally represent and deploy function-word information. We confirm that a language is most learnable when all three statistical properties of function words jointly hold, but we show that these properties do not contribute equally. Specifically, they contribute to language learnability with a clear hierarchy: lexical frequency > structural association > boundary alignment, even though all three properties are cross-linguistically robust. However, frequency alone is not sufficient: we observe a Goldilocks effect, whereby function words must be frequent enough to be reliable, yet sufficiently diverse to remain informative. Finally, our probing results suggest that similar behavioral performance does not necessarily arise from the same internal mechanisms, indicating multiple routes to successful grammatical learning by neural learners. Properties Manipulation Example Lexical Frequency NoFunction dog happily chasing dog garden . MoreFunction thi dog wist happily chasing que dog ap tho garden . FiveFunction the dog will happily chasing the dog at the garden . Standard Function a dog is happily chasing another dog in the garden . Structural Dependency RandomDep by dog in happily chasing at dog by will garden . BigramDep by dog in happily chasing by dog at will garden . PhraseDependency a dog is happily chasing another dog in the garden . Phrase boundary WithinBoundary a dog happily is chasing another dog the in garden . AtBoundary a dog is happily chasing another dog in the garden . Table 1: An overview of manipulated languages 2 Background & Related Work 2.1 Defining Function Words The distinction between function and content words is foundational in linguistic theory ( 45 ; 1 ) and language development research ( 14 ) . Function words are typically characterized as a closed class with high frequency ( 41 ) , reduced prosodic prominence ( 47 ; 3 ) , systematic alignment with phrase boundaries ( 34 ; 10 ; 11 ) , and relatively light semantic content despite high grammatical utility ( 5 ) . Although these properties have been hypothesized to serve as universal cues for acquisition, their cross-linguistic robustness has not been systematically evaluated. 2.2 Function Words in Language Acquisition Although early speech production by children usually lacks function words ( 4 ) , extensive evidence suggests that even in infancy, learners perceive these elements ( 49 ; 50 ) and are sensitive to their distribution ( 26 ; 20 ; 10 ; 30 ) . Experiments with artificial languages have shown that languages are more learnable by humans when function words are (1) more frequent than content words ( 53 ) ; (2) reliable predictors of specific phrase structures ( 23 ; 21 ) ; and (3) systematically positioned at phrase boundaries ( 41 ) . 2.3 Computational Perspectives Computational research on function words has historically received less attention, generally falling into two strands. The first strand uses computational models to simulate human acquisition phenomena. Several studies focus on syntactic categorization, demonstrating that simple distributional cues (e.g., frequent frames) are sufficient to categorize words into nouns and verbs ( 39 ; 7 ; 24 ) . 38 ; 28 found that putting function words at phrase boundaries alone yields accurate categorization or word learning. More recently, 37 attempted to replicate the experiments by 53 with LLMs, but they found that LLMs showed limited sensitivity to marker frequency in small-scale artificial settings. The second strand probes the linguistic knowledge encoded in pre-trained language models. Work by 33 , 15 , and 43 has investigated whether (visual) language models retain knowledge of function words, yielding mixed results. 3 Methodology 3.1 Function Word Identification Although function words are often treated as a closed class, the