Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models
Qing Yao, Kanishka Misra, Leonie Weissweiler, Kyle Mahowald · 2025
Read Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models by Qing Yao, Kanishka Misra, Leonie Weissweiler, Kyle Mahowald (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-20850.txt.
Introduction
Qing Yao and co-authors ask where a language model's grammar tastes arise: direct practice with a construction, or general habits absorbed from all language. Their test case is the English dative pair, the choice between 'gave Y the X' and 'gave the X to Y'. Two forces steer that choice in humans: short things first, living things first. The team raised small models on doctored text to separate the forces.
With direct examples of the bias removed, the short-first and living-first tastes in datives still showed up, carried by the wider language. Scrambling word order across whole sentences, keeping grammar links intact, shifted the models' dative choices. The conclusion is a mix: direct evidence matters, but a model's sense of a construction grows from broad patterns elsewhere in language. The authors say this blend explains how models pick up untaught preferences.
Catalog record
| CREATOR | Qing Yao, Kanishka Misra, Leonie Weissweiler, Kyle Mahowald |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-20850
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models Qing Yao, Kanishka Misra, Leonie Weissweiler, Kyle Mahowald Abstract Language models (LMs) tend to show human-like preferences on a number of syntactic phenomena, but the extent to which these are attributable to direct exposure to the phenomena or more general properties of language is unclear. We explore this with the English dative alternation (DO: gave Y the X vs. PO: gave the X to Y ), using a controlled rearing paradigm wherein we iteratively train small LMs on systematically manipulated input. We focus on properties that affect the choice of alternant: length and animacy. Both properties are directly present in datives but also reflect more global tendencies for shorter elements to precede longer ones and animates to precede inanimates. First, by manipulating and ablating datives for these biases in the input, we show that direct evidence of length and animacy matters, but easy-first preferences persist even without such evidence. Then, using LMs trained on systematically perturbed datasets to manipulate global length effects (re-linearizing sentences globally while preserving dependency structure), we find that dative preferences can emerge from indirect evidence. We conclude that LMs’ emergent syntactic preferences come from a mix of direct and indirect sources. 1 Introduction Consider the dative alternation. Roughly the same real-world event is conveyed by She gave the boy who signed up for class and was excited it (using the Double Object or DO form) and She gave it to the boy who signed up for class and was excited (the Prepositional Object or PO form). Figure 1 : The dative alternation allows for either PO or DO realization. We highlight the arguments ( theme and recipient ) that are relevant to our feature analysis. Yet most people would prefer the latter. The reason is that users of English typically show a “short-first” preference in the dative alternation. That is, if the theme is longer than the recipient they prefer the DO. If the theme is shorter, the PO. See an illustration of this in Figure 1 . Besides length, there are a host of other factors that influence the choice of alternation used, such as animacy and pronominality ( Bresnan et al. 2007 ; de Marneffe et al. 2012 ) . Human-like preferences in the dative alternation are learned well by even relatively small LMs ( Hawkins et al. 2020 ; Misra & Kim 2024 ) . But what has to be true of the training data for these preferences to emerge? Recent work has considered the role of indirect evidence in language learning, asking whether LMs learn syntactic phenomena by memorizing only the most similar instances or whether they rely on structurally similar instances in order to generalize. Evidence so far has been mixed, with some work arguing for generalization from indirect evidence. Misra & Mahowald 2024 find that a somewhat rare syntactic structure is learned, in part, from indirect evidence from more common related structures. Patil et al. 2024 find that grammatical generalizations are possible even when never encountered. Oba et al. 2024 , though, find that indirect evidence is less important than direct evidence. Anonymous 2025 finds that binomial preferences (e.g., “peanut butter and jelly” vs. “jelly and peanut butter”) in models largely depends on memorization, with little evidence of generalization to human-like preferences for novel binomial pairs. Jumelet et al. 2024 find that both direct evidence and indirect evidence matter for adjective order (e.g., “big black boxes” vs. “black big boxes”). We posit that the dative alternation is an elegant testing ground for exploring these questions, since it is linguistically well-studied and shown to be sensitive to a variety of factors that are directly observable in extant dative data, but which also depend on more general constraints in the language. One possibility (direct evidence) is that LMs prefer dative constructions that are highly similar to those observed in training. That is, if She gave it to the boy who signed up for class and was excited is attested in the training but She gave the boy who signed up for class and was excited it is not, then LMs might assign higher probability to strings that are very similar to the former. A second possibility (indirect evidence) is that the observed dative preferences are emergent from more general features of the input data. Specifically, we hypothesize that the LMs’ preference for “short-first” and “animate-first” dative utterances ( MacDonald 2013 ) might follow not from memorizing instances of such datives but from those preferences existing more generally in the data. To adjudicate between these possibilities, we use a controlled rearing paradigm ( Jumelet et al. 2021 ; Frank 2023 ; Misra & Mahowald 2024 ; Feng et al. 2024 ; Patil et al. 2024 ; Leong & Linzen 2024 ; Kallini et al. 2024 ; Xu et al. 2025 ) , whereby we train small models on carefully controlled input data and then test whether the predicted preferences emerge. We first train a default model on a roughly 88M-word corpus and test whether these preferences emerge in the dative. We then manipulate the input, balancing length and animacy related statistics in the training set to remove any order bias in the instances of the dative construction. We do this by creating a matching PO for every DO, and a matching DO for every PO. We show that dative preferences still persist—although they get weaker. So we train another model just like default except all datives are inserted in the reverse alternant, such that the direct evidence is now the opposite of what it normally would be. We find that this neutralizes the standard dative preferences. Then, we train models which do not have direct exposure to datives, and see that length preferences still emerge. The above results show that direct evidence matters but also that human-like trends emerge even without direct evidence, especially for length. 1 1 1 Code and data for the experiments are available at https://github.com/dounick/dativelm.git To ascertain whether these trends come from more global statistical patterns, we turn to manipulating global preferences directly. We do this by systematically reordering all constituents in training sentences based on their length, in order to create input corpora with systematically controlled length effects. We find that, while the usual dative length preferences emerge in the “short-first” corpus, the preferences are gradually lost as training sentences increasingly become more “long-first”. We take this as evidence that more general properties of the language non-trivially contribute to the observed dative preference in LMs . As such, this work joins a larger body of work suggesting the emergence of abstract linguistic behavior in LMs from statistical regularities in their training (see Futrell & Mahowald 2025 , for a summary) . 2 Data, Methods, and Models 2.1 Corpus Our training data is a subset of the 2023 BabyLM corpus with 100M words ( Warstadt et al. 2023 ) , which we choose for its tractable and human-scale size. We exclude the QED portion, since it includes special symbols and lacked proper sentence breaks, making dative detection difficult. This leaves our initial training set at around 88M words. 2.2 Defining and Detecting Datives Our approach crucially