Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMs

Dongxin Guo, Jikun Wu, Siu Ming Yiu · 2026

Read Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMs by Dongxin Guo, Jikun Wu, Siu Ming Yiu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23039.txt.

Introduction

Guo, Wu, and Yiu study how language models learn what not to say, testing statistical preemption against entrenchment in one converging design. Across four experiments with 120 English verb-construction pairings, model surprisal tracks human acceptability at r equals 0.79. The patterns follow rival-form frequency, not overall verb use. Preemption sensitivity grows as a power law with model size. A fine-tuning intervention causally shifts preemption by manipulating rival-form frequencies, with reverse controls ruling out mere rate sensitivity. Neural models, they conclude, learn negative knowledge through distributional competition.

Catalog record

CREATORDongxin Guo, Jikun Wu, Siu Ming Yiu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67345
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.23039

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMs Dongxin Guo, Jikun Wu, Siu Ming Yiu Abstract How do learners acquire knowledge of what is unacceptable without negative evidence? Construction Grammar proposes statistical preemption : exposure to a conventional form (e.g., “ donated the books to the library ”) preempts structurally possible but unattested alternatives (“ *donated the library the books ”). We present a computational study that, for the first time, directly dissociates statistical preemption from the competing entrenchment hypothesis in large language models within a single converging design. Across four experiments spanning 120 English verb–construction pairings (dative, causative, locative), we show that (1) LLM surprisal patterns correlate strongly with human acceptability judgments ( r = 0.79 r=0.79 ), validated against three independent behavioral datasets; (2) these patterns are driven by competing-form frequency rather than overall verb frequency, confirmed by non-circular partial correlations; (3) preemption sensitivity scales as a power law with model size; and (4) a controlled fine-tuning intervention causally demonstrates that manipulating competing-form frequencies shifts preemption behavior in the predicted direction, with reverse-direction controls ruling out frequency-sensitivity confounds. These results provide converging evidence that neural language models acquire negative linguistic knowledge through distributional competition, the core mechanism posited by Construction Grammar. 1 Introduction How do language learners come to know what not to say? A child who hears “ She donated the books to the library ” must eventually learn that “ *She donated the library the books ” is unacceptable, despite never being told so and despite the double-object construction being perfectly productive with semantically similar verbs like give ( 56 ; 28 ) . This “retreat from overgeneralization” constitutes Baker’s Paradox ( 5 ) , one of the deepest puzzles in language acquisition. Construction Grammar offers an influential solution through statistical preemption : learners acquire negative knowledge by tracking the frequency of competing conventional forms ( 25 ; 26 ) . When a speaker repeatedly encounters donate in the prepositional dative in contexts where the double-object would be functionally equivalent, this accumulated evidence “preempts” the unattested alternative ( 23 ; 24 ) . Preemption differs from entrenchment , the proposal that hearing a verb frequently in any construction reduces willingness to use it in novel ones ( 12 ; 4 ) . Preemption requires exposure to a specific competing form with the same communicative function ( 73 ; 57 ) . We ask whether LLMs, trained purely on distributional statistics without explicit grammatical instruction, capture the distributional signature of statistical preemption paralleling that observed in human speakers of English. This question is significant because: LLMs provide a controlled test of whether distributional learning alone suffices to acquire negative knowledge ( 51 ; 77 ; 74 ) ; per-verb LLM effects can be compared item-by-item to human behavioral data using validated linking hypotheses ( 36 ) ; and controlled training interventions can provide causal evidence that preemption operates through distributional competition. Baker’s Paradox Construction Grammar motivates Preemption Entrenchment competing forms \Delta S\!\propto\! competing form frequency \Delta S\!\propto\! overall verb frequency Correlational (Exps 1–3) Causal (Exp 4) test test Human Behavioral Ground Truth validate validate Figure 1: Conceptual framework. Baker’s Paradox motivates two competing accounts from Construction Grammar, namely preemption and entrenchment, which make distinct predictions about LLM surprisal patterns. We test these with correlational evidence (Experiments 1–3) and causal evidence from controlled training interventions (Experiment 4), triangulated against human behavioral data from three independent sources. Following recent work using LLMs as instruments for testing scientific hypotheses about language ( 48 ; 20 ; 72 ; 7 ) , we design four experiments: (1) testing whether LLM surprisal distinguishes preempted from non-preempted English forms and correlates with human acceptability; (2) dissociating preemption from entrenchment using 73 ’s ( 73 ) +Competing/ - Competing design with non-circular partial correlations against human data; (3) fitting a formal scaling law across 14 models; and (4) providing converging causal evidence via fine-tuning with manipulated frequencies, replicated across five seeds with a reverse-direction control. We build on a growing body of causal work using controlled-input training in this area ( 51 ; 50 ; 77 ) ; our specific methodological contribution is the combination of preemption–entrenchment dissociation in LLMs with non-circular human-data validation ( r_{\text{partial}}=0.58 ), strong item-level LLM–human correlations ( r=0.79 ) triangulated across three behavioral datasets, a formal scaling analysis, and a reverse-direction asymmetry control, with converging evidence across dative, causative, and locative alternations in English. 2 Background 2.1 Statistical Preemption Theory Within the broader framework of Construction Grammar ( 22 ; 25 ; 13 ; 16 ) , statistical preemption was formalized by 23 as follows: a verb v is preempted from construction A when speakers have accumulated sufficient evidence that a competing construction B is conventionally used with v in the same communicative contexts. Formally: P(\text{Cx}_{B}\mid\text{context for Cx}_{A},\;v)\gg 0 (1) 26 situated this mechanism within a broader framework balancing coverage (the pressure to extend constructions productively) against competition (the inhibitory force of established alternatives). The key prediction is gradient : unacceptability varies continuously with the strength of competing evidence ( 24 ; 11 ; 6 ) . The behavioral evidence for preemption has grown substantially. 9 first demonstrated the mechanism for a-adjective production. 73 provided the crucial dissociation: for verbs with a competing form, competing-form frequency predicted unacceptability; for verbs without a competing form, this effect vanished. 65 provided the first causal evidence in humans. 57 , using five artificial-language-learning studies, found all five supported preemption while three showed null entrenchment effects. 74 demonstrated distributional learning of argument structure. Bayesian approaches ( 55 ) show that negative knowledge can arise from Bayesian inference over positive data. The preemption-entrenchment debate has been nuanced. 2 found evidence for entrenchment but not preemption, though with high collinearity. 1 showed both effects are reliable but rarely separable. 63 argued that negative evidence from corpus frequencies constrains overgeneralization. Our computational approach offers a complementary resolution: by using models for which the training distribution is fully known, we can directly compute preemption and entrenchment scores and assess their independent contributions ( 54 ; 53 ; 18 ) . 2.2 The Dative Alternation as a Test Case The English dative alternation is ideal for testing preemption because it involves many verbs distributed across the full range of alternation behavior. Many verbs alternate freely, but others are restricted: donate , explain , and whisper resist the double-object, while cost and fine resist the prepositional ( 41 ; 33 ; 28 ) . 10 modeled the alternation as a probabilistic choice governed by approximately ten factors including animacy, pronominality, and given/new status. 31 created the DAIS benchmark, comprising 50,000 forced-choice human judgments across 200 dative verbs: the gradient, item-level human data essential for testing preemption. 2.3 Surprisal as a Linking Hypothesis We use word-level surprisal, S(w_{t})=-\log_{2}P(w_{t}\mid w_{<t}) , as our linking hypothesis between LLM representations and human acceptability ( 30 ; 42 ) . This relationship is well-established across six orders of magnitude ( 62 ; 60 ) , with better models yielding better cognitive predictors ( 27 ; 49 ) . 36 further demonstrated that minimal-pair surprisal differences predict item-level variation in human grammaticality judgments. 2.4 LLMs and Construction Grammar

Read the complete text