The potential – and the pitfalls – of using pre-trained language models as cognitive science theories

Raj Sanjay Shah, Sashank Varma · 2025

Read The potential – and the pitfalls – of using pre-trained language models as cognitive science theories by Raj Sanjay Shah, Sashank Varma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-12651.txt.

Introduction

Raj Sanjay Shah and Sashank Varma weigh a bold idea in cognitive science in this two thousand twenty-five paper. The idea is to treat pre-trained language models as theories of the human mind. Such models now predict language behavior with uncanny skill. That skill tempts researchers to read them as explanations, not just tools. The authors sort the real promise from the traps in that move.

A model can match behavior while working nothing like a brain. It may also encode the test data in ways that fake understanding. The paper sets out what a model must show before it counts as a theory. It urges tests that probe mechanism, not just fit. Cognitive scientists and AI researchers share the audience for this caution.

Catalog record

CREATORRaj Sanjay Shah, Sashank Varma
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-12651

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The potential – and the pitfalls – of using pre-trained language models as cognitive science theories Raj Sanjay Shah, Sashank Varma Abstract Many studies have evaluated the cognitive alignment of Pre-trained Language Models (PLMs), i.e., their correspondence to adult performance across a range of cognitive domains. Recently, the focus has expanded to the developmental alignment of these models: identifying phases during training where improvements in model performance track improvements in children’s thinking over development. However, there are many challenges to the use of PLMs as cognitive science theories, including different architectures, different training data modalities and scales, and limited model interpretability. In this paper, we distill lessons learned from treating PLMs, not as engineering artifacts but as cognitive science and developmental science models. We review assumptions used by researchers to map measures of PLM performance to measures of human performance. We identify potential pitfalls of this approach to understanding human thinking, and we end by enumerating criteria for using PLMs as credible accounts of cognition and cognitive development. 1 Introduction With the improving performance of pre-trained language models Touvron et al. (2023) ; Gemini Team (2023) ; OpenAI (2023) ; Wei et al. (2022) , researchers are increasingly advocating for their use as computational models of cognition Piantadosi (2023) ; Mahowald et al. (2024) ; Warstadt and Bowman (2024) ; Coda-Forno et al. (2024) . This is true for many domains including mathematical reasoning Shah et al. (2023) ; Ahn et al. (2024) , language comprehension Warstadt et al. (2020) ; Li et al. (2024) ; Hu et al. (2024a) , concept understanding Vemuri et al. (2024) , spatial reasoning Ramakrishnan et al. (2024) and analogical reasoning Webb et al. (2023) ; Hu et al. (2023) . For example, Shah et al. (2023) investigated the latent number representations of PLMs, finding that they showed the distance , size , and ratio effects observed in humans and understood to be the behavioral signatures of a “mental number line” Moyer and Landauer (1967) ; Parkman (1971) ; Halberda et al. (2008) . To take another example, Raven’s Progressive Matrices test is the standard psychometric measure of fluid reasoning . Although Raven’s problems are visual, Webb et al. (2023) translated them to equivalent "digit encodings" and showed that PLMs perform as well as humans on this test. More generally, many works combine tests across multiple task domains to form comprehensive benchmarks that enable scientific evaluation of the alignment of ML models to human cognition Chang et al. (2024) ; Zhuang et al. (2023) ; Shah et al. (2024) ; Coda-Forno et al. (2024) ; Wang et al. (2024) ; Tan et al. (2024) . Recently, researchers have begun using PLMs to additionally model the development of cognition in children Hosseini et al. (2022) ; Kosoy et al. (2023) ; Frank (2023) ; Shah et al. (2024) ; Tan et al. (2024) . For example, Portelance et al. (2023) suggest the use of language models to predict the age of acquisition of words in children. Wang et al. (2024) use Piaget’s Theory of Cognitive Development to estimate that models like GPT-4o show cognitive abilities similar to 20-year-old humans. Instead of just looking at model end states, Shah et al. (2024) investigated developmental trajectories and path dependence: whether the performance improvements of PLMs over training track the growth of cognitive abilities in children over development. Similarly, Tan et al. (2024) explored developmental parallels by comparing the learning trajectories of vision-language models to both child and adult behavioral data. To take a final example, researchers have begun varying the exposure of PLMs to multiple languages during pre-training to understand the differential rates of bilingual language development Evanson et al. (2023) ; Marian (2023) ; Sharma et al. (2024) . Our Contributions Developmental Alignment We highlight the alignment of PLMs with human cognitive development and discuss how model checkpoints and training trajectories can mimic children’s developmental trajectories. Pitfalls of Commission and Omission We introduce a new categorization of pitfalls into those of commission (methodological mistakes, such as distal linking hypotheses) and those of omission (neglecting broader contexts like psychometric and developmental data), providing a newer framework for identifying error types. Linking Hypotheses and Measures We provide a detailed critique of linking hypotheses, focusing on the need for a robust mapping between model outputs and human cognitive measures, and we warn about distal links that can obscure interpretability and scientific validity. Promises of Adjacent Fields We draw interdisciplinary parallels, such as comparing functional mapping in neuroscience to mechanistic interpretability in PLMs, and discuss how these analogies can generate hypotheses about neural activity. Best Practices for Credibility in Cognitive Science We enumerate explicit criteria for using PLMs as credible theories in cognitive science, such as focusing on functional alignment over explanatory adequacy and ensuring sufficiency in mapping between PLMs and human behavior. Table 1: Summary of our contributions - distinctions between our work and previous work by Ivanova (2023) and Mahowald et al. (2024) . Reader’s Guide to the paper In this paper, we describe the pitfalls and promise of using PLMs and candidate theories in cognitive and developmental science. The contributions of this paper are threefold. (1) We first review the pitfalls of using PLMs in psychological science and caution researchers against over-interpreting the alignment of PLMS to human cognition and its trajectory over development. (2) Next, we review the standard assumptions researchers use to map measures of PLM performance to human performance measures. (3) Finally, we build upon previous work Ivanova (2023) ; Mahowald et al. (2024) ; Cuskley et al. (2024) ; Birhane and McGann (2024) in enumerating best practices for cognitive evaluations of PLMs (refer to table 1 ). 2 PLMs as theories in cognitive and developmental science Figure 1: The three-stage mapping between human data and model performance when establishing the sufficiency and utility of PLMs as cognitive and developmental science models. PLMs make promising candidates as theories of cognitive science due to the nature of language model training Hardy et al. (2023) . PLMs acquire a wide range of capabilities in the pre-training phase, essentially obtaining human-like behaviors “for free” without the need for extensive task-specific tuning or adaptations Demszky et al. (2023) ; Weng (2024) ; Yang et al. (2024) ; Minaee et al. (2024) ; OpenAI (2023) . For example, the probabilistic nature of PLM text generation lures researchers to draw parallels with the human decision-making process, where humans anticipate future courses of action based on their past experiences and context. These models can provide insights into how humans may handle ambiguous situations where multiple interpretations are possible McGrath et al. (2020) ; Gawlikowski et al. (2023) ; Dong et al. (2024) ; Belem et al. (2024) . Moving from cognitive science to developmental science, PLMs have been shown to gradually learn linguistic sensitivities to syntax, semantics, morphology, etc., from mere exposure to (i.e., masked word prediction of) large amounts of text data. This raises the possibility that their learning trajectories mimic the development trajectories of children acquiring language Aher et al. (2023) ; Duan et al. (2024) ; Shah et al. (2024) . This allure of PLMs as theories of cognitive science also extends to adjacent research areas. One example

Read the complete text