Modeling the Human Lexicon under Temperature Variations: Linguistic Factors, Diversity and Typicality in LLM Word Associations
2026
Read Modeling the Human Lexicon under Temperature Variations: Linguistic Factors, Diversity and Typicality in LLM Word Associations (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-18171.txt.
Introduction
Andueza, Candito, and Huyghe model the human lexicon in LLMs through word association tests. They compare human pairs from SWOW with new output from Mistral, Llama, and Qwen at varied temperatures. All models mirror human trends for word frequency and concreteness. Larger Qwen acts like one typical human: typical but low-variety replies. Smaller models vary more but stay less typical, and higher temperature trades typicality for variety.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41039 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.18171
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Modeling the Human Lexicon under Temperature Variations: Linguistic Factors, Diversity and Typicality in LLM Word Associations Abstract Large language models (LLMs) achieve impressive results in terms of fluency in text generation, yet the nature of their linguistic knowledge – in particular the human-likeness of their internal lexicon – remains uncertain. This study compares human and LLM-generated word associations to evaluate how accurately models capture human lexical patterns. Using English cue-response pairs from the SWOW dataset and newly generated associations from three LLMs (Mistral-7B, Llama-3.1-8B, and Qwen-2.5-32B) across multiple temperature settings, we examine (i) the influence of lexical factors such as word frequency and concreteness on cue-response pairs, and (ii) the variability and typicality of LLM responses compared to human responses. Results show that all models mirror human trends for frequency and concreteness but differ in response variability and typicality. Larger models such as Qwen tend to emulate a single “prototypical” human participant, generating highly typical but minimally variable responses, while smaller models such as Mistral and Llama produce more variable yet less typical responses. Temperature settings further influence this trade-off, with higher values increasing variability but decreasing typicality. These findings highlight both the similarities and differences between human and LLM lexicons, emphasizing the need to account for model size and temperature when probing LLM lexical representations. Keywords: word association, large language model, mental lexicon, frequency, concreteness, temperature Modeling the Human Lexicon under Temperature Variations: Linguistic Factors, Diversity and Typicality in LLM Word Associations Maria A. Rodriguez 1,3 , Marie Candito 2 , Richard Huyghe 1 1 University of Fribourg 2 LLF (Université Paris Cité / CNRS) 3 Lucerne University of Applied Sciences and Arts [email protected] Abstract content 1. Introduction The capacity of language models to generate texts of remarkable linguistic quality has spurred increasing interest in their underlying linguistic knowledge. Despite comparable abilities in written text production, human beings and language models differ fundamentally in the process of language acquisition: while human language learning is grounded in real-world experience and extralinguistic context, language models learn exclusively through large-scale exposure to text. This raises the question of whether these different learning processes result in comparable linguistic competence, understood as the implicit knowledge of linguistic systems and grammatical structures (see 20 for an overview). A particularly relevant domain for this comparison is the organization of the lexicon, paralleling research into the human mental lexicon. To explore lexical structures, recent investigations have examined the ability of large language models (LLMs) to generate word associations, comparing these with human word associations. These studies have employed diverse methodologies and yielded mixed results, revealing both shared characteristics and notable divergences between LLM- and human-generated word associations. In this paper, we contribute to this line of inquiry by examining the lexical factors that determine word associations in both groups and by proposing refined metrics for evaluating the human-likeness of associations generated by LLMs. Our results provide new evidence of similarities between human and LLM word associations, while also showing that LLMs vary considerably in their ability to capture the diversity of human responses, with variability depending on models, settings, and cue words. 2. Related work The word association task is a classic psychological experiment in which participants respond spontaneously with the first word(s) that come to mind when presented with a specific cue word (e.g., holiday \rightarrow vacation , birth \rightarrow baby , curly \rightarrow hair ). Word associations have long been used by linguists and psychologists to investigate cognitive processes, psychological behavior patterns, language acquisition, multilingualism, as well as the organization of the mental lexicon (see 16 ; 5 ; 13 ; 7 ; 9 ; a.o.). In particular, word association norms, assembled into networks linking cue and response words, have been shown to closely align with psycholinguistics results related to the structure of the mental lexicon – as evidenced by analyses of the SWOW-EN dataset 6 , the largest existing collection of human-generated word associations in English. Given their capacity to reveal lexical organization, word associations provide a useful proxy for probing underlying lexical structures in LLMs and for comparing them with those of humans. The ability of LLMs to replicate human word associations has been explored in several recent studies. 19 investigated differences between human and model-generated word associations in Slovene and English. They fine-tuned encoder-decoder models to generate an open-ended list of associates given a cue. Based on examples derived from the English and Slovene SWOW datasets, the authors reported a low overlap between LLM and human responses. They further compared the types of responses provided by humans and LLMs, according to the classification into 4 categories (form, meaning, position, and erratic) and found a significantly higher proportion of erratic responses, i.e., responses apparently unconnected to cues, in LLMs. However, across relevant responses, they found similar distributions of meaning, form, and position (i.e., syntagmatic) relations between cues and responses. 2 ; 1 conducted a more direct comparison of human and LLM word associations by using similar prompts for LLMs as for humans (i.e., asking them to provide three response words for each cue). Based on the cue set from SWOW-EN, they created the LWOW datasets by eliciting English word associations from three LLMs (Mistral-7b, Llama3.1-8b, and Claude-3.5-Haiku-latest). The prompting was performed 100 times for each cue, and answers from these prompting repetitions were considered as parallels to answers from 100 human participants. The authors built 4 weighted networks from cue–response pairs – one for SWOW-EN and one for each prompted LLM – where edge weights reflected participant or prompt repetition. They validated the LLM networks using semantic priming data, showing that normalized spreading activation levels predicted lexical decision reaction times. Additionally, the authors conducted an exploratory analysis of the overlap between LLM and human responses, finding that human responses were considerably more diverse. This comparison was made between the sets of unique responses, irrespective of the associated cues. Unfortunately, the authors did not specify the temperature setting used in generating the LWOW datasets, although this hyperparameter may greatly impact the variability and human-likeliness of responses (see Section 4 ). 1 1 1 The authors did not report the temperature parameter in their code and did not provide this information upon request. Our preliminary experiments using LWOW suggested that the default temperature of 1 was unlikely to have been applied. This led us to create new datasets to examine the effects of temperature variation. 21 also created datasets of LLM-generated word associations based on the SWOW-EN cues, but using larger models (GPT-4o and Llama-3.1-70b-instruct), 2 2 2 These datasets are not publicly available. while focusing on evaluating whether LLMs capture core psycholinguistic dimensions of the mental lexicon. The authors found mixed results regarding the similarity between LLM- and human-derived networks. They observed reduced alignment for models compared to humans, such as lower correlations between association frequency and reaction times in lexical decision and naming tasks, reduced lexical diversity, and differences in response distributions. Yet, correlations between random-walk relatedness scores and human judgments were nearly as high for human and model-derived networks. In the present study, we further investigate the ability of LLMs to generate human-like word associations. Rather than analyzing association networks, we focus directly on cue–response pairs and conduct two experiments using the