Human–LLM Collaboration Is Transforming Complexity Metrics in Scientific Texts

R. Alexander Bentley, Blai Vidiella, Damian J. Ruck, Senjuti Dutta, Kai Li, Sergi Valverde · 2026

Read Human–LLM Collaboration Is Transforming Complexity Metrics in Scientific Texts by R. Alexander Bentley, Blai Vidiella, Damian J. Ruck, Senjuti Dutta, Kai Li, Sergi Valverde (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-27052.txt.

Introduction

Bentley and colleagues search millions of research abstracts for signs that large language models are reshaping scientific writing. Language models now help draft and revise papers, and their output feeds back into training data for later models. That loop could slowly shift shared writing habits across science. Abstracts from arXiv between 2010 and 2025 form a natural experiment, with early 2023 as the turning point. The team tracks word-pattern laws and a combined style index linked to model-like writing, in a study appearing in 2026.

After early 2023, the style index jumps sharply. Core word laws, named for Zipf and Heaps, shift only slightly in their exponents. Two subtler changes stand out more clearly. Turnover among the most-used words speeds up markedly. Also, the link between model-like style and three complexity measures flattens after 2022. Together, the patterns suggest human and model writing now form one mixed, evolving system.

Catalog record

CREATORR. Alexander Bentley, Blai Vidiella, Damian J. Ruck, Senjuti Dutta, Kai Li, Sergi Valverde
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.27052

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Human–LLM Collaboration Is Transforming Complexity Metrics in Scientific Texts R. Alexander Bentley, Blai Vidiella, Damian J. Ruck, Senjuti Dutta, Kai Li, Sergi Valverde Abstract While human language has long been studied as a complex system, Large Language Models (LLMs) are rapidly becoming contributors to its dynamics. Because LLMs are trained on human language use, their effects on the broader human–AI linguistic ecosystem are likely subtle at first. As their use becomes more widespread, however, LLMs may alter emergent properties of language, particularly as models increasingly train on mixed human–LLM textual data. Here, we draw on complexity science to look for subtle LLM effects in millions of arXiv abstracts from 2010 to 2025. The year 2023, when LLMs rapidly became widely used, serves as a landmark in a natural experiment. While we find a sharp increase in a composite LLM-associated style index after early 2023, we observe only subtle changes in the exponents of Zipf’s law and Heaps’ law. More compelling, however, are two subtle changes in complexity metrics that emerge from 2023 onward. First, turnover among top-ranked words increases sharply. Second, the positive relationship between the LLM-associated style index and three complexity metrics—vocabulary size and the exponents of Heaps’ and Zipf’s laws—becomes flatter after 2022. Together, these patterns are consistent with changes in the emergent properties of scientific text in a mixed human–AI linguistic ecosystem. Keywords: Generative AI; AI-generated content, retrieval collapse; cultural evolution Introduction Large Language Models (LLMs), rapidly adopted since early 2023, are transforming education, scientific research, and cultural communication, reshaping how knowledge is produced, evaluated, and transmitted. As LLMs support tasks ranging from literature reviews and data analysis to collaborative reasoning [ 1 , 2 , 3 , 4 , 5 ] , surveys indicate that a substantial fraction of users also engage with LLMs for psychological support [ 6 ] . As human trust in scientific expertise evolves [ 7 ] , dyadic conversations with an LLM can reduce conspiracy beliefs [ 8 ] , and in group settings they guide diverse participants toward consensus [ 9 ] . As LLMs become embedded in daily communication, humans and models increasingly form a mixed linguistic population [ 10 ] . There is bidirectional learning: LLMs are trained on human texts [ 11 ] , while human language adapts to LLM outputs [ 12 , 13 ] . This reciprocal coupling makes it possible to treat LLMs as a new feedback within cultural evolution [ 10 , 14 , 15 , 16 , 17 ] , potentially as consequential as early writing, markets, or early governments [ 18 ] . Unlike Deep-Q systems of the 2010s, which learned through largely solitary reinforcement learning [ 19 ] , LLMs generate responses by sampling from probability distributions learned from large corpora of human-generated text [ 20 ] . What will be the future of knowledge or language in this increasingly mixed human-AI population? There are at least two perspectives. On one hand, LLMs may reduce lexical complexity, partly through sentence simplification [ 21 ] but also through recycling their training data with limited novelty/invention [ 22 , 23 , 24 , 25 ] . Additionally, if AI-generated content begins to overwhelm human-authored content in AI model training data [ 26 , 27 ] , successive generations of AI models may increasingly be trained on their own outputs, potentially diminishing quality and complexity [ 28 , 29 ] . On the other hand, interacting LLM systems may produce emergent dynamics that cannot be reduced to individual models [ 30 ] . Collective biases can arise that are not simply the average of individual tendencies, with outcomes shaped by feedback, coordination, and the formation of shared conventions [ 10 ] . In this view, collective LLM outputs are better understood as emergent outcomes of complex interactions between pre-trained knowledge, embedded priors, and social context [ 31 ] . To connect LLM text generation with models of cultural evolution, we begin with the core sampling process. In LLMs, the probability, P_{i} , of choosing token i essentially follows a softmax function involving logit scores z_{i} of the tokens and temperature T : P_{i}=\frac{e^{z_{i}/T}}{\sum_{j=1}^{N}e^{z_{j}/T}} (1) This is similar to a general model of cultural evolution [ 32 , 33 , 25 ] , in which the probability of choosing token i at time t is: P_{i}=p_{i}(t)^{J}\frac{e^{\beta U_{i}}}{\sum_{j=1}^{N(t)}p_{j}(t)^{J}e^{\beta U_{j}}} (2) where p_{i}(t) is the relative frequency of token i among the N(t) different tokens at time t , U_{i} is its utility, J is social learning intensity and \beta is transparency of utility [ 32 , 25 ] . Equation 2 captures realms of complexity, including rational individual choice (high \beta , low J ), following experts (high \beta , high J ) and random choice ( J=0 and \beta=0 ), with a rich diversity of dynamics from this simple model [ 25 ] . When J=1 and \beta=0 in Equation 2 , the process should resemble a neutral copying model [ 22 ] , where choices are made with probabilities proportional to frequency p_{i} . An important difference with LLMs in equation ( 2 ) is the p_{i}^{J} term, which captures the popularity bias. In contrast, z_{i}/T in Equation ( 1 ) does not directly weight choices by token frequency. At T=1 , the LLM samples from its native learned distribution, while increasing T progressively flattens differences among token logits. Using a first-order expansion for large T , P_{i}\approx\frac{1}{N}+\frac{1}{NT}(z_{i}-\bar{z}), (3) where \bar{z} is the mean logit across candidate tokens, showing that as T\to\infty the distribution approaches uniform random choice, P_{i}\to 1/N , analogous to lowering \beta in Equation 2 with J=0 (not J=1 ). Because randomness in LLM outputs arises from sampling procedures rather than individual stochastic choice, it may be that noisy LLM input into the lexical ecosystem increases turnover in word rankings. Introducing LLMs among humans as a mixed population, then, could either increase or decrease randomness, depending on whether LLM outputs amplify conformity or inject novelty. A comprehensive survey of the linguistic characteristics of AI-generated text found that a large number of studies converge on the conclusion that AI-generated text is generally less lexically diverse than human-written text, also exhibiting a smaller vocabulary size, higher repetition rates, and higher n-gram reuse. A comparison of human versus LLM-generated news text found that human texts consistently exhibited higher lexical diversity than all LLM families tested [ 34 ] . Another study, however, found greater lexical diversity in LLM-generated text than human student-written texts [ 35 ] . This suggests a range of predictions for the stochastic process of word selection, drawing on iterated simulations of the unified cultural evolution model [ 25 ] as well as neutral copying models [ 22 , 36 , 23 ] . These models generate several empirical signatures, including the well-known Zipf distribution of variant frequencies and Heaps’ law between corpus size and the number of unique tokens, and turnover in the top- y ranked items, denoted z_{y} , measured as the fraction of new items entering a ranked list of the top y items at each time step [ 37 , 38 , 39 ] . In neutral models, turnover z_{y}\propto\mu^{0.55}y^{0.86}N^{0.1} , where N is population size,

Read the complete text