Cognitive Limits Shape Language Statistics

Alessandro Bellina, Vito D. P. Servedio · 2025

Read Cognitive Limits Shape Language Statistics by Alessandro Bellina, Vito D. P. Servedio (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17512.txt.

Introduction

Alessandro Bellina and Vito Servedio present a 2025 paper on word use in human language. Word frequency follows a known pattern, with a few words used very often. Less used words break that pattern, and the authors ask why. Their frame joins ideas from information study, networks, and how new options open up.

The authors argue that limits in the human mind shape the pattern. Common words form a core that is easy to move through. Rare words sit in a less neat zone and need word pairs to carry new ideas. A simple model based on new options matched real word counts well. The work may guide tools that work with language.

Catalog record

CREATORAlessandro Bellina, Vito D. P. Servedio
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-17512

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Cognitive Limits Shape Language Statistics Alessandro Bellina, Vito D. P. Servedio Abstract Statistical regularities in human language have fascinated researchers for decades, suggesting deep underlying principles governing its evolution and information structuring for efficient communication. While Zipf’s Law describes the frequency-rank distribution of words, deviations from this pattern—particularly for less frequent words—challenge the notion of an entirely optimized communication system. Here, we present a theoretical framework that integrates concepts from information theory, network science, and adjacent possible to explain these deviations. We propose that language evolves through optimization processes constrained by the finite cognitive capacities of humans. This results in a dual structure within language: while frequent words form an optimized, highly navigable core, less frequent words reside in a suboptimal regime, requiring more complex combinations to convey meaning. Our findings reveal that Zipf’s exponents’ deviation to larger values—from 1 to 2—marks a transition from an optimal to a suboptimal state, dictated by cognitive limits. This transition imposes a fundamental limit on communication efficiency, where cognitive constraints lead to a reliance on combinations of words rather than the creation of new vocabulary to express an open-ended conceptual space. A simple model based on the adjacent possible remarkably aligns with the empirical frequency-rank distribution of words in a language. These insights have significant implications for natural language processing and the design of artificial linguistic models, offering new perspectives on optimizing human and machine communication. A Preprint today 1 Introduction Human language, with its complex structures and dynamics, has been the subject of extensive scientific research [ 1 , 2 ] . Over the years, researchers have identified intriguing statistical properties that govern language usage [ 3 , 4 , 5 , 6 ] . One of the most significant regularity is Zipf’s Law [ 1 , 7 , 8 , 9 ] , which states that the frequency f(R) of the R -th most frequent word is inversely proportional to its rank R , following f(R)\sim R^{-1} [ 10 ] . This relation, formulated by George Kingsley Zipf in 1936, can be interpreted as the result of the Principle of Least Effort applied to communication [ 1 , 3 , 11 , 12 , 13 ] . In essence, Zipf’s Law reflects an optimization process in communication [ 14 , 15 , 16 , 17 , 18 ] , interpreted as minimizing the effort between the speaker and the hearer [ 19 , 20 , 21 , 22 ] . However, the availability of Big Data has allowed for larger-scale analyses, revealing significant deviations from Zipf’s Law in the tail of the word distribution [ 23 , 24 ] . Several studies have recognized that Zipf’s Law primarily applies to the most frequent words, often referred to as the kernel or core of the language [ 25 , 26 , 27 , 28 ] . For infrequent words, which lie in the tail of the distribution, the scaling relation changes to f(R)\sim R^{-\alpha} with \alpha\approx 2 . Recent research have explored the emergence of statistical criticalities as a fundamental characteristic of maximally informative samples [ 17 , 29 , 30 ] , a concept that resonates with the theory of effort minimization in communication. In this context, language evolution can be seen as an optimization process aimed at conveying the maximum possible information about an unknown generative process, and Zipf’s Law naturally emerges as the footprint of maximal optimization [ 31 ] . Another perspective to uncover the statistical properties of natural language is network representation [ 26 , 32 , 33 , 34 ] . In particular, Word Co-occurrence or Word Adjacency Networks consider words as nodes, with links between any adjacent words in the text. A striking property of these networks is that the kernel or core , consisting of the most frequent words, reveals a topology reminiscent of a Barabási-Albert model [ 26 ] . This network structure offers high navigability due to its scale-free properties [ 35 ] , which align with the optimization required for efficient communication [ 15 ] . In contrast, nodes representing infrequent words still exhibit scale-free properties, but with an anomalous degree exponent [ 36 ] . These networks lack the optimized organization needed to enhance navigability, which characterizes the kernel [ 37 ] . In this paper, we propose a theoretical framework to understand the evolution of human language by integrating information theory [ 38 , 39 ] , complex network theory [ 32 ] and innovation dynamics concepts [ 40 ] . Our main hypothesis is that language tends toward optimization but is constrained by finite cognitive capacities, leading to a suboptimal regime for infrequent words. We argue that humans optimize communication for a limited number of frequently used words, while performance decreases as the lexicon expands. This loss of optimization manifests itself in a slowdown in vocabulary growth [ 41 ] , which forces the need to associate meanings with increasingly complex combinations of words, thus extending the length of the text. Additionally, our approach reveals structural limits on non-optimization that still preserve communication capabilities. This explains why the exponent \alpha\approx 2 of the frequency-rank distribution serves as an intrinsic limit in linguistic systems. Finally, we introduce a generative model based on the concept of the Adjacent Possible [ 42 , 40 , 43 , 44 , 45 ] , which reproduces the evolution mechanism and provides a quantitative account of the observed scaling exponents. 2 Zipf’s Law and optimal compression Figure 1: Statistical properties of languages. Top left: Normalized frequency-rank distribution p(R) of single words (1-grams) for English, French, Italian, Spanish, and German. For each of these languages, the most frequent words follow a power law with exponent -1, known as Zipf’s Law. In the tail of the distribution, the exponent shifts to 2 for most languages, indicating a transition from an optimal to a suboptimal regime. This shift reflects the need to form combinations of words to represent an increasingly complex conceptual space using a finite vocabulary. In German, however, the exponent bends less sharply, with a value of approximately 1.5. This reflects its strategy of expanding the vocabulary by creating new words from combinations of existing ones [ 91 ] , as shown in the bottom panel. Top right: Frequency-rank distribution of n-grams (obtained aggregating from 1-grams to 5-grams) for the five languages. Here, n-grams are treated as representations of individual concepts, and the resulting distribution follows Zipf’s Law without deviation, indicating efficient concept representation through word combinations. Notably, similar words in related languages appear at comparable positions in the distribution. For instance, in German, the compound word “Wasserflasche” appears close in rank to the corresponding English bigram “water bottle”. This suggests that, despite differences in word formation strategies, these distributions directly reflect underlying conceptual representations. Plots are generated using n-gram frequencies from the Google Ngram database [ 46 ] . Bottom: In this panel, we illustrate the effect of using word combinations to represent complex concepts. Cognitive constraints limit the indefinite growth of vocabulary, leading to an increased reliance on word combinations as concepts grow more complex. This change impacts the word frequency distribution, causing a shift from an exponent of 1 to 2. German

Read the complete text