Using Information Theory to Characterize Prosodic Typology: The Case of Tone, Pitch-Accent and Stress-Accent

Ethan Gotlieb Wilcox, Cui Ding, Giovanni Acampa, Tiago Pimentel, Alex Warstadt, Tamar I. Regev · 2025

Read Using Information Theory to Characterize Prosodic Typology: The Case of Tone, Pitch-Accent and Stress-Accent by Ethan Gotlieb Wilcox, Cui Ding, Giovanni Acampa, Tiago Pimentel, Alex Warstadt, Tamar I. Regev (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-07659.txt.

Introduction

Ethan Gotlieb Wilcox and co-authors use information theory to sort languages by how they use pitch. Some languages, like Cantonese, use pitch to tell words apart, while others use it only for stress or tune. Their 2025 claim: pitch should carry more word linked information in the first kind.

The team recorded speakers reading sentences aloud in ten languages from five families, then measured how pitch curves link to the text. Pitch curves held like amounts of entropy across all languages. But in tonal languages, the text predicted the curve far better, so the shared information ran higher. The result backs a gradient view of language types over hard boxes.

Catalog record

CREATOREthan Gotlieb Wilcox, Cui Ding, Giovanni Acampa, Tiago Pimentel, Alex Warstadt, Tamar I. Regev
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-07659

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Using Information Theory to Characterize Prosodic Typology: The Case of Tone, Pitch-Accent and Stress-Accent Ethan Gotlieb Wilcox, Cui Ding, Giovanni Acampa, Tiago Pimentel, Alex Warstadt, Tamar I. Regev Abstract This paper argues that the relationship between lexical identity and prosody—one well-studied parameter of linguistic variation—can be characterized using information theory. We predict that languages that use prosody to make lexical distinctions should exhibit a higher mutual information between word identity and prosody, compared to languages that do not. We test this hypothesis in the domain of pitch, which is used to make lexical distinctions in tonal languages, like Cantonese. We use a dataset of speakers reading sentences aloud in ten languages across five language families to estimate the mutual information between the text and their pitch curves. We find that, across languages, pitch curves display similar amounts of entropy. However, these curves are easier to predict given their associated text in tonal languages, compared to pitch- and stress-accent languages; the mutual information is thus higher in these languages, supporting our hypothesis. Our results support perspectives that view linguistic typology as gradient, rather than categorical. https://github.com/picol-georgetown/Prosody_Typology 1 Introduction One central tension in linguistics is between linguistic universality and diversity . The world contains some 7,000 languages ( Ethnologue, 2023 ) , each with its unique and idiosyncratic lexicon, phonological inventory, and grammar. At the same time, linguistic properties are shared between sets of related languages ( Croft, 2002 ) , and some features appear, or covary, across languages, giving rise to the hypothesis that human language is governed by a set of universal principles ( Greenberg, 2005 ) . Major advances in the study of language have been made through the introduction of frameworks that can describe both the typological variation observed between languages as well as the universal consistencies observed across languages. Examples of such frameworks are the Principles and Parameters approach for syntactic structure ( Chomsky, 1993 ; Culicover, 1997 ) and Optimality Theory for phonological systems ( Prince and Smolensky, 2004 ) . One promising candidate for this type of framework is information theory ( Shannon, 1948 ) . Studies have argued that information-theoretic approaches can explain universal principles in languages, including the distribution of word lengths ( Zipf, 1949 ; Piantadosi et al., 2011 ; Pimentel et al., 2023 ) , the organization of semantic systems ( Kemp et al., 2018 ; Zaslavsky et al., 2018 ; Zaslavsky et al., 2021 ) , word orders ( Dyer et al., 2021 ) as well as language processing phenomena ( Futrell et al., 2020 ; Wilcox et al., 2023 ) . However, information-based approaches are less widely used to describe typological variation (although cf., Futrell et al., 2020 ; Pimentel et al., 2020 ; Socolof et al., 2022 ; Steuer et al., 2023 ). In this paper, we take one well-studied crosslinguistic parameter—whether or not a language has lexical tone—and argue that it can be characterized information-theoretically, as the amount of mutual information between a lexical item (i.e., a word) and the pitch curve associated with that word. Our goal is to demonstrate how an information-based approach can be used to characterize crosslinguistic variation, as well as to showcase how NLP methods can be used to formally quantify properties that are debated in the formal linguistics literature, e.g., whether, or to what extent, a given language or dialect is tonal ( Hyman, 2006 ) . The domain we are interested in is prosody—the melody of speech. A word’s prosody is transmitted through several unique features, including its duration, energy (perceived as loudness), and fundamental frequency (perceived as pitch). Pitch, specifically, is the main focus of our study. Crucially, the role that pitch plays varies across languages, with phonologists traditionally placing languages in three broad categories: In tonal languages such as Vietnamese, Mandarin, and Yoruba, all or most syllables carry one of several discrete pitch contours which differentiate between lexical items; in stress-accent languages such as English and Italian pitch does not differentiate between lexical identity at all, playing other roles like providing cues for stress placement, or indicating whether or not a sentence is a question. In an intermediate set of languages, called pitch-accent languages, such as Swedish or Japanese, pitch contours are lexically contrastive, but they are not present on every word. The reason why we focus on pitch and its relationship to tone is that in the phonology literature, this issue has been at the forefront of debates about how one ought to make typological distinctions. Using tonal systems as an example, some have argued that the job of typology is to identify language “types” ( Hagège, 1992 , i.e., the three in the above paragraph;) , while others have argued that typology should be viewed as laying out a typological “continuum” over several prosodic properties ( Hyman, 2006 ) , and have questioned whether stress-accent languages are a single, naturally-occurring linguistic category. As an example, Western Basque, Tokyo Japanese, and Luganda (a Bantu language) make some lexical distinctions based on pitch. However, the number of such words varies between the languages, and their “tonal” systems interact idiosyncratically with other aspects of the language’s phonology ( Hyman, 2006 ) . Is it fair to say that these languages belong to a single type? And if they fall on a continuum, then what metric should one use as its basis? Information-theoretic approaches offer a new way of exploring such continua, and can offer new evidence for or against typological clusters. Our approach offers theoretically motivated quantities, which are estimated from raw audio data, and therefore inherently capture several of the prosodic properties that have been hypothesized to make up tonal continua. Our contribution is in line with several recent studies that have recast other aspects of typological variation in information-theoretic terms, e.g., for morphological fusion ( Socolof et al., 2022 ) and vowel harmony ( Steuer et al., 2023 ) . Specifically, we hypothesize that because tonal languages use pitch to distinguish lexical identity, given a lexical item’s identity, it should be easier to predict a word’s pitch curve in tonal compared to non-tonal languages. Information-theoretically, this means there should be more mutual information between lexical identity and pitch in tonal languages, such as Cantonese, than in non-tonal languages, such as English. To test this hypothesis, we use a pipeline ( Wolf et al., 2023 ) originally developed in English to measure the mutual information ( \mathrm{MI} ) between prosody and written text; where text is used as a proxy for lexical identity. We make several technical contributions to this pipeline, enabling it to produce more accurate \mathrm{MI} estimates across languages. We measure mutual information for ten typologically distinct languages: English, French, Italian, German, Swedish (Indo-European), Mandarin, Cantonese (Sino-Tibetan), Japanese (Japonic), Thai (Kra–Dai), and Vietnamese (Austroasiatic). These languages are traditionally classified as either stress-accent, pitch-accent, or tonal. We find that, across languages, pitch curves display similar amounts of entropy, suggesting that the information conveyed by the pitch channel is conserved cross-linguistically. However, these curves are easier to predict given their associated text in the tonal languages, compared to pitch- and stress-accent languages, and thus the \mathrm{MI} is higher in these languages, supporting our hypothesis. Interestingly, the mutual information

Read the complete text