Information-Theoretic Storage Cost in Sentence Comprehension

Kohei Kajikawa, Shinnosuke Isono, Ethan Gotlieb Wilcox · 2026

Read Information-Theoretic Storage Cost in Sentence Comprehension by Kohei Kajikawa, Shinnosuke Isono, Ethan Gotlieb Wilcox (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-18217.txt.

Description

Kohei Kajikawa and two co-authors propose a new way to measure the memory load of reading sentences. Their 2026 paper notes that real-time reading forces listeners to hold context in mind to guess what comes next. Older measures score that load with fixed, symbolic grammar rules that give every prediction the same discrete cost.

The new measure counts, in information terms, how much past words reveal about future context under uncertainty. It is continuous, chance-based, theory-neutral, and can be read off pretrained language models. Three English tests back it: it recovers known asymmetries in nested clauses, tracks a grammar-based cost in an annotated corpus, and predicts reading-time variation in two large natural datasets beyond standard predictors. The code is public for reuse.

Catalog record

CREATORKohei Kajikawa, Shinnosuke Isono, Ethan Gotlieb Wilcox
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-18217

Browse the full library · About the Tower of Babel