Likelihood Variance as Text Importance for Resampling Texts to Map Language Models

Momose Oyama, Ryo Kishino, Hiroaki Yamagiwa, Hidetoshi Shimodaira · 2025

Read Likelihood Variance as Text Importance for Resampling Texts to Map Language Models by Momose Oyama, Ryo Kishino, Hiroaki Yamagiwa, Hidetoshi Shimodaira (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15428.txt.

Description

Momose Oyama and co-authors present, in a 2025 paper, a cheaper way to map the world of language models. A model map places many models in one shared space by comparing how likely each finds the same texts. Scoring every text with every model is costly. Their fix resamples the text pool, keeping texts in proportion to how much models disagree on them, measured by the variance of their log-likelihoods.

Texts that split the models carry the most signal about how they differ. With that weighting, about half as many texts gave maps as accurate as the full pool. Adding a fresh model to an old map also got cheaper. The method opens the door to maps that track a fast-growing model zoo without a matching growth in computing cost.

Catalog record

CREATORMomose Oyama, Ryo Kishino, Hiroaki Yamagiwa, Hidetoshi Shimodaira
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-15428

Browse the full library · About the Tower of Babel