A Distributional Perspective on Word Learning in Neural Language Models

Filippo Ficarra, Ryan Cotterell, Alex Warstadt · 2025

Read A Distributional Perspective on Word Learning in Neural Language Models by Filippo Ficarra, Ryan Cotterell, Alex Warstadt (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05892.txt.

Description

How language models learn words over time is reframed by Filippo Ficarra, Ryan Cotterell, and Alex Warstadt in a 2025 paper. Models are often read as stand-ins for child learners, but word learning paths in models lack agreed measures. Child paths, by contrast, are well charted.

The team defines word knowledge through the model's own probability patterns for each word. Older signatures miss where a word cannot go and how strongly contexts are favored, they argue. New signatures capture both fit and misfit, plus graded preference. Training small models from scratch yields learning curves for picked words. The measures give a cleaner bridge between model training and child language research.

Catalog record

CREATORFilippo Ficarra, Ryan Cotterell, Alex Warstadt
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE75 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-05892

Browse the full library · About the Tower of Babel