Rapid Word Learning Through Meta In-Context Learning

Wentao Wang, Guangyuan Jiang, Tal Linzen, Brenden M. Lake · 2025

Read Rapid Word Learning Through Meta In-Context Learning by Wentao Wang, Guangyuan Jiang, Tal Linzen, Brenden M. Lake (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-14791.txt.

Description

Wentao Wang and co-authors teach language models to learn new words from a few examples, much as children do, in a 2025 paper. A child can hear a fresh word twice and then use it in new settings. Models, by contrast, usually need huge amounts of text before such learning sticks.

Their method, Minnow, trains a model to make up new uses of a word from a few shown uses, with a placeholder token standing in for the word. Repeating this across many words builds a general skill. Models trained this way on child-scale speech matched the few-shot word learning of a far larger model. Tuning big models with Minnow sharpened their word sense too.

Catalog record

CREATORWentao Wang, Guangyuan Jiang, Tal Linzen, Brenden M. Lake
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE120 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-14791

Browse the full library · About the Tower of Babel