Unlabeled Data Can Provably Enhance In‑Context Learning of Transformers

Renpu Liu, Jing Yang · 2026

Read Unlabeled Data Can Provably Enhance In‑Context Learning of Transformers by Renpu Liu, Jing Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-10058.txt.

Description

Renpu Liu and Jing Yang show that free text can boost how models learn in context. A model can learn a task from a few shown cases in its prompt. But marked cases cost time and funds. Few can fit in one prompt. That caps how well the model does.

Raw text with no marks is vast and grows each day. Some of it links to the task at hand. The pair show that such text can help for sure, not just by luck. They give a proof based claim. It says raw text can lift in context grades past the cap set by marked cases.

Catalog record

CREATORRenpu Liu, Jing Yang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE96 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-10058

Browse the full library · About the Tower of Babel