Model

Corpus · 2026

Read Model by Corpus (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-32002.txt.

Description

Alimaskina and five colleagues, at BRAIn Lab and Yandex Research, probe a hidden weakness in teaching language models from self-generated question-answer pairs. A model reads a document, invents questions about it, answers them, and the pairs become training data. That generation step quietly acts as a policy: it chooses which evidence becomes signal and decides how the evidence gets answered.

Coverage levels off quickly and gathers around the most prominent passages, with varied prompts ending up asking about the same passages, while messy markup can steer what gets asked. Answering models also obey instruction-like passages embedded in the source text. When each generated question is anchored to a fixed target, the selection of evidence becomes less biased. Filtering instruction-like spans before answering cuts mean injection compliance from 88 percent to 13 percent in their evaluation.

Catalog record

CREATORCorpus
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.32002

Browse the full library · About the Tower of Babel