Warmup Generations: A Task-Agnostic Approach for Guiding Sequence-to-Sequence Learning with Unsupervised Initial State Generation

Senyu Li, Zipeng Sun, Jiayi Wang, Xue Liu, Pontus Stenetorp, Siva Reddy, David Ifeoluwa Adelani · 2025

Read Warmup Generations: A Task-Agnostic Approach for Guiding Sequence-to-Sequence Learning with Unsupervised Initial State Generation by Senyu Li, Zipeng Sun, Jiayi Wang, Xue Liu, Pontus Stenetorp, Siva Reddy, David Ifeoluwa Adelani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12304.txt.

Description

Senyu Li and co-authors warm up language models with self-made starting points in a 2025 paper. Standard fine-tuning teaches a model to jump straight from input to final output. Recent work shows that giving the model intermediate steps, like outlines or keyword plans, improves quality and clarity of the result.

Those guides usually need human labels or a teacher model. The authors instead let the model generate its own initial state without supervision, then condition the full answer on it. The trick works across tasks because it is task-agnostic. Tests compare it with direct fine-tuning on sequence tasks. The method offers a cheap way to add planning to models where labeled reasoning traces do not exist.

Catalog record

CREATORSenyu Li, Zipeng Sun, Jiayi Wang, Xue Liu, Pontus Stenetorp, Siva Reddy, David Ifeoluwa Adelani
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-12304

Browse the full library · About the Tower of Babel