Exact Sequence Interpolation with Transformers

Albert Alcalde, Giovanni Fantuzzi, Enrique Zuazua · 2025

Read Exact Sequence Interpolation with Transformers by Albert Alcalde, Giovanni Fantuzzi, Enrique Zuazua (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-02270.txt.

Description

Albert Alcalde and co-authors prove that transformers can fit sequence data exactly in a 2025 maths paper. Transformers power modern language tools, yet sharp proofs of what they can represent stay rare. Exact fit means a network that returns each target output with no error at all.

The team builds such a network for any finite set of input and output sequences. Its size grows with total output length, not input length, a neat saving. Layers swap between feed-forward steps and attention that clusters like items. The proof starts with a strict attention form, then extends to the soft form used in practice, with training guarantees added.

Catalog record

CREATORAlbert Alcalde, Giovanni Fantuzzi, Enrique Zuazua
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE93 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-02270

Browse the full library · About the Tower of Babel