Transducing Language Models

Vésteinn Snæbjarnarson Samuel Kiegeland Tianyu Liu, Reda Boumasmoud Ryan Cotterell Tim Vieira · 2026

Read Transducing Language Models by Vésteinn Snæbjarnarson Samuel Kiegeland Tianyu Liu, Reda Boumasmoud Ryan Cotterell Tim Vieira (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05193.txt.

Introduction

Vésteinn Snæbjarnarson and team formalize a new class of language models built by transforming a model's output deterministically. A byte-pair model does not yield word guesses, nor a DNA model amino-acid chains, yet a change converts outputs to the desired form. Their method uses finite-state transducers, with algorithms composing model and transducer to marginalize over source strings and condition on transformed outputs. Tests converting tokens to bytes, tokens to words, and DNA to amino acids show models adapting at inference time to output needs.

Catalog record

CREATORVésteinn Snæbjarnarson Samuel Kiegeland Tianyu Liu, Reda Boumasmoud Ryan Cotterell Tim Vieira
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE307617
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.05193

Browse the full library · About the Tower of Babel