Hidden Biases in Conditioning Autoregressive Models
François Pachet, Pierre Roy · 2026
Read Hidden Biases in Conditioning Autoregressive Models by François Pachet, Pierre Roy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-07855.txt.
Description
Pachet and Roy expose a hidden bias in constrained generation from autoregressive models, used for rhyming lines, fixed meter, infilling, and positional endings. These systems often look strong, yet their sampling procedures are not exact conditioning of the underlying model. Samples therefore skew away from the true constrained distribution, with no guarantee of covering all valid completions or weighting them correctly. The paper formalizes several exact inference tasks for such models and proves matching hardness results, separating this inferential bias from bias inherited from training data.
Catalog record
| CREATOR | François Pachet, Pierre Roy |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25432 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.07855