Agnostic Language Identification and Generation
Mikael Møller Høgsgaard, Chirag Pabbaraju · 2026
Read Agnostic Language Identification and Generation by Mikael Møller Høgsgaard, Chirag Pabbaraju (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-23258.txt.
Description
Mikael Møller Høgsgaard and Chirag Pabbaraju study language identification without assuming the data fit a known language. Recent work set tight statistical rates for identifying and generating languages. Those results rest on realizability: the input is drawn from some language in a given collection.
Their agnostic setting drops that comfort, so the true source may sit outside the collection. Identification must then pick the closest match and still generate sensibly. Agnostic guarantees matter for messy real text, from dialects to mixed-language posts. Dropping realizability changes the statistics fundamentally, since the best available language may only approximate the true source.
Catalog record
| CREATOR | Mikael Møller Høgsgaard, Chirag Pabbaraju |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-23258