Agnostic Language Identification and Generation

Mikael Møller Høgsgaard, Chirag Pabbaraju · 2026

Read Agnostic Language Identification and Generation by Mikael Møller Høgsgaard, Chirag Pabbaraju (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-23258.txt.

Description

Mikael Møller Høgsgaard and Chirag Pabbaraju study language identification without assuming the data fit a known language. Recent work set tight statistical rates for identifying and generating languages. Those results rest on realizability: the input is drawn from some language in a given collection.

Their agnostic setting drops that comfort, so the true source may sit outside the collection. Identification must then pick the closest match and still generate sensibly. Agnostic guarantees matter for messy real text, from dialects to mixed-language posts. Dropping realizability changes the statistics fundamentally, since the best available language may only approximate the true source.

Catalog record

CREATORMikael Møller Høgsgaard, Chirag Pabbaraju
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-23258

Browse the full library · About the Tower of Babel