Micro Language Models Enable Instant Responses

Wen Cheng, Tuochao Chen, Karim Helwani, Sriram Srinivasan, Luke Zettlemoyer, Shyamnath Gollakota · 2026

Read Micro Language Models Enable Instant Responses by Wen Cheng, Tuochao Chen, Karim Helwani, Sriram Srinivasan, Luke Zettlemoyer, Shyamnath Gollakota (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-19642.txt.

Description

Cheng and colleagues introduce micro language models, ultra-compact models with 8 million to 30 million parameters that instantly generate the first 4 to 8 words of a grounded response on-device. Watches and glasses lack the power to keep even the smallest 100 million to 1 billion parameter language models running nonstop, while cloud inference adds multi-second delays. Useful generation survives at this extreme scale, with the new models matching several 70 million to 256 million parameter existing models. The authors reframe the cloud model as a continuator rather than a respondent.

Catalog record

CREATORWen Cheng, Tuochao Chen, Karim Helwani, Sriram Srinivasan, Luke Zettlemoyer, Shyamnath Gollakota
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70917
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.19642

Browse the full library · About the Tower of Babel