Raising Bars, Not Parameters: LilMoo Compact Language Model for Hindi

Project Polyglot, Leads: Shiza Fatimah, Nicholas Kluge Corrêa, Core Team: Shiza Fatimah, Sophia Falk, Aniket Sen, Nicholas Kluge Corrêa, Other Contributors: Lucie Flek, Florian Mai Affiliations: · 2026

Read Raising Bars, Not Parameters: LilMoo Compact Language Model for Hindi by Project Polyglot, Leads: Shiza Fatimah, Nicholas Kluge Corrêa, Core Team: Shiza Fatimah, Sophia Falk, Aniket Sen, Nicholas Kluge Corrêa, Other Contributors: Lucie Flek, Florian Mai Affiliations: (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03508.txt.

Description

Project Polyglot, led by Shiza Fatimah and Nicholas Kluge Corrêa, trains a Hindi language model from zero instead of borrowing one. Giant multilingual models leave low-resource languages behind, and earlier Hindi models quietly extended those opaque foundations. Their 2026 model, LilMoo, is a compact model of well under a billion parameters, built through a fully transparent and reproducible pipeline designed for limited computing budgets.

The training corpus, GigaLekh, was filtered by rules and by a language model acting as judge, then strengthened with curated English data. Recipe experiments explore how small models learn best under such constraints. Across broad evaluation suites, LilMoo consistently beats similarly sized multilingual rivals from the Qwen families. The team argues that careful language-specific pretraining can rival giant multilingual models at this scale.

Catalog record

CREATORProject Polyglot, Leads: Shiza Fatimah, Nicholas Kluge Corrêa, Core Team: Shiza Fatimah, Sophia Falk, Aniket Sen, Nicholas Kluge Corrêa, Other Contributors: Lucie Flek, Florian Mai Affiliations:
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81792
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.03508

Browse the full library · About the Tower of Babel