From Text to DSL: Evaluating Grammar-Based Model Generation Using Open LLMs

Junaid Baber, Nicolas Hili, Didier Schwab, Léo Challier, Cécilia Satrin · 2026

Read From Text to DSL: Evaluating Grammar-Based Model Generation Using Open LLMs by Junaid Baber, Nicolas Hili, Didier Schwab, Léo Challier, Cécilia Satrin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-15865.txt.

Description

Junaid Baber and four colleagues test whether open language models can build models that fit domain specific languages from plain text, using only few-shot prompts with no fine-tuning. They grade 39 models from 0.5B to 32B parameters on syntax, meaning and cross-model link consistency, building both user interface and data models from scratch. Several small models, such as gemma3:12b and mistral:7b-instruct, neared or matched much larger models. The results back smaller open models as a cheap, deployable choice over closed models for grammar-conformant generation.

Catalog record

CREATORJunaid Baber, Nicolas Hili, Didier Schwab, Léo Challier, Cécilia Satrin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30175
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.15865

Browse the full library · About the Tower of Babel