Small Open Models Achieve Near-Parity with Large Models in Low-Resource Literary Translation at a Fraction of the Cost

Mihai Nadăş [email protected], Laura Dioşan [email protected], Andreea Tomescu [email protected], Andrei Pişcoran [email protected] · 2025

Read Small Open Models Achieve Near-Parity with Large Models in Low-Resource Literary Translation at a Fraction of the Cost by Mihai Nadăş [email protected], Laura Dioşan [email protected], Andreea Tomescu [email protected], Andrei Pişcoran [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-07829.txt.

Introduction

Literary translation between English and Romanian gets a full pipeline in a 2025 paper by Mihai Nadas and 3 co-authors. The team builds the TinyFabulist Translation Framework, a system for making data, training models, and scoring results. Romanian has few digital resources, so small open models often translate its literature poorly.

Starting from a large set of synthetic English fables, the pipeline uses a strong large model to create fifteen thousand Romanian reference translations. A twelve-billion-parameter open model is then tuned in two stages, first for story style and then for efficient use. Judging covers five dimensions of quality. The work shows how low-resource languages can gain literary translation tools without closed commercial models.

Catalog record

CREATORMihai Nadăş [email protected], Laura Dioşan [email protected], Andreea Tomescu [email protected], Andrei Pişcoran [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-07829

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Small Open Models Achieve Near-Parity with Large Models in Low-Resource Literary Translation at a Fraction of the Cost Mihai Nadăş [email protected], Laura Dioşan [email protected], Andreea Tomescu [email protected], Andrei Pişcoran [email protected] Abstract Literary translation has recently gained attention as a distinct and complex task in machine translation research. However, the translation by small open models remains an open problem. We contribute to this ongoing research by introducing TinyFabulist Translation Framework (TF2) , a unified framework for dataset creation, fine-tuning, and evaluation in English → \rightarrow Romanian literary translations, centred on the creation and open release of both a compact, fine-tuned language model ( TF2-12B ) and large-scale synthetic parallel datasets ( DS-TF2-EN-RO-3M and DS-TF2-EN-RO-15K ). Building on DS-TF1-EN-3M (TF1), the largest collection of synthetic English fables to date, we address the need for rich, high-quality literary datasets in low-resource languages such as Romanian. Our pipeline first generates 15k high-quality Romanian references from the TF1 pool using a high-performing LLM. We then apply a two-stage fine-tuning process to a 12B-parameter open-weight model: (i) instruction tuning to capture genre-specific narrative style, and (ii) adapter compression for efficient deployment. Evaluation combines corpus-level BLEU and a five-dimension LLM-based rubric (accuracy, fluency, coherence, style, cultural adaptation) to provide a nuanced assessment of translation quality. Results show that our fine-tuned model achieves fluency and adequacy competitive with top-performing large, proprietary models, while being open, accessible, and significantly more cost-effective. Alongside the finetuned model, and both datasets, we publicly release all scripts and evaluation prompts. TF2 thus provides an end-to-end, reproducible pipeline for research on cost-efficient translation, cross-lingual narrative generation, and the broad adoption of open models for culturally significant literary content in low-resource settings. 1 Introduction Translating literature into low-resource languages poses unique challenges, given both the scarcity of parallel data and the need to faithfully preserve stylistic nuance and cultural context. Romanian---a Latin language spoken by over 24 million people---remains notably underserved in the realm of high-quality literary translation resources. Existing machine translation (MT) benchmarks, such as WMT 1 1 1 WMT stands for the Conference on Machine Translation (formerly “Workshop on Machine Translation”). The WMT organizes annual shared tasks for benchmarking machine translation systems, with its “news translation” task being the most widely used standard evaluation for translation of contemporary news articles. See: https://www.statmt.org/wmt23/ . , predominantly focus on informational or news text, thus providing little guidance for creative, narrative-driven translation tasks. The rapid advancement of large language models (LLMs) has, however, enabled synthetic text generation at unprecedented scale, opening up new opportunities for augmenting MT datasets [ 14 , 19 ] . Notably, projects like TinyStories [ 4 ] have shown that even compact models can produce coherent narratives when trained on thoughtfully curated synthetic corpora. While these advances have transformed data generation and model pretraining, much of the prior work has centered on monolingual story generation in English [ 6 , 4 ] or improvements to translation via back-translation [ 22 ] , rather than the systematic creation of large-scale, bilingual literary datasets. As a result, there remain open questions about how both proprietary and open-source LLMs can be harnessed to create and evaluate high-quality literary translations in genuinely low-resource settings. Significant progress has recently been made toward building massive bilingual and multilingual resources—for example, CLIRMatrix [ 26 ] compiles parallel data across 139 languages for cross-lingual retrieval, and mT5 [ 33 ] pretrains a text-to-text transformer on a vast array of language pairs and domains. However, such resources are primarily oriented toward general-domain or retrieval-centric MT, with limited representation of literary content. Even in language pairs like English–Romanian, coverage is often restricted to news, web, or miscellaneous domains. To date, there is no open, large-scale, high-quality English–Romanian parallel corpus specifically curated for creative narratives or fables. TinyFabulist Translation Framework (TF2) addresses this gap. DS-TF2-EN-RO-3M is a 3-million-example synthetic dataset of English–Romanian parallel fables, accompanied by fine-tuned open-source translation models and a literary translation benchmark. It extends our earlier TinyFabulist-TF1 corpus of English-only fables [ 18 ] into a bilingual setting, explicitly targeting creative narrative translation and cultural adaptation. By combining parameter-efficient adaptation with narrative-aware evaluation, TF2 provides both a large-scale resource and a practical benchmark for studying literary MT in a genuinely low-resource language pair. While recent multilingual models such as NLLB-200 [ 27 ] and EuroLLM [ 16 ] have demonstrated impressive performance, these systems are generally optimized for breadth and general-domain translation. Their large parameter counts also limit practical deployment in cost-constrained or on-device scenarios. Moreover, our experiments show that such models, despite their coverage, often fall short in producing the nuanced style, coherence, and cultural adaptation required for professional literary translation (see Section 4 ). Despite recent advances in both synthetic data creation and multilingual modeling, there remains a critical lack of resources and benchmarks for literary translation in low-resource settings—especially for genres demanding nuanced handling of narrative style and cultural meaning. Existing open and proprietary translation models are typically not tuned for creative content, and English–Romanian parallel datasets remain scarce, limited in scope, or non-public. Prior benchmarks overwhelmingly favor news or technical content, leaving literary and didactic narratives comparatively neglected. Building on the English-only fable generation efforts of DS-TF1-EN-3M [ 18 ] , we systematically extend to cross-lingual translation, focusing on moral fables—short stories that require not only semantic accuracy but also fluency, narrative coherence, stylistic faithfulness, and nuanced cultural adaptation. Generating high-quality parallel data under resource constraints is especially challenging for such genres, motivating our use of LLMs for both dataset creation and the development of an automated, multi-dimensional evaluation framework. To address the challenges outlined above, our work makes the following key contributions: • Narrative-Aware Evaluation Protocol. We show how to bootstrap high-quality literary data with instruction-tuned LLMs, combining corpus-level BLEU [ 25 ] with a five-dimension LLM rubric for reproducible, fine-grained assessment. • Two new openly licensed corpora for literary MT. (i) DS-TF2-EN-RO-15K , a 15 thousand English \rightarrow Romanian parallel set of moral fables, and (ii) DS-TF2-EN-RO-3M , a 3 million corpus automatically translated with our fine-tuned models. 2 2 2 https://huggingface.co/datasets/klusai/tf2-en-ro-15k , https://huggingface.co/datasets/klusai/tf2-en-ro-3m • Three open, fine-tuned translation models. We release three open, fine-tuned translation models— tf2-1b , tf2-4b , and tf2-12b ---LoRA-adapted checkpoints that substantially close the quality gap to proprietary systems while remaining efficient enough to run locally on commodity GPUs. 3 3 3 https://huggingface.co/klusai/tf2-1b , https://huggingface.co/klusai/tf2-4b , https://huggingface.co/klusai/tf2-12b • Fully transparent artifacts. All data, code, prompts, and evaluation scripts are published under permissive licenses to foster reproducibility and further research in low-resource literary NLP. By focusing on literary translation in a genuinely low-resource language, TF2 fills a gap left by existing datasets and MT models, demonstrating that high-quality, culturally adapted translation is feasible within tight cost and resource constraints. Building on this foundation, the remainder of the paper is organized to systematically explore the practical and methodological questions that arise in the creation and evaluation of large-scale literary translation resources for low-resource languages. We motivate our experimental design, comparative analyses, and evaluation framework in the context of both real-world constraints and the broader landscape of open and proprietary language technologies. 1.1 Research Questions In formulating our research questions, we recognize that the development of large-scale translation datasets and models is

Read the complete text