Improving Low-Resource Machine Translation via Round-Trip Reinforcement Learning

Ahmed Attia, Alham Fikri · 2026

Read Improving Low-Resource Machine Translation via Round-Trip Reinforcement Learning by Ahmed Attia, Alham Fikri (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12535.txt.

Introduction

Ahmed Attia and Alham Fikri work on machine translation for low resource languages. These are languages with little paired text to train on. More paired data is now being gathered from such communities. Yet many ideas for better low resource translation remain untested, and the pair set out to try one.

Their method fine tunes a translation model with reinforcement learning, with no human labels. The trick is round trip bootstrapping: translate out and back, and learn from how well the trip preserves meaning. They build on the No Language Left Behind family of models. Those models already cover many low resource languages. A self supervised route like this could lift quality where paired data stays scarce.

Catalog record

CREATORAhmed Attia, Alham Fikri
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-12535

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Improving Low-Resource Machine Translation via Round-Trip Reinforcement Learning Ahmed Attia, Alham Fikri Abstract Low-resource machine translation (MT) has gained increasing attention as parallel data from low-resource language communities is collected, but many potential methods for improving low-resource MT remain unexplored. We investigate a self-supervised reinforcement-learning-based fine-tuning for translation in low-resource settings using round-trip bootstrapping with the No Language Left Behind (NLLB) family of models. Our approach translates English into a target low-resource language and then back into English, using a combination of chrF++ and BLEU as the reward function on the reconstructed English sentences. Using the NLLB-MD dataset, we evaluate both the 600M and 1.3B parameter NLLB models and observe consistent improvements for the following languages: Central Aymara, Friulian, Wolof and Russian. Qualitative inspection of translation outputs indicates increased fluency and semantic fidelity. We argue that our method can further benefit from scale, enabling models to increasingly leverage their pretrained knowledge and continue self-improving. Figure 1: Overview Of Our Low-Resource Machine Translation With Round-Trip Reinforcement Learning Method. 1 Introduction Low-resource machine translation remains one of the most persistent challenges in natural language processing. Modern neural translation systems achieve impressive performance for high-resource languages ( 29 ; 3 ; 12 ) , where millions of parallel sentence pairs are available for supervised training. However, thousands of languages lack parallel corpora of sufficient scale ( 16 ; 10 ) , leading to models that perform poorly or fail to generalize outside narrow training domains. As a result, speakers of low-resource languages continue to face limited access to digital information, reduced participation in multilingual technologies, and diminished inclusion in global communication systems. To address these limitations, we introduce a framework to improve MT system without any parallel corpus that integrates reinforcement learning (RL) via round-trip bootstrapping. Our system operates in a self-supervised manner by translating from English into a low-resource language and then back to English. In this setup, the intermediate low-resource translation is treated as a latent step required to reconstruct the original English sentence. We use RL to optimize this process, rewarding the model based on lexical and semantic metrics such as BLEU and chrF++. This allows the model to improve translation quality in the low-resource language without requiring any human-annotated parallel data. The necessity for this approach arises from the inherent flaws in standard machine translation training. Most systems rely on maximum likelihood estimation (MLE) ( 7 ) , which teaches models to predict the next token given ground-truth history. While effective, MLE suffers from exposure bias ( 1 ; 30 ; 11 ) , arising from discrepancies between training and inference conditions, and an objective mismatch ( 18 ) , where the training loss does not directly optimize evaluation metrics like chrF++ ( 22 ) . In low-resource settings, these issues are amplified because models have fewer "gold" examples to prevent them from drifting during inference. Furthermore, while back-translation ( 26 ) is currently the dominant method for low-resource MT, its effectiveness is fundamentally constrained by the quality of the reverse model. When initial translations are weak, the synthetic data produced by back-translation can reinforce existing model errors. This creates a feedback loop that limits gains, particularly for morphologically rich or typologically distant languages ( 19 ) . Standard self-supervised approaches lack a mechanism to prioritize high-quality synthetic examples or directly optimize task-specific translation metrics. RL ( 28 ) provides a natural solution to these bottlenecks. By treating translation as a sequential decision-making process, RL allows models to directly optimize non-differentiable metrics and exposes them to their own predictions during training. While foundational work introduced policy gradient methods ( 23 ; 32 ) to reduce exposure bias, and recent studies have applied RL to large-scale models ( 5 ) , the application of RL to bridge the specific gap in round-trip low-resource translation remains relatively under-explored. In this work, we demonstrate that our RL-based bootstrapping method significantly improves both the adequacy and fluency of translations. Our main contributions are: • We propose a novel RL-based framework for round-trip translation that operates entirely without parallel data. • We show that optimizing for reconstruction rewards significantly improves surface-level well-formedness and semantic adequacy in low-resource settings. • We provide an extensive analysis across several languages. 2 Related Work 2.1 Improving NMT with Monolingual Data and Roundtrip Translations Back-translation has been a foundational technique for leveraging monolingual data in neural machine translation. 26 introduced back-translation and demonstrated that target-side monolingual data can substantially improve translation quality by generating synthetic parallel data; mixing these synthetic examples with genuine bitext yields strong performance gains and supports effective domain adaptation. 17 showed that unsupervised machine translation is feasible by combining denoising autoencoding, back-translation, and a cycle-consistency objective that enforces accurate reconstruction of source sentences after a forward–backward translation loop. This line of work treats forward and backward translation models as cooperative components whose outputs must remain mutually consistent. 13 is an early work that explicitly uses round-trip translation as a quality check for “target language inexpert” users: texts are translated into a foreign language and immediately back so users can spot major errors without knowing the target language. 4 extend this idea to paraphrasing, systematically studying “multilingual whispers,” where sentences are translated through one or more pivot languages and back to generate diverse paraphrases, showing that round-trip translation can serve not only for quality control but also as an effective paraphrase generation method. 2.2 Reinforcement Learning and GRPO for NMT 32 provided a practical recipe for RL in NMT, showing that multinomial sampling, reward shaping, and objective mixing with MLE lead to consistent gains. 5 proposed MT-R1-Zero, applying R1-Zero–style RL to MT without supervised warm-starts. Their rule-metric mixed reward improved lexical and semantic quality via GRPO 27 , suggesting that pure RL can rival SFT for LLM-based translation. GRPO extends Proximal Policy Optimization (PPO) 25 and removes the value critic. 3 Method We propose a reinforcement learning framework that leverages round-trip translation consistency to fine-tune pretrained multilingual language models for low-resource machine translation. Our approach combines Group Relative Policy Optimization (GRPO) with back-translation-based rewards, enabling effective adaptation without parallel data augmentation. Preliminary For each input x , GRPO samples a group of G candidate outputs \{y_{1},\dots,y_{G}\}\sim\pi_{\theta_{\text{old}}}(y\mid x) from the current policy. The update maximizes \displaystyle\mathcal{J}_{\text{GRPO}}(\theta) \displaystyle=\mathbb{E}_{q\sim P(Q),\,\{o_{i}\}_{i=1}^{G}\sim\pi_{\theta_{\text{old}}}(O\mid q)}\Bigg[ (1) \displaystyle\frac{1}{G}\sum_{i=1}^{G}\frac{1}{|o_{i}|}\sum_{t=1}^{|o_{i}|}\min\!\Big(r_{i,t}(\theta)\,\hat{A}_{i,t}, \displaystyle\operatorname{clip}\big(r_{i,t}(\theta),1-\varepsilon,1+\varepsilon\big)\,\hat{A}_{i,t}\Big) \displaystyle-\beta\,D_{\mathrm{KL}}\!\big[\pi_{\theta}\|\pi_{\mathrm{ref}}\big]\Bigg], \displaystyle\text{where }r_{i,t}(\theta)=\frac{\pi_{\theta}(o_{i,t}\mid q,o_{i,<t})}{\pi_{\theta_{\text{old}}}(o_{i,t}\mid q,o_{i,<t})}. GRPO differs from PPO in how it defines the advantages A_{i} . Instead of subtracting a critic prediction, GRPO normalizes rewards within the sampled group. Let R_{i}=R(x,y_{i}) ; then A_{i}=\frac{R_{i}-\operatorname{mean}(\{R_{1},\dots,R_{G}\})}{\operatorname{std}(\{R_{1},\dots,R_{G}\})}. (2) When all or none of the candidates solve the problem, every A_{i} is zero and the policy gradient vanishes except for the KL term. 3.1 Problem Formulation We consider the setting where no parallel corpus is available for the source and target language pair. Instead, we assume access to a pretrained multilingual neural machine translation (NMT) model \pi_{\theta} and monolingual corpora in the source language (English). Our goal is to improve the translation quality in both directions for this low-resource pair without relying on any parallel data for training. We formulate this adaptation problem as a reinforcement learning problem. At each step, the model samples translation candidates and receives rewards that reflect translation quality, defined purely from intrinsic signals.

Read the complete text