Source-Grounded Semantic Reinforcement Learning for Low-Resource Target-Language Generation

Zeli Su 1,2 Ziyin Zhang 3 Zewei Pan 3 Zhou Liu 4 Dingcheng Huang 5 Dehan Li 6 Zhankai Xu 2 Longfei Zheng 2 Xiaolu Zhang 2 Jun Zhou 2,† Wentao Zhang 4,† 1 Minzu University of China 2 Ant Group 3 Shanghai Jiao Tong University 4 Peking University 5 Harbin Institute of Technology 6 South China University of Technology † Corresponding authors · 2026

Read Source-Grounded Semantic Reinforcement Learning for Low-Resource Target-Language Generation by Zeli Su 1,2 Ziyin Zhang 3 Zewei Pan 3 Zhou Liu 4 Dingcheng Huang 5 Dehan Li 6 Zhankai Xu 2 Longfei Zheng 2 Xiaolu Zhang 2 Jun Zhou 2,† Wentao Zhang 4,† 1 Minzu University of China 2 Ant Group 3 Shanghai Jiao Tong University 4 Peking University 5 Harbin Institute of Technology 6 South China University of Technology † Corresponding authors (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29502.txt.

Introduction

From plentiful monolingual text in a high-resource language, Zeli Su and ten colleagues train low-resource generation that ordinary supervised tuning cannot exploit directly. Their SG-SRL method runs reference-free reinforcement learning on source-language data. A cross-lingual reranker scores whether target-language output matches the source meaning. A short recovery stage on a small parallel set then restores fluency and brevity after the reward encourages wordiness. Chinese-to-Thai tests show stronger semantic grounding and factual coverage than cold-start tuning.

Catalog record

CREATORZeli Su 1,2 Ziyin Zhang 3 Zewei Pan 3 Zhou Liu 4 Dingcheng Huang 5 Dehan Li 6 Zhankai Xu 2 Longfei Zheng 2 Xiaolu Zhang 2 Jun Zhou 2,† Wentao Zhang 4,† 1 Minzu University of China 2 Ant Group 3 Shanghai Jiao Tong University 4 Peking University 5 Harbin Institute of Technology 6 South China University of Technology † Corresponding authors
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39945
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.29502

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Source-Grounded Semantic Reinforcement Learning for Low-Resource Target-Language Generation Zeli Su 1,2 Ziyin Zhang 3 Zewei Pan 3 Zhou Liu 4 Dingcheng Huang 5 Dehan Li 6 Zhankai Xu 2 Longfei Zheng 2 Xiaolu Zhang 2 Jun Zhou 2,† Wentao Zhang 4,† 1 Minzu University of China 2 Ant Group 3 Shanghai Jiao Tong University 4 Peking University 5 Harbin Institute of Technology 6 South China University of Technology † Corresponding authors Abstract Low-resource target-language generation is often limited by scarce parallel data, while high-resource source-language monolingual data is abundant but difficult to use with standard supervised fine-tuning. We propose Source-Grounded Semantic Reinforcement Learning (SG-SRL) , a resource-utilization framework that converts source-language monolingual data into cross-lingual semantic supervision for target-language generation. SG-SRL performs reference-free reinforcement learning (RL) on source-language data using a cross-lingual semantic reward model, instantiated by a cross-lingual reranker that scores the semantic relevance between the source input and the target-language generation. While this induces severe verbosity-based reward hacking, a lightweight recovery stage using a small parallel corpus restores fluency, conciseness, and task format while preserving the semantic gains. Experiments on Chinese-to-Thai generation show that SG-SRL improves semantic grounding and factual coverage over cold-start SFT. Additional analyses on long-form transfer and Tibetan embedding-based rewards clarify the generalization behavior of SG-SRL and show that an encoder-based semantic reward can substitute for an LLM-based reranker in a realistic low-resource language setting. 1 Introduction Figure 1: Reranker-based source-grounded semantic reward. A source-language input and a generated target-language output are treated as a cross-lingual query–candidate pair. The reranker estimates their semantic match and provides a scalar reward for RL without requiring a target-language reference. Large language models (LLMs) have achieved strong general-purpose generation abilities through large-scale pretraining and post-training ( 7 ; 6 ; 22 ) , but their performance remains highly uneven across languages. High-resource languages benefit from abundant pretraining text, instruction data, and task-specific supervision, whereas low-resource languages often suffer from unstable generation, hallucinated content, poor factual grounding, and weak task adaptation ( 18 ; 14 ) . Continual pretraining on target-language corpora followed by supervised fine-tuning (SFT) on downstream tasks has been employed for extending LLMs to new languages ( 20 ; 9 ) , but it still relies on substantial target-language text or target-language task supervision, which is often unavailable for genuinely low-resource languages. This data bottleneck becomes even more severe in cross-lingual generation. For many emerging or weakly supported target languages, high-quality parallel data is expensive to collect, and task-specific target-language annotations are even scarcer. In contrast, high-resource source-language data, such as Chinese or English news articles, is often abundant and reliable. Standard SFT cannot directly use such source-language monolingual data, because it requires a target-language reference for each training instance. This leads to the central question of this work: Can abundant source-language monolingual data be converted into useful supervision for low-resource target-language generation, without requiring target-language references during training? To this end, we need a supervision signal that can compare a source-language input with a target-language generation without a gold target-language reference. Multilingual rerankers provide a practical approximation: they score the relevance between a query and a candidate text ( 11 ) , and recent LLM-based variants can distinguish semantically aligned cross-lingual pairs from unrelated ones ( 17 ; 24 ; 25 ) . As illustrated in Figure 1 , we treat the source input as the query and the generated target-language output as the candidate, and use a reranker as a source-grounded semantic reward. This converts source-language monolingual data into scalable RL supervision without target-language references. Empirically, direct optimization of this reward exposes a semantic–form trade-off. The relevance-oriented reward encourages the policy to generate longer outputs that cover more source-side concepts, leading to verbosity-based reward hacking : the intermediate RL checkpoint improves semantic coverage but becomes verbose, poorly formatted, and less fluent ( 1 ) . Stronger length or format constraints can reduce verbosity, but may also suppress useful source-to-target semantic learning. We therefore treat source-grounded semantic RL as semantic mid-training rather than final optimization. Based on this insight, we propose Source-Grounded Semantic Reinforcement Learning (SG-SRL) , a train–reinforce–recover framework for low-resource target-language generation. SG-SRL first learns target-language form from a small parallel corpus, then reinforces source-grounded semantics on source-language monolingual data, and finally reuses the parallel corpus to recover fluent and concise target-language generation. On Chinese-to-Thai generation, starting from SmolLM3-3B 3 , a model with weak Thai support, SG-SRL substantially improves over cold-start SFT, showing that semantic RL as mid-training can strengthen new-language semantic alignment. We further examine a Tibetan setting, where a strong LLM-based reranker is not available for the target language. By training an encoder-based Tibetan–Chinese embedding scorer, we demonstrate that SG-SRL can generalize to a realistic low-resource language scenario with a different form of relevance reward. Our contributions are summarized as follows: • We propose SG-SRL , a train–reinforce–recover framework that converts source-language monolingual data into semantic supervision to enable reference-free RL in the low-resource target-language generation setting. • We instantiate SG-SRL with a multilingual reranker reward, and identify a semantic–form trade-off: direct optimization induces verbosity-based reward hacking, but the intermediate checkpoint can still learn useful cross-lingual semantic alignment. • We show on Chinese-to-Thai generation with SmolLM3-3B that semantic RL as mid-training improves low-resource target-language generation over cold-start SFT, and we further verify language generalization in a Tibetan setting where an encoder-based embedding reward replaces the LLM-based reranker. 2 Related Work Figure 2: Overview of SG-SRL. The framework uses a small parallel corpus \mathcal{D}_{\text{para}} for target-form initialization and target-form recovery, and a large source-language monolingual corpus \mathcal{D}_{\text{src}} for source-grounded semantic RL. The RL stage uses a multilingual reranker as a weak semantic reward model, with language, length, and repetition safeguards. A final recovery stage reuses \mathcal{D}_{\text{para}} to restore fluency, conciseness, and task format. Low-resource language expansion. Expanding language models to low-resource languages has commonly been approached through multilingual pretraining ( 5 ; 21 ) , continued pretraining ( 20 ; 9 ) , or supervised fine-tuning on target-language data. However, these approaches still depend on the availability of target-language text or task-specific supervision. In genuinely low-resource settings, both pretraining corpora and high-quality downstream parallel data may be limited, motivating methods that can exploit abundant source-language data without requiring target-language references for every instance. Semantic rewards for low-resource language learning. Recent work has explored reinforcement learning with semantic rewards as an alternative to token-level likelihood optimization for low-resource language expansion. 16 propose using embedding-level semantic rewards with GRPO to improve low-resource language capabilities while reducing alignment tax, showing that semantic-space optimization can preserve general capabilities better than conventional SFT. Our work shares the same motivation, and further extends the horizon to a reference-free, cross-lingual generation setting. Cross-lingual semantic matching and LLM-based reranking. Dense multilingual representations and cross-lingual retrieval models provide a basis for measuring semantic correspondence across languages ( 8 ; 4 ) . Recent LLM-based rerankers further formulate relevance estimation as an instruction-following judgment over a query–candidate pair, often deriving a score from the model’s preference for an affirmative answer ( 17 ; 24 ; 25 ) . We adopt this formulation by treating the source-language input as the query and the generated target-language output as the candidate. Unlike standard retrieval, however,

Read the complete text