Cycle-Instruct : Fully Seed-Free Instruction Tuning via Dual Self-Training and Cycle Consistency

Zhanming Shen, Hao Chen, Yulei Tang, Shaolin Zhu, Wentao Ye, Xiaomeng Hu, Haobo Wang, Gang Chen, Junbo Zhao · 2025

Read Cycle-Instruct : Fully Seed-Free Instruction Tuning via Dual Self-Training and Cycle Consistency by Zhanming Shen, Hao Chen, Yulei Tang, Shaolin Zhu, Wentao Ye, Xiaomeng Hu, Haobo Wang, Gang Chen, Junbo Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-16100.txt.

Introduction

Shen and eight co-authors tuned language models with no seed examples in this paper from twenty twenty-five. Teaching models to follow instructions usually needs human-written examples or a stronger teacher. Back-translation methods still lean on a starter seed set, importing its biases. Cycle-Instruct removes the seed entirely.

Two models bootstrap each other from raw, unlabeled text. One generates answers, the other generates matching questions. Each learns by rebuilding the original text from its partner's output, a cycle consistency check. Tests covered general instructions, specialist tasks, dialogues, and plain text. The method beat seed-driven back-translation and rivaled strongly supervised training. The authors pitch it as full automation for instruction data.

Catalog record

CREATORZhanming Shen, Hao Chen, Yulei Tang, Shaolin Zhu, Wentao Ye, Xiaomeng Hu, Haobo Wang, Gang Chen, Junbo Zhao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-16100

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Cycle-Instruct : Fully Seed-Free Instruction Tuning via Dual Self-Training and Cycle Consistency Zhanming Shen, Hao Chen, Yulei Tang, Shaolin Zhu, Wentao Ye, Xiaomeng Hu, Haobo Wang, Gang Chen, Junbo Zhao Abstract Instruction tuning is vital for aligning large language models (LLMs) with human intent, but current methods typically rely on costly human-annotated seed data or powerful external teacher models. While instruction back-translation techniques reduce this dependency, they remain fundamentally tethered to an initial seed set, which limits full automation, introduces biases, and can lead to inefficient use of unlabeled corpora. In this paper, we propose Cycle-Instruct , a novel framework that achieves fully seed-free instruction tuning. Inspired by cycle consistency, Cycle-Instruct employs a dual self-training loop where two models—an answer generator and a question generator—are bootstrapped solely from raw, unlabeled text. These models mutually supervise each other by reconstructing original text segments from their counterpart’s generated pseudo-labels, effectively learning from the intrinsic structure of the data without any human-provided seeds. We demonstrate Cycle-Instruct ’s efficacy across four diverse data tracks, including general instruction-following, domain-specific tasks, dialogue logs, and plain text. Our extensive experiments show that Cycle-Instruct not only outperforms seed-driven back-translation baselines but also achieves performance comparable to strongly supervised methods. 1 Introduction Instruction tuning ( Ouyang et al. 2022 ) has emerged as a crucial technique for aligning large language models (LLMs) with human intent, enabling effective generalization across diverse instruction-based tasks ( Ouyang et al. 2022 ; Wei et al. 2021 ; Touvron et al. 2023 ) . However, conventional instruction tuning typically requires extensive human-annotated data ( Köpf et al. ; Conover et al. 2023 ) or relies on powerful external teacher models ( Taori et al. 2023 ; Yin et al. 2023 ) . These dependencies are not only costly and limit scalability ( Wang et al. 2022 ) but are also often inapplicable in certain settings—for example, privacy-preserving scenarios ( Zhang et al. 2024 ) . Recent advances in instruction back-translation ( Li et al. 2023a ; Köksal et al. 2023 ) have sought to mitigate above issues by leveraging unlabeled text. These methods typically train a model on a small seed set of (instruction, answer) pairs to generate candidate instructions for unlabeled documents, which are then filtered to create additional training examples ( Chen et al. 2024 ; Chen et al. 2023 ) . While reducing reliance on extensive human labeling, these back-translation pipelines still critically hinge on an initial, manually-curated seed set. Figure 1: Seed-dependency bottleneck: (1) Costly seed curation; (2) Data wastage & mis-paired; (3) Bias transfer & low diversity. This seed-dependency causes a core challenge: the reliance on seed data inherently limits the full automation, diversity, and data efficiency of instruction tuning . Specifically, as illustrated in Figure 1 , (i) assembling a diverse, high-quality seed set remains labour-intensive and costly; (ii) conditioning generation on a small seed corpus can transfer its stylistic and topical biases to the synthetic pairs, curbing diversity and generalisation; and (iii) the common practice of treating all unlabeled passages as answers can lead to data wastage, as question-formatted segments in raw corpora are often discarded or receive low-quality synthetic instructions. This raises a critical question: How can we unlock the full potential of abundant raw text for instruction tuning, without the bottleneck of seed data, while ensuring high-quality and diverse instruction-following capabilities? To address this challenge, we propose Cycle-Instruct , a novel and fully seed-free framework for instruction tuning that requires no manually written seeds or external teacher models. Inspired by cycle consistency in unsupervised machine translation ( He et al. 2016 ; Lample et al. 2018 ) , Cycle-Instruct employs a dual self-training loop. We begin by automatically partitioning a large unlabeled corpus into potential question passages and potential answer passages , reformatted into instruction-tuning compliant <instruction> and <response> slots. An answer generator produces pseudo-responses for question passages, while a question generator back-translates pseudo-instructions for answer passages. These two models then act as mutual teachers: each learns to reconstruct the original passage by conditioning on the pseudo-label from its counterpart, using the reconstruction error as the training objective. Crucially, because every sample is supervised by its own ground-truth content , the learning signal closely approximates fully supervised training, allowing the models to internalise the true data distribution from the entire unlabeled corpus. We thoroughly evaluate Cycle-Instruct across four diverse tracks. Our results consistently demonstrate that Cycle-Instruct generates coherent and relevant instruction-following data, establishing its potential as a scalable, especially when human-labeled resources are scarce or unavailable. Our main contributions are as follows: • We propose Cycle-Instruct , a fully seed-free instruction-tuning framework that entirely eliminates reliance on human-written seeds and external teacher models, directly addressing the core bottleneck of current back-translation methods. • We introduce a dual self-training loop with cycle consistency as the core mechanism, enabling two models to mutually supervise each other by reconstructing raw passages, yielding a high-quality learning signal akin to fully supervised training from unlabeled text alone. • Through extensive experiments on four diverse data tracks, we demonstrate that Cycle-Instruct achieves performance comparable to strong supervised methods and significantly outperforms seed-driven back-translation baselines, all while requiring zero human annotations. Figure 2: The Overall Framework of our Fully Seed-Free Instruction Tuning Method. 2 Background 2.1 Back-Translation for Instruction Tuning Back-translation (BT) was introduced in NMT to exploit monolingual target data by “translating it back” into the source language ( Sennrich et al. 2016 ) . Recent work ( Li et al. 2023a ; Köksal et al. 2023 ) adapts this idea to instruction tuning. Let \mathcal{S}=\{(q,a)\} be a small seed set of question–answer pairs and \mathcal{D}_{A} an unlabeled corpus of free-form answers. 1. Seed training. Train an inverse model F_{A\!\to\!Q} to predict a question from an answer: \mathcal{L}_{\text{inv}}\;=\;-\!\!\sum_{(q,a)\in\mathcal{S}}\!\log p_{\psi}\bigl(q\,\big|\,a\bigr). (1) 2. Pseudo-pair generation. For each a\in\mathcal{D}_{A} , create a pseudo-question \hat{q}=F_{A\!\to\!Q}(a) and form a synthetic pair (\hat{q},a) . 3. BT fine-tuning. Fine-tune the forward model G_{Q\!\to\!A} on all synthetic pairs via \mathcal{L}_{\text{BT}}=\mathbb{E}_{(\hat{q},a)}\!\bigl[-\log p_{\theta}(a\mid\hat{q})\bigr]. (2) The resulting model G_{Q\!\to\!A} benefits from a much larger, automatically generated corpus, while all notation remains consistent with the cycle-consistency formulation that follows. 2.2 Cycle Consistency Cycle consistency traces its roots to unsupervised machine translation and dual learning, where two models—one translating from language X to Y and the other from Y to X —are trained jointly by enforcing that translating “there and back” recovers the original sentence ( He et al. 2016 ) . Concretely, given mappings G:X\to Y,\quad F:Y\to X, (3) one adds the reconstruction loss \displaystyle\mathcal{L}_{\mathrm{cycle}} \displaystyle=\mathbb{E}_{x\sim\mathcal{D}_{X}}\!\bigl[\ell\bigl(x,\,F(G(x))\bigr)\bigr] (4) \displaystyle+\mathbb{E}_{y\sim\mathcal{D}_{Y}}\!\bigl[\ell\bigl(y,\,G(F(y))\bigr)\bigr], where \ell(\cdot,\cdot) is typically the negative log-likelihood of reconstructing the original sequence. CycleGAN ( Chu et al. 2017 ) applied a similar cycle constraint in image translation by using \ell_{1} pixel losses, illustrating the concept across modalities. In Cycle-Instruct, we reinterpret this mechanism for instruction tuning by viewing questions and answers as two analogous “languages.” Let G_{\!Q\to A}:\mathcal{Q}\to\mathcal{A},\quad F_{\!A\to Q}:\mathcal{A}\to\mathcal{Q} (5) be the question-to-answer and answer-to-question models, respectively. We then minimize \displaystyle\mathcal{L}_{\mathrm{cycle}} \displaystyle=\mathbb{E}_{q\sim\mathcal{Q}}\!\Bigl[\ell\bigl(q,\,F_{A\to Q}\!\bigl(G_{Q\to A}(q)\bigr)\bigr)\Bigr] (6) \displaystyle+\mathbb{E}_{a\sim\mathcal{A}}\!\Bigl[\ell\bigl(a,\,G_{Q\to A}\!\bigl(F_{A\to Q}(a)\bigr)\bigr)\Bigr]. By enforcing F_{A\to Q}(G_{Q\to A}(q))\approx q and G_{Q\to A}(F_{A\to Q}(a))\approx a on unlabeled text, the two models effectively teach one another. This self-supervised reconstruction signal enables the generation of high-quality question–answer pairs directly from raw corpora,

Read the complete text