Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models
Youmi Ma, Sakae Mizuki, Kazuki Fujii, Taishi Nakamura, Masanari Ohi, Hinari Shimada, Taihei Shiotani, Koshiro Saito, Koki Maeda, Kakeru Hattori, Takumi Okamoto, Shigeki Ishida, Rio Yokota, Hiroya Takamura, Naoaki Okazaki · 2025
Read Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models by Youmi Ma, Sakae Mizuki, Kazuki Fujii, Taishi Nakamura, Masanari Ohi, Hinari Shimada, Taihei Shiotani, Koshiro Saito, Koki Maeda, Kakeru Hattori, Takumi Okamoto, Shigeki Ishida, Rio Yokota, Hiroya Takamura, Naoaki Okazaki (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23714.txt.
Introduction
Youmi Ma and co-authors build in 2025 instruction tuning data sets for large language models from human written instructions, paired with model generated answers. Tuning on instructions is key to making models useful for real tasks. Past work had shown that data made wholly by models can work, raising the question of whether human input is still needed.
Models tuned on their data beat those tuned on existing data sets. The method adapts easily to other languages; a Japanese version reached top performance for that language. Analysis suggests tuning in a new language teaches instruction following, though culture specific knowledge stays thin. Data and models are released under open licenses.
Catalog record
| CREATOR | Youmi Ma, Sakae Mizuki, Kazuki Fujii, Taishi Nakamura, Masanari Ohi, Hinari Shimada, Taihei Shiotani, Koshiro Saito, Koki Maeda, Kakeru Hattori, Takumi Okamoto, Shigeki Ishida, Rio Yokota, Hiroya Takamura, Naoaki Okazaki |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-23714
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models Youmi Ma, Sakae Mizuki, Kazuki Fujii, Taishi Nakamura, Masanari Ohi, Hinari Shimada, Taihei Shiotani, Koshiro Saito, Koki Maeda, Kakeru Hattori, Takumi Okamoto, Shigeki Ishida, Rio Yokota, Hiroya Takamura, Naoaki Okazaki Abstract Instruction tuning is crucial for enabling Large Language Models (LLMs) to solve real-world tasks. Prior work has shown the effectiveness of instruction-tuning data synthesized solely from LLMs, raising a fundamental question: Do we still need human-originated signals for instruction tuning? This work answers the question affirmatively: we build state-of-the-art instruction-tuning datasets sourced from human-written instructions, by simply pairing them with LLM-generated responses. LLMs fine-tuned on our datasets consistently outperform those fine-tuned on existing ones. Our data construction approach can be easily adapted to other languages; we build datasets for Japanese and confirm that LLMs tuned with our data reach state-of-the-art performance. Analyses suggest that instruction-tuning in a new language allows LLMs to follow instructions, while the tuned models exhibit a notable lack of culture-specific knowledge in that language. The datasets and fine-tuned models will be publicly available 1 1 1 https://huggingface.co/datasets/tokyotech-llm/lmsys-chat-1m-synth . Our datasets, synthesized with open-weight LLMs, are openly distributed under permissive licenses, allowing for diverse use cases. 1 Introduction Open-weight Large Language Models (LLMs) have seen rapid advancements over the past year. Models like Llama-3.1 ( Grattafiori et al. 2024 ) and Qwen-2.5 ( Qwen et al. 2024 ) now achieve performance comparable to proprietary models such as GPT-4 ( OpenAI et al. 2024 ) and Gemini ( Google et al. 2024b ) . A crucial technology behind this is instruction-tuning ( Wei et al. 2022 ) , a supervised fine-tuning process that equips models with abilities to engage in conversations and generate responses that satisfy human needs. Unfortunately, even for LLMs whose weights are open, data used for instruction tuning are mostly closed. The data, typically consisting of ( instruction , response ) pairs, are keys to democratizing LLM: it remains an open question what kind of instructions are effective in enhancing LLMs’ instruction-following abilities. Xu et al. 2025 proposed to “extract” the instructions directly from the instruction-tuned models by prompting with a pre-query template. Datasets constructed through this process, named Magpie, have demonstrated remarkable effectiveness, being state-of-the-art among all publicly available instruction-tuning datasets. The success of the self-synthesized data raises an important question: Do we still need human-originated signals for instruction tuning? In other words, are LLM-generated instructions truly superior to human-written ones? This work seeks an answer to the above question. To this end, we synthesize datasets from human-written instructions, as in Figure 1 . For instructions, we extract user utterances from LMSYS-Chat-1M ( Zheng et al. 2024 ) , a dataset composed of real-world conversations between humans and chatbots. For responses, we synthesize assistant utterances using open-weight LLMs, namely Llama-3.1 ( Grattafiori et al. 2024 ) and Gemma-2 ( Google et al. 2024a ) utilized for synthesizing Magpie ( Xu et al. 2025 ) . We fine-tuned four leading base LLMs on our datasets and compared their performance against those fine-tuned on Magpie. Three out of four models demonstrated superiority with our dataset, while the remaining model performed comparably. The results highlight the superiority of human-written instructions compared to LLM-generated ones. Pairing human-written instructions with LLM-generated responses thus leverages the strengths of both humans and LLMs, resulting in state-of-the-art datasets for instruction tuning. Figure 1 : Overview of how Magpie ( Xu et al. 2025 ) , the existing state-of-the-art dataset and our dataset are collected. Starting from instructions written by humans, we synthesize responses by open-weight LLMs. Dataset Instruction Response Language # Instances Dolly ( Conover et al. 2023 ) Human Human en. 15,011 OpenAssistant2 ( Köpf et al. 2023 ) Human Human en. 128,575 OpenOrca ( Lian et al. 2023 ) Human GPT-3.5, GPT-4 en. 4,233,923 LMSYS-Chat-1M ( Zheng et al. 2024 ) Human 25 LLMs en. 1,000,000 WildChat ( Zhao et al. 2024 ) Human GPT-3.5, GPT-4 en. 529,428 Alpaca ( Taori et al. 2023 ) GPT-3.5 GPT-3.5 en. 52,002 Baize ( Xu et al. 2023 ) GPT-3.5 GPT-3.5 en. 210,311 UltraChat ( Ding et al. 2023 ) GPT-3.5 GPT-3.5 en., zh. 1,468,352 Evol-Instruct ( Xu et al. 2024 ) GPT-3.5 GPT-3.5 en. 70,000 Magpie-Ultra-v0.1 ( Xu et al. 2025 ) Llama-3.1-405B-Instruct Llama-3.1-405B-Instruct en. 49,052 Magpie-Pro-Filtered ( Xu et al. 2025 ) Llama-3.1-70B-Instruct Llama-3.1-70B-Instruct en. 300,000 Ja-Self-Instruct ( Sun et al. 2024 ) GPT-4 GPT-4 ja. 52,002 This work Proposed-Llama-3.1-En Human Llama-3.1-405B-Instruct en. 453,055 Proposed-Gemma-2-En Human Gemma-2-27B-IT en. 453,671 Proposed-Llama-3.1-Ja Human Llama-3.1-405B-Instruct ja. 453,802 Proposed-Gemma-2-Ja Human Gemma-2-27B-IT ja. 451,450 Table 1 : Statistics of existing and our datasets. The column Language shows the major language in each dataset. We are the first to publish datasets with human-written instructions and responses synthesized by open-weight LLMs. A practical issue of our proposed method could be the difficulty of collecting large-scale human-written instructions, since existing resources are limited to English. To this end, we verify if our method is still effective in non-English languages based on machine translation. Specifically, we translated English human-written instructions into the target language and then prompted LLMs to synthesize responses. We chose Japanese as the target language and synthesized two datasets. Fine-tuning with the datasets significantly boosted the instruction-following abilities of all five base LLMs with different levels of Japanese proficiency. In total, this work built four datasets, with statistics shown in Table 1 . We further analyzed how the instruction-following abilities transfer between languages. From the analyses, LLMs trained with Japanese datasets can solve surface-level editing (creative writing and roleplay) and region-agnostic knowledge domain (STEM) but are worse at instructions that require culture-specific knowledge. In short, the contributions of this work are: (1) We construct and publish state-of-the-art instruction-tuning datasets with permissive licenses, allowing for a wide range of use cases. Fine-tuning LLMs with our datasets yields the best performance among all publicly available datasets. (2) We highlight the importance of human-written instructions collected from real-world human-chatbot interactions. By leveraging both human- and LLM-originated signals we obtain datasets with quality higher than those solely synthesized from LLMs. (3) We demonstrate that instruction-tuning in a new language equips LLMs with abilities to follow instructions, but it does not improve culture-specific knowledge. 2 Synthesized Data Generation Starting from human-written instructions, we synthesize responses with open-weight LLMs to construct instruction-tuning datasets. Here we note the dataset as \mathcal{D}=\{(I_{1},R_{1}),(I_{2},R_{2}),\cdots,(I_{n},R_{n})\} , where I_{k} and R_{k} (k\in\{1,2,\cdots,n\}) represent an instruction and its corresponding response. Instructions. We collected instructions \mathcal{I}=\{I_{1},I_{2},\cdots,I_{n}\} from LMSYS-Chat-1M ( Zheng et al. 2024 ) 2 2 2 https://huggingface.co/datasets/lmsys/lmsys-chat-1m , a dataset of one million real-world conversation records between human and LLMs. The dataset was selected as our foundation as it covers a wide range of topics, with a permissive license for both academic and commercial use. Notably, some of the conversations in LMSYS-Chat-1M are flagged as toxic; these conversations were excluded as our focus here is to improve LLMs’ instruction-following abilities, i.e., how accurately LLMs can fulfill user requests. After filtering out toxic conversations, we obtained 732,392 instances, from which the first user utterance is selected as the instruction. Further removal of duplications left us with 453,889 instructions directly adopted for synthesizing responses. Responses. Instances in LMSYS-Chat-1M include model