Prompt-MII: Meta-Learning Instruction Induction for LLMs
Emily Xiao, Yixiao Zeng, Ada Chen, Chin-Jou Li, Amanda Bertsch, Graham Neubig, Hugging Face Collection · 2025
Read Prompt-MII: Meta-Learning Instruction Induction for LLMs by Emily Xiao, Yixiao Zeng, Ada Chen, Chin-Jou Li, Amanda Bertsch, Graham Neubig, Hugging Face Collection (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-16932.txt.
Introduction
Emily Xiao and five co-authors shrink the cost of teaching chatbots in a 2025 paper. Showing a model many examples works, but long prompts cost money. Their goal is replacing examples with one compact instruction. The instruction should still guide the model just as well.
Their framework, Prompt-MII, learns to write such instructions for new datasets. It trains with feedback scores across thousands of classification datasets. At test time it produces a short prompt on the fly. Quality rose several points on ninety unseen tasks. At the same time, token use fell by a large factor. The method matters wherever models must adapt quickly and cheaply.
Catalog record
| CREATOR | Emily Xiao, Yixiao Zeng, Ada Chen, Chin-Jou Li, Amanda Bertsch, Graham Neubig, Hugging Face Collection |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-16932
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Prompt-MII: Meta-Learning Instruction Induction for LLMs Emily Xiao, Yixiao Zeng, Ada Chen, Chin-Jou Li, Amanda Bertsch, Graham Neubig, Hugging Face Collection Abstract A popular method to adapt large language models (LLMs) to new tasks is in-context learning (ICL), which is effective but incurs high inference costs as context length grows. An alternative approach is to perform instruction induction : taking training examples and reducing them to a compact but descriptive prompt that can achieve performance comparable to ICL over the full training set. We propose Prompt-MII , a reinforcement learning (RL) based framework to meta-learn an instruction induction model that can generate compact instructions on the fly for an arbitrary new dataset. We train on over 3,000 diverse classification datasets from the HuggingFace hub, and evaluate on 90 unseen tasks. Prompt-MII improves downstream model quality by 4-9 F1 points (10-20% relative), matching ICL performance while requiring 3-13x fewer tokens. 1 Introduction Figure 1: Classification task performance averaged over 90 datasets, using the Llama-3.1-8B-Instruct model. Prompt-MII achieves performance comparable to ICL while using 13\times fewer tokens. One common use pattern for large language models (LLMs) is to adapt them to a specific downstream task. In a supervised adaptation scenario, we are given n labeled demonstrations S_{\text{train}}=\{(x_{k},y_{k})\}_{k=1}^{n} and are interested in the problem of how to accurately predict labels for a set of test examples S_{\text{test}}=\{(x_{j},y_{j})\}_{j=1}^{m} drawn from the same distribution. There are multiple typical ways to incorporate the given examples: (1) Prompting with instructions , where a natural language task description I is appended to the model prefix, (2) In-context learning (ICL) , which directly uses examples in S_{\text{train}} as context during inference, and (3) Supervised fine-tuning (SFT) , which performs gradient updates on S_{\text{train}} to condense the information into model parameters. Each method has its advantages. Prompting with instructions is concise and efficient but requires extensive prompt engineering ( Sahoo et al. 2024 ; Schulhoff et al. 2024 ) . ICL achieves highly competitive performance but can be inefficient as the number of examples grows larger ( Xiao et al. 2025 ) . SFT is efficient at test time but uses significant compute at training time, requires storage of model weights, and underperforms ICL in many cases ( Bertsch et al. 2024 ) . As a method to bridge the gap between ICL and prompting, there exists instruction induction , which takes training data S_{\text{train}} and generates an instruction I that achieves good performance. Representative methods for instruction induction such as APE ( Zhou et al. 2022 ) and GEPA ( Agrawal et al. 2025 ) typically do so through expensive evolutionary search algorithms at test time that generate multiple candidates for prompts and evaluate them to choose a well-performing prompt. This raises the question: is there a way to perform instruction induction in a way that is both effective and efficient over a wide variety of tasks? As an answer to this question, we propose Prompt-MII , where we frame instruction induction as a meta-learning problem: instead of individually optimizing I for each individual task, we train an instruction induction policy \pi_{\theta} that can effectively generate instructions in a single pass across diverse task distributions, conditioned only on the in-context examples: I=\pi_{\theta}(S_{\text{train}}^{(i)}) (1) There are two major advantages to this approach. First, it allows \pi_{\theta} to share knowledge about how to construct effective prompts across a wide number of datasets, instead of requiring the re-discovery of this knowledge for each dataset. Second, it has significant efficiency benefits – generating an instruction I for a new dataset simply requires a single forward pass through the language model, instead of a costly optimization process. Experiments demonstrate that Prompt-MII is highly effective. For instance, in Figure 1 we show how Prompt-MII can achieve performance comparable to 100-shot ICL while consuming 13x fewer tokens. In the following sections, we discuss the methodological details of Prompt-MII ( section 2 ), experimental details ( section 3 ), and results and analysis ( section 4 ). 2 Prompt-MII : Meta-learning Instruction Induction Figure 2: Overview of Prompt-MII . We train an Instruction Generator LLM’s ability to perform instruction induction. At inference time, given dataset examples of an unseen task, it can automatically generate a reusable task instruction in a single pass, which then guides a black-box Instruction Follower LLM to make predictions. The main challenge in developing a method to generate instructions I from a dataset S_{\text{test}} is learning an effective policy \pi_{\theta} that can generate these instructions in a way that will achieve good test performance. We use reinforcement learning (RL) to train this policy because ground-truth dataset-instruction pairs are often not available for existing public datasets. However, we can evaluate instruction quality through downstream task performance, which serves as a natural reward signal for RL. In this section, we develop our method for meta-learning such a policy, also shown in Figure 2 . 2.1 Training Objective Let \mathcal{S}=\{S_{1},S_{2},\ldots,S_{N}\} be a collection of datasets that we will use in the meta-learning of \pi_{\theta} . For each dataset S_{i} , we sample training examples S_{\text{train}}^{(i)} for instruction generation and test examples S_{\text{test}}^{(i)} for reward computation. We define a meta-prompt template T(S_{\text{train}}^{(i)})) , which converts the dataset into a prompt to the model, as detailed in subsection 2.2 . Then, \pi_{\theta} generates an instruction prompted by this meta-prompt, I\sim\pi_{\theta}(T(S_{\text{train}}^{(i)})) . To assess the quality of the generated instruction, we use a separate frozen language model \text{LM}_{\text{eval}} as the instruction follower. This LM then processes the test set S_{\text{test}} using this instruction, generating results \hat{y}_{j}=\text{LM}_{\text{eval}}(I+\text{"Input: "}+x_{j}+\text{"Label:"}) . We use a task-dependent evaluation metric over m test examples to assess the model performance E\!\left(\{\hat{y}_{j}\}_{j=1}^{m},\;\{y_{j}\}_{j=1}^{m}\right) . In principle, this metric can range from classification metrics such as accuracy and macro-F1 to generation based metrics such as LLM-as-a-judge, but in this work we focus on classification tasks and use macro-F1 as our target reward metric and m=20 to balance stability and efficiency. To avoid training the model to learn the format requirement that is easily enforced manually, we add the custom format line Only return one of these options: {label_names}. Do not output "Label:" or any extra text. after the generated instruction, before calculating the reward. This constraint is equally added to all baseline methods we compare in the results. Together, this results in a reward for our generated instruction of R(I,S_{\text{test}})=E\!\left(\{\hat{y}_{j}\}_{j=1}^{m},\;\{y_{j}\}_{j=1}^{m}\right) (2) Once we have defined this reward, it can be optimized with an RL algorithm of choice. We use Group Relative Policy Optimization (GRPO; Shao et al. 2024 ) and enhance the algorithm with asymmetric clipping and removal of KL loss, which has been shown to encourage more exploration ( Yu et al. 2025 ) . Full details of the RL objective are in subsection A.1 . 2.2 Meta-Prompt Template One key element of our method is the use of a meta-prompt template T that encourages the LLM to generate instructions with generalizable patterns rather than regurgitating specific examples or simply summarizing the label space. Meta-prompt design impacting prompt quality is a known phenomenon in automatic prompt optimization (APO) methods ( Ding et al. 2025 ) . Our ablation studies in subsection 2.2 reveal model-dependent preferences, and accordingly, we use model-specific meta-prompts optimized for each model, but fix the