REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedback
Aniruddha Roy, Pretam Ray, Abhilash Nandy, Somak Aditya, Pawan Goyal · 2025
Read REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedback by Aniruddha Roy, Pretam Ray, Abhilash Nandy, Somak Aditya, Pawan Goyal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06548.txt.
Description
Aniruddha Roy and four co-authors test whether small open language models can write their own training instructions, in a research paper from 2025. Fine-tuning needs instruction data, but human-written sets are costly and narrow. Letting a model generate its own instructions is cheaper, though earlier work leaned on huge closed models.
Their framework adds reinforcement learning with automated feedback to the self-generation loop. They try it with three open models of modest size. The reinforcement step improved results on most tasks over earlier self-generation methods. Dataset checks confirm the gains come from the added training signal, not from more data. The authors argue this lowers the cost of aligning smaller models.
Catalog record
| CREATOR | Aniruddha Roy, Pretam Ray, Abhilash Nandy, Somak Aditya, Pawan Goyal |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-06548