REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedback

Aniruddha Roy, Pretam Ray, Abhilash Nandy, Somak Aditya, Pawan Goyal · 2025

Read REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedback by Aniruddha Roy, Pretam Ray, Abhilash Nandy, Somak Aditya, Pawan Goyal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06548.txt.

Description

Aniruddha Roy and four co-authors test whether small open language models can write their own training instructions, in a research paper from 2025. Fine-tuning needs instruction data, but human-written sets are costly and narrow. Letting a model generate its own instructions is cheaper, though earlier work leaned on huge closed models.

Their framework adds reinforcement learning with automated feedback to the self-generation loop. They try it with three open models of modest size. The reinforcement step improved results on most tasks over earlier self-generation methods. Dataset checks confirm the gains come from the added training signal, not from more data. The authors argue this lowers the cost of aligning smaller models.

Catalog record

CREATORAniruddha Roy, Pretam Ray, Abhilash Nandy, Somak Aditya, Pawan Goyal
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-06548

Browse the full library · About the Tower of Babel