Proximal Supervised Fine-Tuning

Wenhong Zhu Ruobing Xie Rui Wang Xingwu Sun Di Wang Pengfei Liu · 2025

Read Proximal Supervised Fine-Tuning by Wenhong Zhu Ruobing Xie Rui Wang Xingwu Sun Di Wang Pengfei Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-17784.txt.

Description

Wenhong Zhu and five co-authors propose, in a 2025 paper, a method called Proximal Supervised Fine-Tuning, or PSFT, for adapting large models to new tasks. Ordinary supervised fine-tuning often improves performance on the new task but damages abilities the model had before. PSFT borrows the trust-region idea from reinforcement learning methods such as proximal policy optimization, which limits how far each update can move the model.

The authors show that supervised fine-tuning can be viewed as a policy gradient method with constant positive advantages. From that view, they derive an objective that constrains drift during training while still learning the new task effectively. In experiments on mathematical and human-value tasks, PSFT matched ordinary fine-tuning on the trained tasks. It generalized better to new domains, stayed stable during prolonged training without entropy collapse, and provided a stronger starting point for later training stages.

Catalog record

CREATORWenhong Zhu Ruobing Xie Rui Wang Xingwu Sun Di Wang Pengfei Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-17784

Browse the full library · About the Tower of Babel