Fine-Tuning without Performance Degradation

Han Wang, Adam White, Martha White · 2025

Read Fine-Tuning without Performance Degradation by Han Wang, Adam White, Martha White (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-00913.txt.

Description

Han Wang, Adam White, and Martha White fix a fine-tuning flaw in twenty twenty-five. Policies learned offline often get worse before they get better online. Early exploration overrides what offline training taught, causing a performance dip. Most prior progress chased learning speed instead, leaving the dip in place.

Across many settings, the authors show algorithms either dip or learn too slowly. Their new algorithm builds on one called Jump Start. It gradually allows more exploration, guided by online estimates of performance. That pacing protects the offline policy while still improving it. Readers in reinforcement learning get fine-tuning that aims for monotonic improvement rather than a valley first.

Catalog record

CREATORHan Wang, Adam White, Martha White
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-00913

Browse the full library · About the Tower of Babel