Iterative Importance Fine-tuning of Diffusion Models

Alexander Denker, Shreyas Padhy, Francisco Vargas, Johannes Hertrich · 2025

Read Iterative Importance Fine-tuning of Diffusion Models by Alexander Denker, Shreyas Padhy, Francisco Vargas, Johannes Hertrich (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-04468.txt.

Description

Alexander Denker and co-authors fine-tune image-generating diffusion models by teaching them an optimal steering control, in a 2025 paper. Diffusion models are prized priors for imaging and protein design. Steering their sampling toward a desired condition, like a class label or a measurement, is the hard part. A classical trick, Doob's h-transform, describes the ideal steering but not how to learn it.

Their self-supervised algorithm learns the control in rounds. Each round resamples paths with importance weights and refits the control on the improved synthetic data. Class-conditional sampling, image reconstruction problems, and reward tuning for text-to-image models all benefited. Amortized sampling means the steering cost is paid once, up front, not per sample.

Catalog record

CREATORAlexander Denker, Shreyas Padhy, Francisco Vargas, Johannes Hertrich
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE132 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-04468

Browse the full library · About the Tower of Babel