Critic-Guided Reinforcement Unlearning in Text-to-Image Diffusion

Mykola Vysotskyi, Zahar Kohut, Mariia Shpir, Taras Rumezhak, Volodymyr Karpiv · 2026

Read Critic-Guided Reinforcement Unlearning in Text-to-Image Diffusion by Mykola Vysotskyi, Zahar Kohut, Mariia Shpir, Taras Rumezhak, Volodymyr Karpiv (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03213.txt.

Description

Mykola Vysotskyi and four co-authors teach text-to-image diffusion models to forget chosen concepts using reinforcement learning with a critic that watches every step. Unlearning must erase a targeted concept while keeping the model's general usefulness. Earlier reinforcement approaches scored only the finished image, which gives sparse rewards, noisy updates, and poor credit for which denoising steps mattered.

Their framework treats denoising as a sequential decision process. A reward predictor built on CLIP is trained on noisy intermediate images and supplies a per-step signal, from which advantage estimates drive policy-gradient updates of the reverse diffusion process. The method is simple, allows off-policy reuse, and plugs into standard backbones. Across several concepts it matched or beat strong baselines at forgetting while preserving image quality and fidelity on harmless prompts.

Catalog record

CREATORMykola Vysotskyi, Zahar Kohut, Mariia Shpir, Taras Rumezhak, Volodymyr Karpiv
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-03213

Browse the full library · About the Tower of Babel