Critic-Guided Reinforcement Unlearning in Text-to-Image Diffusion
Mykola Vysotskyi, Zahar Kohut, Mariia Shpir, Taras Rumezhak, Volodymyr Karpiv · 2026
Read Critic-Guided Reinforcement Unlearning in Text-to-Image Diffusion by Mykola Vysotskyi, Zahar Kohut, Mariia Shpir, Taras Rumezhak, Volodymyr Karpiv (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03213.txt.
Description
Mykola Vysotskyi and four co-authors teach text-to-image diffusion models to forget chosen concepts using reinforcement learning with a critic that watches every step. Unlearning must erase a targeted concept while keeping the model's general usefulness. Earlier reinforcement approaches scored only the finished image, which gives sparse rewards, noisy updates, and poor credit for which denoising steps mattered.
Their framework treats denoising as a sequential decision process. A reward predictor built on CLIP is trained on noisy intermediate images and supplies a per-step signal, from which advantage estimates drive policy-gradient updates of the reverse diffusion process. The method is simple, allows off-policy reuse, and plugs into standard backbones. Across several concepts it matched or beat strong baselines at forgetting while preserving image quality and fidelity on harmless prompts.
Catalog record
| CREATOR | Mykola Vysotskyi, Zahar Kohut, Mariia Shpir, Taras Rumezhak, Volodymyr Karpiv |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-03213