Enhancing Diffusion Models for Inverse Problems with Covariance-Aware Posterior Sampling
Shayan Mohajer Hamidi · 2024
Read Enhancing Diffusion Models for Inverse Problems with Covariance-Aware Posterior Sampling by Shayan Mohajer Hamidi (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-20045.txt.
Description
Diffusion models can repair or rebuild images from partial data, a family of jobs called inverse problems, but the sampling step leans on an approximated likelihood. Shayan Mohajer Hamidi and co-author En-Hui Yang sharpen that approximation. Earlier methods used only the mean of the reverse process, read off through Tweedie's formula, and ignored how spread out the possibilities are.
The pair derive an exact formula for that spread, the covariance of the reverse process, and estimate it with finite differences from an already trained model, so nothing new must be trained. Feeding both mean and spread into the likelihood gives their method, covariance-aware diffusion posterior sampling. In experiments it rebuilt images clearly better than prior sampling schemes, and it needed no hyperparameter tuning. The authors offer it as a drop-in improvement for diffusion-based solvers of inverse problems such as inpainting, deblurring, and super-resolution.
Catalog record
| CREATOR | Shayan Mohajer Hamidi |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-20045