Enhancing Diffusion Models for Inverse Problems with Covariance-Aware Posterior Sampling

Shayan Mohajer Hamidi · 2024

Read Enhancing Diffusion Models for Inverse Problems with Covariance-Aware Posterior Sampling by Shayan Mohajer Hamidi (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-20045.txt.

Description

Diffusion models can repair or rebuild images from partial data, a family of jobs called inverse problems, but the sampling step leans on an approximated likelihood. Shayan Mohajer Hamidi and co-author En-Hui Yang sharpen that approximation. Earlier methods used only the mean of the reverse process, read off through Tweedie's formula, and ignored how spread out the possibilities are.

The pair derive an exact formula for that spread, the covariance of the reverse process, and estimate it with finite differences from an already trained model, so nothing new must be trained. Feeding both mean and spread into the likelihood gives their method, covariance-aware diffusion posterior sampling. In experiments it rebuilt images clearly better than prior sampling schemes, and it needed no hyperparameter tuning. The authors offer it as a drop-in improvement for diffusion-based solvers of inverse problems such as inpainting, deblurring, and super-resolution.

Catalog record

CREATORShayan Mohajer Hamidi
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-20045

Browse the full library · About the Tower of Babel