Double Blind Imaging with Generative Modeling
Brett Levac, Ajil Jalal, Kannan Ramchandran, Jonathan I. Tamir · 2025
Read Double Blind Imaging with Generative Modeling by Brett Levac, Ajil Jalal, Kannan Ramchandran, Jonathan I. Tamir (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21501.txt.
Introduction
Brett Levac and co-authors solve blind imaging problems with generative modeling when the imaging system itself is unknown, published in 2025. Noise and uncertainty about the device blur the measurements, and recovery normally requires pinning down the system. A common route uses generative models as priors for both the image and the device settings.
Learning those priors the obvious way needs clean images plus samples of the device, which are rarely both available. Their method, built on an AmbientGAN idea, learns the spread of device settings from unpaired clean images and corrupted measurements alone. The learned spread then feeds model-based recovery methods such as diffusion posterior sampling. Tests learned both Gaussian and motion blur priors from noisy data and used them successfully for blind deconvolution.
Catalog record
| CREATOR | Brett Levac, Ajil Jalal, Kannan Ramchandran, Jonathan I. Tamir |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-21501
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Double Blind Imaging with Generative Modeling Brett Levac, Ajil Jalal, Kannan Ramchandran, Jonathan I. Tamir Abstract Blind inverse problems in imaging arise from uncertainties in the system used to collect (noisy) measurements of images. Recovering clean images from these measurements typically requires identifying the imaging system, either implicitly or explicitly. A common solution leverages generative models as priors for both the images and the imaging system parameters (e.g., a class of point spread functions). To learn these priors in a straightforward manner requires access to a dataset of clean images as well as samples of the imaging system. We propose an AmbientGAN-based generative technique to identify the distribution of parameters in unknown imaging systems, using only unpaired clean images and corrupted measurements. This learned distribution can then be used in model-based recovery algorithms to solve blind inverse problems such as blind deconvolution. We successfully demonstrate our technique for learning Gaussian blur and motion blur priors from noisy measurements and show their utility in solving blind deconvolution with diffusion posterior sampling. 1 Introduction Computational imaging techniques are integral across a wide range of disciplines, including astronomy [ 18 , 56 ] , microscopy [ 50 , 12 ] , medicine [ 49 , 22 ] , and consumer electronics [ 16 ] . Under additive noise, the imaging system can be modeled as y=A(x)+\eta, (1) where x\in\mathbb{C}^{n} is the image, A:\mathbb{C}^{n}\rightarrow\mathbb{C}^{m} represents the measurement (forward) operator, \eta\in\mathbb{C}^{m} is random additive noise, and y\in\mathbb{C}^{m} are the measurements. The central challenge is to estimate the clean image x from measurements y , a task known as the inverse problem. This problem is typically ill-posed due to additive noise ( \eta ) and potentially insufficient measurements ( m<n ) for stable inversion. Numerous classical [ 17 , 42 ] and deep learning-based methods [ 21 , 7 , 29 , 48 ] have been proposed to address signal recovery. These methods commonly assume that the measurement process A is known a priori, allowing it to be used directly in the recovery algorithm. This assumption implies that the exact characteristics of the imaging system used to acquire the measurements are known. Figure 1 : Example of imaging system setup where the measurement process is unknown due to hardware, environment, or scene variables. D_{\phi} z G_{\theta} Image dataset \{x_{1},\ x_{2},\ \cdots,x_{N}\} Measurement dataset \{y_{1},\ y_{2},\ \cdots,y_{N}\} \kappa^{g} x^{r} \mathcal{A} y^{r} y^{g} Figure 2 : Schematic block diagram illustrating our algorithm for unsupervised learning of the imaging system parameters. We learn a generative network G_{\theta} that generates system parameters \kappa^{g} . Given a set of training images \{x_{1},x_{2},\cdots,x_{N}\} , we pass these images along with the parameters \kappa^{g} through the imaging system to generate measurements y^{g} . The discriminative network D_{\phi} is trained to distinguish between the generated measurements y^{g} and actual measurements y^{r}\sim\{y_{1},y_{2},\cdots,y_{N}\} . Note that the image dataset and measurement dataset are unpaired , independent, and disjoint of one another. Blind Inverse Problems. In many practical scenarios, the imaging system is not known exactly at the time of signal recovery, but can be modeled as belonging to a family of forward models A_{\kappa} , where \kappa parameterizes the unknown components. The uncertainty in the imaging system can stem from various sources, and can often be modeled statistically. Randomness may be introduced by the imaging hardware itself, such as fixed but unknown lens aberrations. Uncertainty may also arise from hardware-agnostic effects like camera shake [ 15 ] , motion blur due to scene dynamics [ 19 ] , or atmospheric turbulence [ 44 ] . In these cases, even if the imaging hardware is static, the overall system is not, necessitating estimation of the imaging system for each measurement capture. Model-based algorithms for blind inverse problems jointly estimate both the clean image x and the correct forward model A_{\kappa} corresponding to the measurement data y [ 40 , 41 , 35 , 59 , 1 ] . This is typically accomplished by imposing priors on the image, p(x) , and the measurement model, p(A) [ 13 ] . While it is often feasible to learn p(x) through generative modeling using available clean images x\sim p(x) , directly observing or learning p(A) is more challenging. This is because the imaging system itself is not directly observed; rather, only the measurements y\sim p(y) that follow Eq. 1 . Moreover, it is not practical to assume a training set of paired data of images and measurements, i.e. \{(x_{i},y_{i})\}_{i\in[N]} , which further complicates the learning of the imaging system. Our approach. We are interested in the setting where we do not have access to direct observations of the forward process or paired data. We assume there is some low-dimensional structure in the forward process, and we assume access to an independent training set of clean images, and an independent training set of measurements. We build upon the principles of AmbientGAN [ 8 ] , but with a crucial distinction: instead of learning the distribution of clean images, we learn the distribution of forward operators using the unpaired images and measurements as training data. Our underlying assumption is that the measurement process can be represented implicitly by a generative model due to some underlying low-dimensional structure. This approach to learning the imaging system from unpaired examples naturally integrates with downstream system ID tasks that have model uncertainty (e.g. in blind deconvolution). Traditional blind deconvolution methods rely on alternating or joint estimation of the image and the blur kernel by assuming both are unknown and iteratively refining both components [ 35 , 59 , 1 ] . With our generative model-based framework, the learned distribution of forward operators p(A) can be used to provide a robust prior for the measurement operator in these solvers. This prior can guide off-the-shelf blind deconvolution algorithms by constraining the possible forms of the blur kernel, improving convergence to accurate solutions even when no exact operator measurements are available [ 13 ] . Our approach acts as a flexible plug-in for downstream tasks that require a statistical model for the imaging system. Our Contributions 1. We propose an unsupervised algorithm that learns a prior distribution over forward operators using only unpaired clean images and corrupted measurements that are independent and disjoint from one-another. Forward operators are structured operators defined by a physical imaging system, and in this paper we consider Gaussian smoothing and motion blur, though other structure can also be incorporated. We design a network architecture for the generative model which respects the physics of the imaging system and then leverage the loss function in AmbientGAN [ 8 ] to train the model. 2. We experimentally show that our method is robust to additive noise in the imaging system. The experiments in Section 4.3 show that our method is robust in low-SNR settings. 3. Our learning algorithm is independent of the downstream system ID or signal recovery algorithm. In Section 4.4 , we show that this modular approach of learning system priors can be used in off-the-shelf state-of-the-art algorithms [ 13 ] for blind deconvolution. 4. In Section 4.4 , we evaluate our algorithm on the classical problems of Gaussian and motion deblurring. We find that training with imaging forward processes learned with our technique enables