Measurement-Constrained Sampling for Text-Prompted Blind Face Restoration

Wenjie Li, Yulun Zhang, Guangwei Gao, Heng Guo, Zhanyu Ma · 2025

Read Measurement-Constrained Sampling for Text-Prompted Blind Face Restoration by Wenjie Li, Yulun Zhang, Guangwei Gao, Heng Guo, Zhanyu Ma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-14213.txt.

Introduction

Wenjie Li, Yulun Zhang, Guangwei Gao, and two co-authors let text prompts steer the restoration of badly damaged face photos. A very low-quality input can honestly match many plausible clean faces, yet most restoration methods output one fixed answer. Their measurement-constrained sampling approach recasts restoration as a generative inverse problem built from controlled degradations of rough restorations.

Two constraints guide sampling inside a text-to-image diffusion model. The forward constraint keeps results faithful to the input's structure, while the reverse constraint opens projection spaces where different prompts can be honored. Experiments show prompt-aligned, varied reconstructions that beat existing blind face restoration methods.

Catalog record

CREATORWenjie Li, Yulun Zhang, Guangwei Gao, Heng Guo, Zhanyu Ma
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-14213

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Measurement-Constrained Sampling for Text-Prompted Blind Face Restoration Wenjie Li, Yulun Zhang, Guangwei Gao, Heng Guo, Zhanyu Ma Abstract Blind face restoration (BFR) may correspond to multiple plausible high-quality (HQ) reconstructions under extremely low-quality (LQ) inputs. However, existing methods typically produce deterministic results, struggling to capture this one-to-many nature. In this paper, we propose a Measurement-Constrained Sampling (MCS) approach that enables diverse LQ face reconstructions conditioned on different textual prompts. Specifically, we formulate BFR as a measurement-constrained generative task by constructing an inverse problem through controlled degradations of coarse restorations, which allows posterior-guided sampling within text-to-image diffusion. Measurement constraints include both Forward Measurement, which ensures results align with input structures, and Reverse Measurement, which produces projection spaces, ensuring that the solution can align with various prompts. Experiments show that our MCS can generate prompt-aligned results and outperforms existing BFR methods. Codes will be released after acceptance. 1 Introduction Blind face restoration (BFR) aims to reconstruct high-quality (HQ) facial images from low-quality (LQ) inputs with unknown and complex degradations, which has been extensively studied in recent years 21 . This task becomes particularly ambiguous under extremely LQ conditions, where a single input may correspond to multiple semantically plausible HR reconstructions 21 . As illustrated in Fig. 1 , different face images with varying attributes ( e.g . , expressions, hairstyles, accessories) may degrade into visually similar LQ observations. Therefore, this one-to-many nature not only increases the challenge of restoration but also reveals the potential for personalized and controllable BFR. \begin{overpic}[img/CVPR_teaser.pdf] \put(4.0,20.5){\color[rgb]{0,0,0}{Real LQ}} \put(24.0,20.5){\color[rgb]{0,0,0}{DR2\penalty\ \cite[cite]{\@@bibref{Authors Phrase1YearPhrase2}{wang2023dr2}{\@@citephrase{(}}{\@@citephrase{)}}}}} \put(42.6,20.5){\color[rgb]{0,0,0}{DDNM\penalty\ \cite[cite]{\@@bibref{Authors Phrase1YearPhrase2}{wang2022zero}{\@@citephrase{(}}{\@@citephrase{)}}}}} \put(62.9,20.5){\color[rgb]{0,0,0}{PGDiff\penalty\ \cite[cite]{\@@bibref{Authors Phrase1YearPhrase2}{yang2023pgdiff}{\@@citephrase{(}}{\@@citephrase{)}}}}} \put(82.5,20.5){\color[rgb]{0,0,0}{SSDiff\penalty\ \cite[cite]{\@@bibref{Authors Phrase1YearPhrase2}{li2025self}{\@@citephrase{(}}{\@@citephrase{)}}}}} \par\par\put(1.2,-3.2){\color[rgb]{0,0,0}{DifFace\penalty\ \cite[cite]{\@@bibref{Authors Phrase1YearPhrase2}{yue2024difface}{\@@citephrase{(}}{\@@citephrase{)}}}}} \put(21.9,-3.2){\color[rgb]{0,0,0}{DiffBIR\penalty\ \cite[cite]{\@@bibref{Authors Phrase1YearPhrase2}{lin2024diffbir}{\@@citephrase{(}}{\@@citephrase{)}}}}} \put(46.5,-3.2){\color[rgb]{0,0,0}{`Null'}} \put(63.1,-3.2){\color[rgb]{0,0,0}{`Little girl'}} \put(81.4,-3.2){\color[rgb]{0,0,0}{`Straight hair'}} \par\put(5.0,49.2){\color[rgb]{0,0,0}{(a) Different HQ faces may degrade into the same LQ face.}} \put(0.8,-9.5){\color[rgb]{0,0,0}{(b) Our method can generate high-quality prompt-aligned faces.}} \par\end{overpic} Figure 1 : Illustration of the one-to-many nature of blind degradations and the effectiveness of our method in generating diverse, prompt-aligned face images from extremely low-quality inputs. However, existing BFR 32 ; 34 ; 38 ; 24 methods struggle to capture such one-to-many uncertainty in blind degradations, limiting their flexibility in applications. Specifically, with the increasing adoption of diffusion models 15 in BFR, surpassing the performance of GAN-based methods 32 ; 43 , existing diffusion-based methods can be broadly classified into two categories: i) Learning-based methods 34 ; 24 typically rely on supervised training with paired low- and high-quality data, and are often optimized to produce a single deterministic output, making them inadequate for modeling the inherent one-to-many correspondence under severe degradations. ii) Sampling-based methods 33 ; 6 ; 38 ; 22 typically accomplish restoration by modifying the sampling trajectory of pre-trained diffusion models, enabling zero-shot, training-free adaptation to a variety of restoration tasks. \begin{overpic}[img/insight_CVPR.pdf] \par\end{overpic} Figure 2 : ( Left ) While real and constructed degradations differ, the gap diminishes in reverse diffusion, making constructed observations suitable anchors of inverse problem solving for guided sampling. ( Right ) When facial structures in noisy samples are unclear in the reverse diffusion process of T2I diffusion, inverse problem solving centred on constructed degradations amplifies noise, resulting in artifacts. Sampling-based approaches utilizing inverse problem solving 33 ; 6 ; 1 , though not inherently text-driven, show potential for generating diverse outputs when combined with pre-trained text-to-image (T2I) diffusion models. Specifically, these methods define a degradation operation A , guiding each step {{\hat{\boldsymbol{x}}}_{t}} of the reverse diffusion to degenerate to LQ inputs {\boldsymbol{I}_{LQ}} , e.g . , A\left({{{\hat{\boldsymbol{x}}}_{t}}}\right)\!\!\to\!\!{\boldsymbol{I}_{LQ}} . Since HQ faces with varying attributes may map to the same LQ face, this approach, when integrated with text guidance, can produce a solution space with various HQ faces exhibiting diverse attributes. However, its applicability is limited, as it often assumes prior knowledge of the degradation operation A , which is challenging to satisfy in blind or real-world degradation contexts. Building on the successes and limitations mentioned above, we aim to construct a degradation observation that approximates real degradation in the T2I diffusion while enabling the solution of inverse problems. Therefore, we explore the relationship between real degradation and constructed degradation ( e.g . , bicubic interpolation applied to results after a restoration process input). As shown in the left panel of Fig. 2 , during the reverse diffusion process, where noise dominates, the gap between constructed and real degradation significantly narrows. This observation led us to formulate an inverse problem in which constructed degradation can serve as an anchor for reverse diffusion. However, as shown in the right panel of Fig. 2 , this formulation can disrupt the reverse diffusion process when facial structures in noisy samples are unclear, resulting in pronounced artifacts. The solution of the inverse problem can be mitigated when facial structures in the noisy samples are well-defined. Based on these observations, we propose a Measurement-Constrained Sampling (MCS) that unifies text-driven control and measurement guidance within pre-trained T2I diffusion. To solve the inverse problem, we first apply a constructed degradation to the rough estimate from the restorer 4 , creating an anchor. Simultaneously, we introduce a Reverse Measurement to constrain the denoising trajectory within a degradation-consistent projection space, anchored by the constructed degradation and guided by prompts. To avoid artifacts arising from the reverse strategy when facial structures are unclear in denoised samples, as discussed in Sec. 4.1 , we introduce a Forward Measurement in early sampling steps to approximate restoration that aligns with the structures of inputs. Additionally, we employ noise-blended anchors to initialize reverse diffusion, providing an approximate facial structure and bridging the domain gap between synthesis and real degradation under noise dominance. To the best of our knowledge, our MCS is the first BFR method capable of generating text-prompt-aligned faces in a training-free manner. As shown in Fig. 1 , benefiting from balancing Forward and Reverse Measurement, our MCS outperforms existing learning-based 34 ; 24 ; 41 and sampling-based 33 ; 38 ; 22 diffusion methods in both BFR diversity and quality. In summary, our contributions are as follows: • We leverage pre-trained T2I models to reconstruct diverse HR faces from blind LR inputs, guided by both text-based semantics and measurement consistency constraints; • We introduce forward measurements to approximate degradations like facial structures, and reverse measurements to expand generative spaces for satisfying diversity prompts; • Under blind degradation, our MCS produces accurate face results without prompts and generates text-aligned faces when guided, outperforming existing BFR methods. 2 Related Work 2.1 Blind Face Restoration To avoid the limitations of non-blind methods 5 ; 37 ; 20 ; 29 , which are difficult to generalize to realistic degradations, blind FSR has seen rapid progress in recent years. Early approaches 23 ; 35 enhanced networks by incorporating face dictionary priors, but these methods lack robustness under extremely LR conditions. To address this, recent works incorporate generative priors into networks. Specifically, GLEAN 2 , GPEN 40 , GFPGAN 32 , and SGPN 44 integrate StyleGAN-based priors into network architectures to boost performance in specific scenarios. VQFR 13 , CodeFormer 43 , and DAEFR 28 further enhance the determinism of facial feature representations by introducing discretized GAN priors, such as pre-trained codebooks, thereby reducing the likelihood of facial artifacts. With the advent of generative diffusion 25 in generative tasks, methods like StableSR 30 , DiffBIR 24 , OSEDiff

Read the complete text