AnyBokeh: Physics-Guided Any-to-Any Bokeh Editing with Optical Fingerprint Transfer

Xinyu Hou, Xiaoming Li, Zongsheng Yue, Chen Change Loy · 2026

Read AnyBokeh: Physics-Guided Any-to-Any Bokeh Editing with Optical Fingerprint Transfer by Xinyu Hou, Xiaoming Li, Zongsheng Yue, Chen Change Loy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-31959.txt.

Introduction

Hou and three colleagues present AnyBokeh, a framework for editing photographic bokeh from a single image taken under unknown settings. Depth of field shapes attention and depth cues, but most editors assume an all-in-focus input. Refocusing pipelines that first rebuild a sharp image throw away the source blur cues and let reconstruction errors leak into the edit.

AnyBokeh estimates the source blur state directly, using a signed circle-of-confusion map alongside a disparity map. A linear relation between the two yields a source-specific optical fingerprint, which is transferred to the target focus and aperture. A generative editor conditioned on both blur maps then synthesizes the result. A synthetic dataset with exact depth and full camera metadata supports the training.

Catalog record

CREATORXinyu Hou, Xiaoming Li, Zongsheng Yue, Chen Change Loy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.31959

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AnyBokeh: Physics-Guided Any-to-Any Bokeh Editing with Optical Fingerprint Transfer Xinyu Hou, Xiaoming Li, Zongsheng Yue, Chen Change Loy Abstract Depth-of-field control is a fundamental tool in photography, yet post-capture bokeh editing from a single image remains challenging. A practical editor should handle images captured under arbitrary focus and aperture settings. Existing methods typically assume an all-in-focus input, or first recover an all-in-focus image before rendering new bokeh. Such pipelines can discard useful blur cues from the source image and propagate reconstruction artifacts into the final edit. We introduce AnyBokeh , a physics-guided framework for any-to-any bokeh editing . Instead of treating source blur merely as a degradation to be removed, AnyBokeh estimates the source blur state with a signed circle-of-confusion map and a disparity map. By modeling the linear relation between signed circle of confusion and disparity difference, AnyBokeh estimates a source-specific optical fingerprint and transfers the source optical characteristics to the desired focus and aperture setting. A generative editor conditioned on both source and target circle-of-confusion maps then performs relative blur synthesis, enabling spatially adaptive deblurring, preservation, and defocus rendering. To support physically supervised learning, we further construct a high-fidelity synthetic dataset with accurate depth, focus distance, and full EXIF metadata. Experiments on real-world benchmarks show that AnyBokeh achieves faithful and controllable editing across any-to-any bokeh editing, all-in-focus-to-bokeh rendering, and defocus deblurring, while avoiding all-in-focus reconstruction and test-time bokeh-level calibration commonly required by existing approaches. The code and dataset will be available at https://github.com/itsmag11/AnyBokeh . 1 Introduction Depth-of-field (DoF) control is an essential aesthetic tool in photography, guiding visual attention and conveying scene structure through selective focus and optical defocus, commonly perceived as bokeh. The strength and spatial distribution of bokeh are governed by the image formation process: scene points away from the focal plane are projected as circles of confusion (CoC) whose size depends on aperture, focus distance, focal length, and scene depth. In practice, a large physical aperture, corresponding to a small f-number such as f/2.0 , produces a shallow DoF and strong blur in out-of-focus regions, while a small physical aperture like f/22.0 increases the DoF and makes a wider depth range appear sharp. Although these optical effects can be controlled during capture, post-capture DoF manipulation from a single in-the-wild photograph remains highly challenging. The difficulty mainly comes from two unknowns. First, the input may already contain spatially varying defocus blur: some regions are sharp, while others are mildly or severely out of focus. A practical editor must understand this source blur state to decide which regions should be sharpened, preserved, or further defocused. Second, the camera parameters that determine the absolute blur scale, such as focus distance, aperture, focal length, sensor size, and image resolution, are often incomplete or unavailable for in-the-wild images. Together, these two unknowns make any-to-any bokeh editing fundamentally different from standard bokeh rendering: the editor must infer the current blur state of the source image while also determining the target blur scale under the desired optical setting. Existing methods address these two ambiguities only partially. Many bokeh rendering methods assume an all-in-focus (AIF) input [ 14 , 31 , 25 , 24 , 43 , 30 ] , where the source blur state is trivial and only the target bokeh needs to be synthesized. This assumption simplifies the problem, but restricts these methods to already sharp inputs. Recent refocusing pipelines [ 33 ] relax the input constraint by first reconstructing an AIF image from an arbitrary input and then rendering new bokeh. We refer to this forced AIF reconstruction as an AIF bottleneck : the input is projected into a sharp intermediate representation, which removes the observable source blur pattern and makes source-specific optical cues unavailable for subsequent target bokeh synthesis. Moreover, severe defocus deblurring is inherently ambiguous and can introduce unnecessary hallucinated details or structural errors that propagate into the final edit. In addition, because complete optical metadata is usually unavailable, existing pipelines often require bokeh-level calibration [ 24 , 43 , 33 ] , i.e., adjusting a renderer-specific blur scale to match the desired bokeh strength, through per-image ground-truth-guided search. To address these limitations, we propose AnyBokeh , a physics-guided framework for any-to-any bokeh editing. Instead of treating source blur merely as a degradation to be removed, AnyBokeh explicitly estimates the source blur state as a signed source CoC map, together with a disparity map. By modeling the linear relation between signed CoC diameter and disparity difference, we estimate a source-specific CoC–disparity slope, which absorbs the global optical and scaling factors that map disparity differences to image-space defocus. We call this slope an optical fingerprint . This quantity is physically equivalent to the bokeh-level parameter K used in prior rendering methods [ 24 , 43 , 33 ] , as both control how disparity differences are converted into CoC magnitude. In our formulation, we denote it by \kappa_{src} to emphasize that this CoC scale is estimated directly from the observed source blur state and disparity, rather than specified externally. We then transfer this optical fingerprint to the desired focus and aperture setting to obtain the target CoC map, avoiding AIF reconstruction and test-time bokeh-level calibration. AnyBokeh synthesizes the target image with a generative editor conditioned on the source image, source CoC, and target CoC. We refer to this design as dual-CoC conditioning : both the current and desired blur states are provided to the model, turning any-to-any bokeh editing into relative blur synthesis. This allows the editor to explicitly model the transition from source blur to target blur and perform spatially adaptive bokeh synthesis. To train the CoC and disparity estimator under accurate physical supervision, we further construct UnrealBokeh , a high-fidelity synthetic dataset rendered with ground-truth depth, focus distance, aperture, and focal length. The contributions of AnyBokeh are summarized as follows: • We propose a CoC-mediated optical fingerprint transfer framework for any-to-any bokeh editing. By estimating source CoC and disparity, our method extracts a source-specific CoC–disparity slope \kappa_{src} and analytically transfers it to the target focus and aperture setting, avoiding AIF reconstruction and test-time bokeh-level calibration. • We introduce dual-CoC conditioning for relative blur synthesis. By conditioning the editor on both source and target CoC, our model explicitly observes the input blur state and the desired output blur state, enabling spatially selective deblurring, preservation, and defocus synthesis. • We build a high-fidelity synthetic dataset for physically supervised CoC learning. Using Unreal Engine, we render diverse scenes with ground-truth depth, focus distance, aperture, and focal length across multiple optical configurations, enabling robust learning of source CoC and disparity estimation. 2 Related Work Bokeh Rendering. Classical depth-of-field rendering traditionally relies on physically based simulations, which produce realistic bokeh but are computationally expensive and demand complete 3D geometry [ 27 , 11 ] . To improve efficiency, image-space post-filtering methods have been widely adopted [ 15 , 19 , 34 , 3 , 23 ] . However, they inherently struggle with depth discontinuities, causing severe color bleeding and partial occlusion artifacts [ 5 , 31 ] . While subsequent approaches attempted to mitigate these issues via physical lens models, optical aberrations, or multiview and light-field synthesis [ 10 , 36 , 18 , 38 ] , handling complex occlusions remains a

Read the complete text