Breaking the Scalability Limit of Multi-Projector Calibration with Embedded Cameras
Takumi Kawano, Kohei Miura, Daisuke Iwai · 2026
Read Breaking the Scalability Limit of Multi-Projector Calibration with Embedded Cameras by Takumi Kawano, Kohei Miura, Daisuke Iwai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-24024.txt.
Introduction
Kawano, Miura, and Iwai break the linear scaling of multi-projector setup by embedding cameras in the target surface itself. The built-in cameras catch light from all projectors at once and split the patterns by incoming direction, so every projector's geometry can be solved together. Accuracy matches the old step-by-step method while capture cycles stay nearly flat as projectors are added.
Catalog record
| CREATOR | Takumi Kawano, Kohei Miura, Daisuke Iwai |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37102 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.24024
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Breaking the Scalability Limit of Multi-Projector Calibration with Embedded Cameras Takumi Kawano, Kohei Miura, Daisuke Iwai Abstract Conventional multi-projector calibration requires projecting and capturing structured light patterns for each projector sequentially, causing calibration time and effort to increase linearly with the number of projectors. This scalability bottleneck has long limited the deployment of large-scale projection mapping systems. We present a new calibration framework that breaks this limitation by embedding cameras into the surface of the calibration target. The embedded cameras directly capture the incoming projection light, enabling the separation of simultaneously projected structured light patterns from multiple projectors according to their incident directions. Our method establishes correspondences between the optical centers of the embedded cameras and the projector pixels, allowing the intrinsic and extrinsic parameters of all projectors to be simultaneously estimated. We further introduce a correction technique for small misalignments between the calibration board and camera optical centers. As a result, our system achieves calibration accuracy comparable to conventional methods while reducing the required number of projection-capture cycles from linear to nearly constant with respect to the number of projectors, dramatically improving scalability for dense multi-projector systems with overlapping projection regions, such as high-brightness stacking, super-resolution, light-field, and shadow-suppression displays. 1 Introduction Projection mapping (PM) Bimber and Raskar 2005 provides an extended reality (XR) experience by overlaying synthetic content onto physical surfaces. Modern PM applications increasingly rely on systems composed of multiple projectors—sometimes dozens—to achieve sufficient coverage, brightness, and resolution on large-scale architectural surfaces Mine et al. 2012 , immersive entertainment environments Jones et al. 2014 , and complex prototypes in industrial design Cascini et al. 2020 ; Yoshida et al. 2025 ; Takezawa et al. 2019 . Extreme cases include light-field display systems with hundreds of co-located projectors Lee et al. 2013 , as well as high-brightness stacking and super-resolution displays where all projectors share a common projection region. To accurately align the projected imagery with the target surface, geometric calibration of each projector is essential. Since a projector can be modeled as an inverse camera, its calibration can, in principle, follow similar methods used for camera calibration. However, unlike cameras, projectors cannot directly observe the 3D locations of pixels projected in a physical scene, which necessitates a different calibration strategy. In practice, projector calibration is performed indirectly by capturing structured light patterns projected onto a calibration target using an external camera. While this process works well for a single projector, it becomes considerably more challenging when multiple projectors are involved. When several projectors simultaneously project structured light patterns, their images can overlap on the target surface, making the individual patterns indistinguishable in the captured images. Consequently, conventional approaches must calibrate projectors sequentially, one at a time, to avoid pattern interference. This causes the calibration time and effort to increase linearly with the number of projectors—a fundamental scalability bottleneck that severely hinders the practical deployment of large-scale projector arrays. Therefore, reducing this calibration overhead has become as critical as achieving high accuracy. To address this challenge, we propose a parallel calibration method that allows all projectors to simultaneously project structured light patterns that can still be separated and decoded independently (Fig. 1 ). In conventional setups, patterns from different projectors cannot be separated because the calibration surface diffusely reflects the incident light, causing the directional information of each ray to be lost. To overcome this limitation, we embed cameras so that their optical centers are aligned with the calibration plane, allowing them to directly capture light from each projector. Since the incident direction determines the position of the light on the camera’s image sensor, the system preserves directionality even when multiple patterns overlap on the surface, enabling per-projector decoding from a single capture. While effective, this configuration also introduces a unique technical issue: the optical centers of the embedded cameras are slightly offset from the calibration plane in practice. We propose an algorithm that compensates for this issue and validate the system through hardware implementation, showing that it achieves accuracy comparable to existing methods while reducing calibration time to nearly constant with respect to the number of projectors, provided that all projectors share a common projection region. (a) Conventional technique (b) Proposed technique Figure 1 : Imaging principle of simultaneously projected structured light patterns from multiple projectors: (a) Overlapping patterns captured by an external camera are difficult to separate, whereas (b) the embedded cameras directly receive the projection light, enabling separation of individual patterns. 2 Related Work Projector calibration techniques have been extensively studied not only in the context of PM but also for 3D shape measurement. In both applications, the underlying calibration principle is consistent: a projector is modeled as an inverse camera, and the same mathematical formulation used for camera calibration is applied. Specifically, the well-known pinhole camera model is employed, where, given a 3D point (X,Y,Z) , the projected pixel (x,y) is computed as s[x\;y\;1]^{T}=K[R\;|\;t][X\;Y\;Z\;1]^{T} , where s is an arbitrary scale factor. K=[f_{x}\;0\;c_{x};\;0\;f_{y}\;c_{y};\;0\;0\;1] is the intrinsic matrix composed of the focal lengths (f_{x},f_{y}) and optical center (c_{x},c_{y}) , while the skew is typically ignored following prior works. The extrinsic matrix [R\ |\ t] represents the projector’s pose in the world coordinate system. Four major approaches have been proposed for estimating intrinsic and extrinsic parameters. The first approach estimates parameters by collecting multiple 3D-2D correspondences between points in the scene and their corresponding projector pixels. The 3D coordinates of the scene can be obtained either by using a calibration target with known geometry Kurth et al. 2018 or by estimating them using stereo cameras Raskar et al. 1999 ; Griesser and Van Gool 2006 or RGB-D sensors Ibrahim et al. 2024 ; Jones et al. 2014 . The projector’s image coordinates are then determined by projecting structured light patterns (e.g., Gray-codes) or 2D markers such as ArUco markers onto the scene and capturing them with an external camera. The second approach uses a planar calibration target printed with markers such as checkerboard patterns, similar to those used in camera calibration Zhang 2000 . The target is placed at multiple poses to cover the projector’s field of view, and 2D-2D correspondences between printed markers and projected image coordinates are collected for calibration. As in the first approach, the projector image coordinates are obtained by projecting structured light patterns Moreno and Taubin 2012 ; Zhang and Huang 2006 ; Huang et al. 2021 or 2D markers Audet and Cooperstock 2007 ; Ashdown and Sato 2005 ; Audet and Okutomi 2009 ; Gao et al. 2008 and observing them with an external camera. In studies that use projected 2D markers, different color combinations between printed and projected patterns have been utilized to allow stable separation of the two patterns in captured images. The third approach is self-calibration Yamazaki et al. 2011 ; Willi and Grundhöfer 2017 ; Resch et al. 2015 ; Li et al. 2017 , which estimates the intrinsic and extrinsic parameters of a projector-camera pair without prior knowledge about the scene surface shapes. These methods typically project structured light patterns onto a 3D scene to obtain projector-camera correspondences, from which the fundamental matrix is estimated and then decomposed into the intrinsic and extrinsic parameters of both devices. The fourth and most recent line of work