Globally Optimal Pose from Orthographic Silhouettes
Agniva Sengupta Dilara Kuş Jianning Li Stefan Zachow, Freie Universität Berlin Zuse Institute Berlin · 2026
Read Globally Optimal Pose from Orthographic Silhouettes by Agniva Sengupta Dilara Kuş Jianning Li Stefan Zachow, Freie Universität Berlin Zuse Institute Berlin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-09199.txt.
Introduction
Sengupta, Kuş, Li, and Zachow recover the pose of a known 3D shape from its plain silhouettes alone, with global optimality. Their method uses silhouette area continuity as rotation varies, storing precomputed signatures as a response surface over rotation space. Querying that surface prunes the rotation search sharply, and fitted ellipse ratios add a second global shape cue. It needs no point correspondences and works for any shape, beating comparable methods on synthetic and real examples.
Catalog record
| CREATOR | Agniva Sengupta Dilara Kuş Jianning Li Stefan Zachow, Freie Universität Berlin Zuse Institute Berlin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47857 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.09199
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Globally Optimal Pose from Orthographic Silhouettes Agniva Sengupta Dilara Kuş Jianning Li Stefan Zachow, Freie Universität Berlin Zuse Institute Berlin Abstract We solve the problem of determining the pose of known shapes in ℝ 3 \mathbb{R}^{3} from their unoccluded silhouettes. The pose is determined up to global optimality using a simple yet under-explored property of the area-of-silhouette: its continuity w.r.t trajectories in the rotation space. The proposed method utilises pre-computed silhouette-signatures, modelled as a response surface of the area-of-silhouettes. Querying this silhouette-signature response surface for pose estimation leads to a strong branching of the rotation search space, making resolution-guided candidate search feasible. Additionally, we utilise the aspect ratio of 2D ellipses fitted to projected silhouettes as an auxiliary global shape signature to accelerate the pose search. This combined strategy forms the first method to efficiently estimate globally optimal pose from just the silhouettes, without being guided by correspondences, for any shape, irrespective of its convexity and genus. We validate our method on synthetic and real examples, demonstrating significantly improved accuracy against comparable approaches. 1 1 1 Code and data: agnivsen.github.io/pose-from-silhouette/ 1 Introduction Estimating the global pose of 3D objects from a single image typically relies on point correspondences between the object template and the image. We address this pose-estimation problem using a much weaker cue – the object’s projected silhouette; we call this the Pose-from-Silhouette (P f S) problem. Our approach discards all assumptions related to the object’s shape, such as convexity or genus, and solve this P f S problem without additional information such as point correspondences or direct image intensities. Our input requirements are the projective silhouettes and a template of the object (in standard formats, e.g., triangulated mesh or semi-dense point cloud). Our solution is globally optimal when the shape-silhouette combination admits a unique solution. However, for symmetric shapes, global solutions for P f S may not be unique; in such cases, we estimate exactly one of these many redundant solutions – a reasonable outcome from silhouettes alone. This work presents the first globally optimal solution to the P f S problem. Motivation . Solving P f S is appealing due to breadth of its motivating use-cases. Silhouettes are a key visual cue for diverse engineering applications, e.g.: autonomous driving [ 35 ] , robotic manipulation [ 17 ] , robotic navigation [ 1 ] , Augmented Reality (AR) [ 6 ] , surgical AR [ 7 ] , space navigation [ 15 ] , reconstruction from sparse views in medical imaging [ 29 ] , automated flight control [ 39 ] , and industrial quality control [ 26 ] . A closer inspection of these usages reveal silhouettes to be always used in conjunction with additional visual cues, be it feature correspondences, image intensities, or temporal priors – an unsurprising finding. P f S is indeed a fundamentally ill-posed problem; finding its global optima is challenging with no existing solutions. Contributions. We solve the problem of pose-estimation using just the unoccluded silhouettes of rigid objects as input data. Our approach for solving this P f S problem involves pre-computing global shape signature responses for a given object-template and using this pre-computed shape-signature to infer pose from any arbitrary silhouette of this object. Specifically, we contribute the following: • Globally optimal silhouette-based pose estimation. Building upon the framework of 16 , we introduce shape signatures of orthographic silhouettes that vary continuously with an object’s orientation, enabling non-trivial branching strategies that lead to solutions converging to global optima, up to discretisation. • Geometric shape signatures and refinement. We propose two intuitive yet powerful silhouette descriptors: the Area-of-Silhouettes (A o S) and the aspect ratio of ellipses fitted to the silhouettes; we show that they capture sufficient geometric information for accurate pose estimation. Orientation is recovered numerically, followed by a post-hoc non-linear refinement on the \mathbb{SE}(3) manifold. • Generalisation to perspective imaging. Assuming depth priors as input, the same formulation achieves near-optimal accuracy for perspective silhouettes, demonstrating strong practical performance while preserving an explainable, initialisation-free design. We validate our approach on many synthetic and real datasets, strongly outperforming compared approaches. 2 Background Pose estimation from 2D silhouettes has been widely studied when considered in conjunction with 3D-to-2D point correspondences [ 21 , 29 ] or with alternative correspondence cues such as contour generators [ 24 ] , image textures [ 31 , 6 , 9 ] , or both [ 27 ] . However, correspondence-free pose from just the 2D silhouettes/contours has not been solved up to global optimality for general shapes. There exist specialised approaches for specific shapes, such as the case of pose-estimation from ellipse-ellipsoid matches [ 12 ] (which can be considered as pose estimation from elliptical silhouettes), for surfaces of revolution [ 38 ] , and for cylindrical objects [ 14 ] . None of these approaches, however, can be generalised to arbitrary shapes. There exist some older methods that utilise the notion of ‘silhouette-lookup’ [ 19 , 20 ] to estimate poses of deformable structures from learned silhouette appearances along with some other methods [ 33 , 22 ] that recognize objects based on the pencil of viewing tangent planes along the silhouette from smooth objects, but no efforts have been made to solve the pose estimation problem to global optimality, even in the rigid case. Recently, deep-learning based approaches [ 34 ] have been proposed for the P f S problem; unfortunately, they happen to be local methods necessitating an initial pose for their estimation process and requires colour information along object boundary. A Particle Swarm Optimisation (PSO) based approach has recently been proposed for perspective silhouettes [ 9 ] , though it remains dependent on approximate depth bounds and remains stochastic with no optimality guarantees. No globally optimal method exists for pose estimation from silhouettes alone. 3 Method Our methodological description begins by explaining the problem setup ( Section 3.1 ) and problem statement ( Section 3.2 ) followed by our strategy of branching the rotation space with global shape signatures ( Section 3.3 ) which culminates in a globally optimal and efficient pose estimation approach ( Section 3.4 ). 3.1 Problem Setup We assume the availability of any standard shape prior (e.g.: triangulated/tetrahedral mesh, explicit/implicit surfaces, etc.) of the object that is to be localized by its silhouettes. A dense point cloud of 3D surface points is obtained from this shape prior via standard methods (e.g.: sampling [ 8 ] for meshes). We denote this point cloud as \mathbf{Q}=[\mathbf{P}_{1},\ldots,\mathbf{P}_{M}]\in\mathbb{R}^{3\times M} maintaining M at a sufficiently large value to ensure accurate extraction of silhouettes from projections of \mathbf{Q} . We denote by \Pi_{\mathrm{O}}(\mathbf{Q}) , the point cloud orthographically projected into the XY -plane, obtained by simply eliminating the third row of \mathbf{Q} . Silhouettes are represented discretely by the ordered sequence \mathbf{G}=\langle\mathbf{s}_{k}\in\mathbb{R}^{2}|k\in\mathbb{Z}\rangle of 2D points within the XY -plane. There exists a function S(\cdot) that acts on a 2D point cloud computing the outer boundary of this point cloud with standard methods [ 10 ] , i.e., \mathbf{G}=S\big(\Pi_{\mathrm{O}}(\mathbf{Q})\big) if \mathbf{G} is the orthographic silhouette of \mathbf{Q} . The input silhouette is given by \mathbf{G}^{\ast} 3.2 Problem Statement The posed problem is defined in terms of the Hausdorff metric. Given an input silhouette