COMPOSE: Hypergraph C over O ptimization for M ulti-view 3D Human Pose Estimation

COMPOSE Tony Danjun Wang \email [email protected], COMPOSE Tolga Birdal \email [email protected], COMPOSE Nassir Navab \email [email protected], COMPOSE Lennart Bastian \email [email protected] · 2026

Read COMPOSE: Hypergraph C over O ptimization for M ulti-view 3D Human Pose Estimation by COMPOSE Tony Danjun Wang \email [email protected], COMPOSE Tolga Birdal \email [email protected], COMPOSE Nassir Navab \email [email protected], COMPOSE Lennart Bastian \email [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-09698.txt.

Introduction

Tony Danjun Wang, Tolga Birdal, Nassir Navab, and Lennart Bastian rebuild 3D human poses from a few camera views. Their 2026 arXiv paper presents COMPOSE, a hypergraph optimization method for multi-view pose estimation. The task feeds action recognition, sports analysis, and human-robot interaction.

Standard pipelines first find 2D keypoints in each view, then match them across views and triangulate. Matching is the fragile step, since points hide and swap identities. A hypergraph can encode richer matches than pairwise links. Better matching should give steadier skeletons from sparse camera setups in gyms, clinics, and studios.

Catalog record

CREATORCOMPOSE Tony Danjun Wang \email [email protected], COMPOSE Tolga Birdal \email [email protected], COMPOSE Nassir Navab \email [email protected], COMPOSE Lennart Bastian \email [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-09698

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

COMPOSE: Hypergraph C over O ptimization for M ulti-view 3D Human Pose Estimation COMPOSE Tony Danjun Wang \email [email protected], COMPOSE Tolga Birdal \email [email protected], COMPOSE Nassir Navab \email [email protected], COMPOSE Lennart Bastian \email [email protected] Abstract 3D pose estimation from sparse multi-views is a critical task for numerous applications, including action recognition, sports analysis, and human-robot interaction. Optimization-based methods typically follow a two-stage pipeline, first detecting 2D keypoints in each view and then associating these detections across views to triangulate the 3D pose. Existing methods rely on mere pairwise associations to model this correspondence problem, treating global consistency between views (i.e., cycle consistency) as a soft constraint. Yet, reconciling these constraints for multiple views becomes brittle when spurious associations propagate errors. We thus propose COMPOSE , a novel framework that formulates multi-view pose correspondence matching as a hypergraph partitioning problem rather than through pairwise association. While the complexity of the resulting integer linear program grows exponentially in theory, we introduce an efficient geometric pruning strategy to substantially reduce the search space. COMPOSE achieves improvements of up to 23% in average precision over previous optimization-based methods and up to 11% over self-supervised end-to-end learned methods, offering a promising solution to a widely studied problem. Figure 1: Top: Traditional approaches rely on pairwise geometric constraints [ 8 , 34 , 31 ] . As illustrated, these methods generate pairwise associations that, while locally consistent between two views, often fail to form a globally coherent structure. Consequently, algorithms face the difficult task of reconciling these locally plausible but globally conflicting edges to recover the correct 3D poses. Bottom: We propose a hypergraph formulation that jointly models higher-order relationships across views. We re-frame correspondence matching as a hypergraph partitioning problem, where hyperedges encode multi-view consistency. This global formulation effectively resolves ambiguities by enforcing consensus across the entire set of views. 1 Introduction Human pose estimation is a fundamental task in computer vision, yet its deployment in safety-critical scenarios remains a significant challenge. While the research community has primarily focused on monocular settings due to the availability of large-scale annotated datasets, these methods inherently lack depth information and suffer from severe occlusions [ 35 ] . This limitation is unacceptable in high-stakes real-world applications such as collaborative human-robot interaction [ 10 ] and operating room monitoring [ 29 ] , where precise spatial awareness is required to ensure safety. To achieve the necessary robustness and geometric accuracy, standard practice involves deploying calibrated multi-camera setups. By capturing activities from diverse viewpoints, these setups allow for geometric triangulation, thereby resolving the depth ambiguities inherent to single-view imaging. Multi-view 3D human pose estimation has historically relied on these geometric multi-view consistency constraints [ 24 ] . Seminal approaches employ optimization to lift 2D detections from multiple monocular views into 3D space via triangulation [ 2 ] . However, purely geometric methods can be highly sensitive to 2D detection ambiguities and noisy keypoints, leading to suboptimal performance in complex, crowded scenes. The advent of deep learning revolutionized 2D pose estimation, providing robust keypoint detections that serve as improved inputs for geometric lifting. Despite these advances in 2D backbones, the community has largely shifted toward end-to-end learning-based 3D frameworks [ 27 , 33 , 20 ] . While these approaches yield impressive results, they require large annotated datasets. However, obtaining ground-truth 3D annotations is labor-intensive and technically challenging; since skeletal joints are internal to the human body, they cannot be directly observed or accurately annotated from surface scans alone. Furthermore, learning-based models frequently suffer from domain gaps, failing to generalize to unseen environments or novel camera configurations [ 20 , 6 ] . Methods that can generalize without reliance on precise 3D supervision are thus highly sought after. In this paper, we propose COMPOSE , a novel modeling framework for multi-view 3D human pose estimation that effectively captures the complex relations between 2D observations and 3D human poses (see Figure 1 ). Previous approaches tend to model multi-view relationships by synchronizing many dyadic (pairwise) relations, attempting to recover a cycle-consistent matching between views [ 8 , 14 , 14 , 3 , 5 ] . However, such methods are inhibited by their reliance on operating between pairwise matches computed in isolation at a time (see top of Figure 1 ). This makes it challenging to resolve ambiguities when individual views are occluded or noisy [ 11 ] . To alleviate these challenges, we introduce a hypergraph-based formulation that extends beyond pairwise relationships to capture higher-order interactions among multiple views simultaneously. By treating consistent sets of 2D detections across views as hyperedges, our model enforces a holistic consensus across all cameras simultaneously, making it significantly more robust to outliers in individual frames (see Figure 1 , where hyperedges better express global geometric consistency). With a purely optimization-based approach, our method successfully regresses accurate 3D human poses from multi-view 2D observations, even in the presence of severe occlusions. Furthermore, our experiments demonstrate that our hypergraph-based approach outperforms state-of-the-art optimization-based and self-supervised learning methods. The main contributions of this paper are summarized as follows: • We propose COMPOSE , a novel hypergraph formulation for multi-view 3D human pose estimation that advances the paradigm from pairwise association between camera views to considering detections from all cameras simultaneously; this is cast as a partitioning problem over a higher-order graph. • We introduce a robust optimization objective and a geometrically guided pruning strategy that enables the efficient solution of the theoretically exponential correspondence problem via Integer Linear Programming (ILP). • We demonstrate through extensive experiments that our optimization-based method outperforms state-of-the-art optimization-based as well as self-supervised learning-based methods on several standard benchmarks. 2 Related Works 2.1 Optimization-Based Approaches 3D multi-view multi-human pose estimation was first established through optimization-based approaches that lift 2D priors (e.g., off-the-shelf 2D human pose estimator) into 3D by triangulating 2D correspondences across views. As a pioneering work, Belagiannis et al. [ 2 ] introduce a conditional random field framework (3D Pictorial Structures) that uses a discrete state space and multi-view potential functions to resolve identity and body part ambiguities across multiple camera views. Dong et al. [ 8 ] establish multi-view correspondences by formulating the multi-way matching problem as a convex optimization problem that simultaneously clusters 2D detections across all views using appearance similarity and geometric consistency cues while enforcing a cycle-consistency constraint. To address occlusions and crowded scenes, Zhou et al. [ 36 ] introduce multi-view association at the level of partial skeleton proposals instead of body-level. Zhang et al. [ 34 ] introduce the temporal dimension to the task and propose a spatio-temporal graph formulation for both spatial and temporal associations. These optimization-based methods are generally efficient and require minimal computational resources. However, compared to learnable methods, they still struggle to handle noisy 2D detections and occlusions. 2.2 Learning-Based Approaches Currently, the prevalent body of work in 3D multi-view multi-human pose estimation focuses on approaches that learn to regress 3D human poses. Early learning-based methods encode the environment as 3D voxel grids and use 3D CNNs to directly regress 3D human poses from the voxelized representation [ 15 , 27 , 4 , 26 ] . However, these methods are generally computationally expensive due to the cubic complexity of 3D CNNs.

Read the complete text