Mesh-Aware Epipolar Matching for Multi-View Multi-Person 3D Pose Estimation in Basketball

Li Yin, Qin Haobin, Tomohiro Suzuki, Calvin Yeung, Mariko Isogawa, Keisuke Fujii · 2026

Read Mesh-Aware Epipolar Matching for Multi-View Multi-Person 3D Pose Estimation in Basketball by Li Yin, Qin Haobin, Tomohiro Suzuki, Calvin Yeung, Mariko Isogawa, Keisuke Fujii (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29953.txt.

Introduction

Li Yin and five colleagues estimate 3D poses in team sports, where bodies hide each other and kits confuse views. Their training-free MAEM pipeline matches players across views in two epipolar stages from a single-view mesh model. Disjoint-set clustering plus per-joint triangulation lock links and rebuild poses. On two basketball sets it beats rival training-free matchers and RGB-only leaders indoors and out. Scores near sixty and forty-one on one set show mesh shape can carry matching with no target-domain tuning.

Catalog record

CREATORLi Yin, Qin Haobin, Tomohiro Suzuki, Calvin Yeung, Mariko Isogawa, Keisuke Fujii
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60867
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.29953

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Mesh-Aware Epipolar Matching for Multi-View Multi-Person 3D Pose Estimation in Basketball Li Yin, Qin Haobin, Tomohiro Suzuki, Calvin Yeung, Mariko Isogawa, Keisuke Fujii Abstract Multi-view multi-person 3D pose estimation in team sports scenarios remains challenging due to players’ occlusions, appearance similarity from team uniforms, and the scarcity of annotated multi-view data, which limits the effectiveness and generalization of learning-based methods. Conversely, the accuracy of training-free methods is inherently limited by the quality of 2D keypoints detection and the robustness of cross-view association. To address these challenges, we propose Mesh-Aware Epipolar Matching (MAEM), a training-free framework for multi-view multi-person 3D pose estimation. Our method leverages a monocular 3D human mesh recovery model as the frontend. Based on 3D human mesh recovery model outputs, we develop a two-stage epipolar matching strategy that combines disjoint set union-based clustering and per-joint triangulation, enabling robust cross-view association and accurate 3D pose reconstruction. Experiments on two public multi-view basketball datasets show that MAEM improves over training-free association baselines and achieves competitive RGB-only accuracy across indoor and outdoor basketball scenarios. MAEM achieves mean MPJPE/PA-MPJPE of 59.8/40.7 mm on SportCenter EPFL and 74.0/51.8 mm on Human-M3 Basketball, demonstrating the practical utility of dense mesh geometry for cross-view association without target-domain training or fine-tuning. The code and data are available at: https://github.com/Yinlipp/MAEM.git keywords multi-person 3D pose estimation, multi-view fusion, epipolar geometry, body mesh recovery, team sports, training-free 1 Introduction 3D human pose estimation has become an essential component of modern sports analytics 3 ; 38 ; 4 , underpinning applications such as automated tactical analysis, biomechanical monitoring, and broadcast production. Team sports such as basketball and football represent one of the most demanding instantiations of this problem: high player density, frequent mutual occlusions, near-identical team uniforms, and rapid unpredictable motion combine to make multi-view multi-person 3D pose estimation substantially more challenging than in controlled benchmarks 18 ; 2 . Top-tier leagues address occlusion by deploying dozens of cameras throughout their venues, but such infrastructure is unavailable in most real-world settings. For the technology to generalize beyond elite competitions, robust multi-view multi-person 3D pose estimation must be achievable with only a sparse camera setup, placing the burden squarely on the cross-view association algorithm rather than on camera density. The central challenge in multi-view multi-person 3D pose estimation is cross-view association: determining which detections across views correspond to the same physical person. In team sports, this problem is uniquely acute, as occlusion, large camera-to-subject distances, and near-identical uniforms cause conventional association cues to degrade simultaneously. Explicit association methods 41 ; 4 ; 3 establish cross-view correspondences using three types of cues: sparse skeletal keypoints, appearance-based re-identification 12 ; 13 , and epipolar geometric constraints. In team sports, however, all three cues fail simultaneously: keypoints are frequently occluded by other players, appearance features become indistinguishable under identical team jerseys, and epipolar constraints lose discriminative power when cameras are positioned far from the playing field or occluded. Learning-based methods 29 ; 31 ; 20 ; 25 ; 15 ; 21 ; 36 bypass explicit cross-view matching by aggregating multi-view image features within a unified 3D representation space, jointly resolving person detection and cross-view association in a single forward pass. However, they require annotated training data and are sensitive to changes in camera configuration, both of which are practical barriers in team sports where labeled multi-view data are scarce 6 ; 7 ; 17 . Body mesh models, which encode dense geometric structure of the human body, offer a promising alternative cue that is invariant to appearance, yet their potential for cross-view association remains largely untapped. Recent attempts to introduce body mesh priors into this pipeline 5 ; 14 have shown promise in correcting implausible poses and improving cross-view consistency. Nevertheless, existing mesh-aware approaches either invoke the body model only as a post-hoc refinement after association decisions have already been made, or remain unvalidated under the densely crowded, uniform-appearance conditions of competitive team sports. Consequently, robust multi-view multi-person 3D pose estimation in team sports remains an open challenge, as explicit methods break down under uniform appearance and occlusion, while learning-based methods are hindered by scarce annotations and sensitivity to camera configurations. We propose MAEM (Mesh-Aware Epipolar Matching), a training-free framework for multi-view multi-person 3D pose estimation in team sports, to address the challenge of cross-view person matching. The core idea is to use projected dense body mesh vertices as epipolar verification cues for cross-view association, departing from conventional explicit methods that rely on sparse skeletal keypoints or appearance features, both of which fail under occlusion and uniform team jerseys. Dense mesh vertices provide substantially richer geometric coverage of the human body surface, making epipolar correspondences robust precisely where keypoint-based approaches break down. Although body mesh models have been applied to pose refinement and mesh fusion in prior work, their direct use as a cross-view association primitive in a training-free setting has not been explored, nor validated under the densely crowded, uniform-appearance conditions of competitive team sports. The main contributions of this work are as follows: \bullet A pipeline that directly exploits projected dense monocular mesh vertices as epipolar verification cues within the cross-view association stage, enabling robust multi-person matching and generalizing across team sports scenarios. \bullet A two-stage matching mechanism combining a coarse bounding-box center reprojection filter and a mesh-vertex epipolar distance filter, which together suppress false correspondences more effectively than sparse-keypoint alternatives, as confirmed by ablation studies. \bullet MAEM achieves the lowest MPJPE of 74.0 mm on the Human-M3 basketball scenarios benchmark 7 and 59.8 mm on the Sportcenter dataset 6 under uniform-appearance conditions, with MPJPE reductions of 19.1% and 9.0% over the top-performing training-free association baseline, respectively, demonstrating consistent generalization across indoor and outdoor team sports scenarios under varying camera configurations, without any target-domain network training. 2 Related work 2.1 Explicit association multi-view 3D pose estimation Early approaches to multi-view multi-person 3D pose estimation 2 ; 3 ; 41 ; 4 adopt a sequential bottom-up pipeline that explicitly matches detected 2D observations across camera views before reconstructing 3D poses. Belagiannis et al. 2 introduce a 3D Pictorial Structures model that represents the human body as a set of rigid parts connected by pairwise potentials, enabling joint inference of part locations across multiple views within a unified probabilistic framework. By reasoning over individual body parts rather than holistic detections, this approach tolerates partial occlusion more gracefully than whole-body matching methods. Dong et al. 4 propose a multi-way matching algorithm to cluster detected 2D poses across all views by combining appearance-based re-identification features 12 ; 13 and epipolar geometric constraints, from which the 3D pose can be effectively inferred through triangulation. Whereas this approach is fast and robust against missing and false detections, it remains fundamentally sensitive to the quality of upstream 2D pose detections, and appearance-based matching degrades significantly when individuals share similar visual attributes. Furthermore, epipolar constraints derived from sparse skeletal keypoints lose discriminative power under heavy occlusion, as partially visible skeletons yield ambiguous geometric evidence. Zhang et al. 41 unify per-view parsing, cross-view matching, and temporal tracking into a single 4D association graph optimization framework, treating each dimension of image space, viewpoint, and time equally and simultaneously. Although this formulation enables more coherent multi-person tracking across frames, the joint optimization of

Read the complete text