RapidPoseTriangulation: Multi-view Multi-person Whole-body Human Pose Triangulation in a Millisecond

Daniel Bermuth, Alexander Poeppel, Wolfgang Reif · 2025

Read RapidPoseTriangulation: Multi-view Multi-person Whole-body Human Pose Triangulation in a Millisecond by Daniel Bermuth, Alexander Poeppel, Wolfgang Reif (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21692.txt.

Introduction

Daniel Bermuth, Alexander Poeppel, and Wolfgang Reif present a fast pose method in a 2025 paper. Cameras in several spots can track how people move, but joining the views into one body pose is slow and hard. Their method aims to do that step in about a millisecond.

The method tracks many people at once and covers the whole body. It picks up fine detail, from face cues to finger moves, across views. Tests on data sets the method had not seen still gave strong results, which hints it can adapt to new rooms and setups. All of the work is shared in public for others to build on.

Catalog record

CREATORDaniel Bermuth, Alexander Poeppel, Wolfgang Reif
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-21692

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RapidPoseTriangulation: Multi-view Multi-person Whole-body Human Pose Triangulation in a Millisecond Daniel Bermuth, Alexander Poeppel, Wolfgang Reif Abstract The integration of multi-view imaging and pose estimation represents a significant advance in computer vision applications, offering new possibilities for understanding human movement and interactions. This work presents a new algorithm that improves multi-view multi-person pose estimation, focusing on fast triangulation speeds and good generalization capabilities. The approach extends to whole-body pose estimation, capturing details from facial expressions to finger movements across multiple individuals and viewpoints. Adaptability to different settings is demonstrated through strong performance across unseen datasets and configurations. To support further progress in this field, all of this work is publicly accessible. 1 Introduction Figure 1 : Example of a multi-person whole-body pose estimation from multiple camera views in a volleyball game (from the egohumans dataset [ 16 ] ). On top, the full image of one camera with the projections of all the detected poses, on the bottom-left a zoom-in on one player, and on the bottom-right her detected 3D pose. The ability to accurately detect and track human body positions and movements is an important task in many advanced computer vision applications. From enhancing virtual reality experiences to improving the collaboration with humans in robotic systems, precise human poses play a crucial role at connecting digital systems with the physical world. Human pose estimation, which involves determining the spatial locations of different body joints, has seen significant progress in recent years. While traditional approaches relied on wearable markers for precise tracking, the field has increasingly moved towards marker-less methods using standard cameras. This shift offers greater flexibility and usefulness, especially in scenarios where attaching markers is impractical or impossible, such as in public spaces or specialized environments like operating rooms. However, marker-less pose estimation, especially with only a single viewpoint, presents its own set of challenges. Occlusions, varying lighting conditions, and the complexity of human movements can all impact the accuracy and reliability of these systems. To address external and self-occlusion issues, and to some extent also varying lighting, multi-view approaches have become more common. By capturing the scene from multiple angles simultaneously, these systems can overcome many of those limitations. Recent advances in deep learning have dramatically improved the accuracy of pose estimation from individual camera views. However, the task of efficiently combining these multiple 2D estimates into accurate 3D poses, particularly for multiple people in real-time scenarios, remains an active area of research. Current approaches often show notable performance drops when applied to setups they were not trained on, and are normally too slow for real-time usage [ 2 ] . Furthermore, as applications become more advanced, there is an increasing demand for a more detailed whole-body pose estimation, including facial expressions and especially hand gestures (see Figure 1 ). This work addresses these challenges by introducing a novel algorithm for multi-view multi-person human pose estimation. The approach is designed to be both fast and capable of generalizing well across different datasets and application setups. Unlike most previous works, the algorithm is also able to estimate whole-body poses. In a broad evaluation across multiple datasets, it is shown that the proposed algorithm is more reliable than existing methods, while also being significantly faster, which makes it especially suitable for real-time applications. The demonstration that a lightweight algebraic method can outperform many modern learned approaches also raises an important question: Is the current trend toward increasingly complex and novelty-oriented learnable architectures truly the most effective way to improve performance in multi-view pose estimation? To further support future research in all directions, the source-code of the presented algorithm is made available at: https://gitlab.com/Percipiote/ 2 Related Work Most approaches address the problem of human pose estimation in two separate steps. At first, 2D poses are predicted for each image, and in the second step, these are fused to estimate 3D poses. The basic concepts of the different approaches in the second step can be categorized by whether they use algorithmic strategies or learning-based methods. Some of them also make use of temporal information from previous frames to improve the results. On the side of the learning-based methods, VoxelPose [ 24 ] was one of the first concepts, building upon the voxel-based triangulation research of Iskakov et al. [ 13 ] and extending it from single-person to multi-person estimations. Its method projects joint heatmaps from 2D images into a 3D voxelized space, estimates a central point for each individual using a convolution network, extracts a focused cube surrounding each person center, and computes the joint positions using a second net. Faster-VoxelPose [ 28 ] enhanced this technique by restructuring the 3D voxel network into several 2D and 1D projections, thereby boosting efficiency. TesseTrack [ 18 ] and TEMPO [ 8 ] added a temporal dimension into the voxel space to refine poses across sequential frames. Alternative approaches such as PRGnet [ 26 ] employ a graph-based methodology or directly infer 3D poses from 2D features, like in MvP [ 25 ] . SelfPose3d [ 20 ] adopts the framework of VoxelPose and trains both 2D and 3D networks in a self-supervised manner using randomly augmented 2D poses. Regarding algorithmic methodologies, mvpose [ 10 ] solves the estimation problem in two steps. Initially, it identifies corresponding 2D poses across images based on geometric and visual clues, and then triangulates the matching pose groups to calculate the final output. mv3dpose [ 23 ] utilizes a graph-matching approach to assign poses through epipolar geometry and incorporates temporal information to handle missing joint data. PartAwarePose [ 9 ] accelerates the pose-matching process by leveraging poses from preceding frames and applying a joint-based filter to solve keypoint inaccuracies resulting from occlusions. VoxelKeypointFusion [ 2 ] employs a voxel-based triangulation concept to predict 3D joint proposals from overlapping viewpoints. It then uses the joint reprojections to assign them to individuals in the original 2D views, to match them to individual 3D persons before merging them into a final result. Regarding the generalization of learning-based approaches to new datasets, a direct transfer is typically not evaluated. However, VoxelPose , Faster-VoxelPose , MvP , and TEMPO have implemented a finetuning strategy utilizing synthetic data, though only the first two have also published the source-code for this. The idea is to randomly place 3D poses from another dataset in 3D space, and back-project them onto the camera perspectives of the target setup. After adding some augmentations, the network is then trained to learn the 3D reconstruction process. In the closed-source approach of CloseMoCap [ 19 ] this concept was further improved by using a larger 3D pose dataset and additional augmentations. Algorithmic approaches generally evaluated their transferability to a limited number of other datasets, a process that is considerably more straightforward as it doesn’t require any training. 3 RapidPoseTriangulation The new algorithm called RapidPoseTriangulation follows a simple and learning-free triangulation concept. 3.1 Algorithm Similar to most other approaches, the algorithm can be split into two stages as well, with the first one predicting the 2D poses for each image. For this any 2D pose estimator can be integrated, here RTMPose [ 14 ] was used. The second stage can be split into the following steps: 1. Create all possible pairs

Read the complete text