PAWS : P erception of A rticulation in the W ild at S cale from Egocentric Videos
Yihao Wang 1,6,*, Yang Miao 2,*, Wenshuai Zhao 1,6, Wenyan Yang 1, Zihan Wang 1, Joni Pajarinen 1, Luc Van Gool 2,3, Danda Pani Paudel 2, Juho Kannala 1,7, Xi Wang 3,4,5, † \dagger, Arno Solin 1,6 · 2026
Read PAWS : P erception of A rticulation in the W ild at S cale from Egocentric Videos by Yihao Wang 1,6,*, Yang Miao 2,*, Wenshuai Zhao 1,6, Wenyan Yang 1, Zihan Wang 1, Joni Pajarinen 1, Luc Van Gool 2,3, Danda Pani Paudel 2, Juho Kannala 1,7, Xi Wang 3,4,5, † \dagger, Arno Solin 1,6 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25539.txt.
Introduction
Wang and co-authors recover how jointed objects move by watching hands use them in daily life videos. Drawers and cupboards reveal their hinges through touch, not labels. PAWS mines large sets of first-person video for those interactions. It pulls out the motion and structure of each jointed item at scale. On HD-EPIC and Arti4D it beats prior methods by a clear margin. The harvested joints also help train 3D predictors and guide robot hands. No costly 3D annotation is needed.
Catalog record
| CREATOR | Yihao Wang 1,6,*, Yang Miao 2,*, Wenshuai Zhao 1,6, Wenyan Yang 1, Zihan Wang 1, Joni Pajarinen 1, Luc Van Gool 2,3, Danda Pani Paudel 2, Juho Kannala 1,7, Xi Wang 3,4,5, † \dagger, Arno Solin 1,6 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66478 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.25539
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PAWS : P erception of A rticulation in the W ild at S cale from Egocentric Videos Yihao Wang 1,6,*, Yang Miao 2,*, Wenshuai Zhao 1,6, Wenyan Yang 1, Zihan Wang 1, Joni Pajarinen 1, Luc Van Gool 2,3, Danda Pani Paudel 2, Juho Kannala 1,7, Xi Wang 3,4,5, † \dagger, Arno Solin 1,6 Abstract Articulation perception aims to recover the motion and structure of articulated objects ( e.g . , drawers and cupboards), and is fundamental to 3D scene understanding in robotics, simulation, and animation. Existing learning-based methods rely heavily on supervised training with high-quality 3D data and manual annotations, limiting scalability and diversity. To address this limitation, we propose PAWS, a method that directly extracts object articulations from hand–object interactions in large-scale in-the-wild egocentric videos. We evaluate our method on the public data sets, including HD-EPIC and Arti4D data sets, achieving significant improvements over baselines. We further demonstrate that the extracted articulations benefit downstream tasks, including fine-tuning 3D articulation prediction models and enabling robot manipulation. See the project website at https://aaltoml.github.io/PAWS/ . (a) Egocentric Videos (b) Articulation Perception Robot Manipulation (c) Downstream Applications Articulation Prediction Zero-shot Finetuned + Figure 1 : PAWS : Articulation perception and localization from in-the-wild egocentric videos. (a) From raw videos of human interactions, (b) our method reconstructs the 3D scene and object articulations using hand cues, geometric recovery, and VLM reasoning. (c) These serve as annotations to improve downstream articulation prediction models via finetuning, while also providing 3D priors for real-world robotic manipulation. 1 Introduction Interactions with articulated objects occur everywhere in daily life, from opening a fridge to get breakfast to opening a laptop in the office, and closing a bedside drawer before sleeping. While current scene understanding enables functional object segmentation [ 62 , 10 ] and scene graph construction [ 20 , 92 ] , it remains insufficient for agents to reliably interact with articulated parts in the scene ( e.g . , doors, drawers, and lids) and their interaction interfaces ( e.g . , knobs and handles). Studying scene-level articulation is therefore a crucial component of scene understanding, with broad applications in 3D animation and simulation [ 86 ] , object generation [ 44 , 54 , 8 , 27 , 40 ] , and robotic manipulation [ 2 , 36 , 55 ] . Recently, learning-based articulation prediction [ 31 , 45 , 73 ] and generation-based [ 42 ] methods have been proposed. These approaches are typically trained on data sets annotated with articulation information, requiring large-scale labeled data to generalize to unseen objects and scenes. However, collecting and manually annotating such data sets is costly and labor-intensive. While several object-level articulation data sets have been introduced [ 41 , 17 ] , scene-level data sets remain limited in scale [ 56 , 12 , 23 ] . Other methods infer articulation from multi-view observations or temporal data, such as capturing start and end states from image pairs or 3D structure [ 32 , 45 ] , or leveraging demonstration videos containing articulated motion [ 67 , 34 , 91 , 4 , 82 ] . Most of these methods still rely on carefully collected data with multi-stage articulation observations, stable lighting conditions, limited occlusions, or additional sensing, such as depth [ 82 ] , significantly hindering scalability and real-world deployment. In contrast, in-the-wild egocentric videos [ 18 , 19 , 78 , 64 ] are highly diverse, large-scale, and naturally contain rich real human-object interaction information and object kinematic structure. Despite these advantages, leveraging in-the-wild egocentric videos for articulation understanding remains non-trivial. These videos are monocular, noisy, sometimes poorly illuminated, and unstructured, making conventional multi-view or temporal methods inapplicable. We address these limitations by introducing a training-free approach, PAWS, that infers scene-level articulation directly from in-the-wild monocular egocentric videos. Our key insight is that hand-object interactions provide strong physical cues. By integrating estimated 3D hand trajectories with reconstructed scene geometry, we recover plausible articulation motion and kinematic structure in real-world scenes. We further leverage foundation-model priors to improve robustness under the noise and occlusions, and validate the resulting annotations through extensive experiments and downstream robotic evaluation. Our main contributions are as follows: • We introduce a fully automatic, training-free pipeline for scene-level articulation perception from in-the-wild monocular RGB egocentric videos. • We propose a foundation-model-driven approach that combines VLMs and LLMs priors to infer plausible articulation motion and structure without task-specific training. • We demonstrate through extensive experiments that our method consistently outperforms strong baselines, including Articulation3D and ArtiPoint. • We validate the effectiveness and generalizability of the proposed pipeline across downstream 3D articulation prediction and real-world robotic manipulation tasks. 2 Related Work Articulation Modeling from Static Input This research direction has seen growing research interest in recent years, since static observations comprise a substantial fraction of real-world visual data. Existing works have explored articulation prediction from 3D geometry, taking static point clouds as input and jointly predicting part segmentation and motion parameters at either the object level [ 80 , 41 , 89 , 13 , 43 , 50 ] or the scene level [ 23 ] . [ 31 , 73 ] infer articulation directly from a single RGB(D) image by detecting openable parts and estimating motion parameters. A separate line of research investigates the generation of articulated 3D objects from image(s) [ 44 , 54 , 5 , 40 ] . For instance, PhysX-3D [ 5 ] extends diffusion-based 3D generation frameworks [ 84 ] to produce simulation-ready articulated assets from a single image. While effective, these methods rely on articulation ground-truth annotations and/or high-fidelity 3D structures for training, which are expensive to obtain in unconstrained real-world environments. The limited availability of such data constrains their scalability and generalization. More fundamentally, methods based on static inputs inherently lack access to object motion cues and must rely on learned priors or category-level regularities. As a result, they struggle to reliably recover articulated motion and kinematic structure, particularly in the presence of complex geometries, occlusions, or novel object categories. In contrast, our approach leverages dynamic observations from in-the-wild egocentric videos, which naturally capture object motion and physical constraints induced by human–object interactions. Articulation Modeling from Dynamic Input Recent work captures articulations from dynamic inputs using 3D structures [ 45 ] , RGB images [ 67 , 37 , 30 , 29 , 74 , 71 , 34 , 85 , 24 , 93 ] , or RGB-D images [ 81 , 52 , 90 , 4 , 82 , 91 , 14 , 26 , 21 ] . RSRD [ 34 ] utilizes a hand model to assist articulated object reconstruction, but its reliance on multi-view image inputs is difficult to satisfy in practice. Tracking-based methods [ 91 , 82 , 51 , 21 ] reconstruct articulation through explicit tracking of openable object parts, which degrades under occlusion, low texture, and motion blur. Another line of work models human–object interactions [ 85 , 24 , 35 , 59 ] , leveraging parametric human body models [ 53 ] as constraints for articulation reasoning. However, these approaches often suffer from reduced reliability due to frequent self-occlusions introduced by the human body during interactions. In contrast, our