UPTor: Unified 3D Human Pose Dynamics and Trajectory Prediction for Human-Robot Interaction

Nisarga Nilavadi, Andrey Rudenko, Timm Linder · 2025

Read UPTor: Unified 3D Human Pose Dynamics and Trajectory Prediction for Human-Robot Interaction by Nisarga Nilavadi, Andrey Rudenko, Timm Linder (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14866.txt.

Introduction

Nisarga Nilavadi, Andrey Rudenko, and Timm Linder present UPTor, a unified system forecasting both human body pose and walking trajectory, in a two thousand twenty-five robotics paper. Robots sharing space with people need both predictions at once, yet most studies tackle only one.

UPTor starts from a short sequence of observed poses. An off-the-shelf three-dimensional pose estimator feeds a graph attention network encoding the skeleton, then a compact non-autoregressive transformer predicts future keypoints in global coordinates. The team also releases DARKO, a dataset of navigation activities for human-aware mobile robots. Evaluations on Human3.6M, CMU-Mocap, and DARKO show accurate real-time predictions.

Catalog record

CREATORNisarga Nilavadi, Andrey Rudenko, Timm Linder
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-14866

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

UPTor: Unified 3D Human Pose Dynamics and Trajectory Prediction for Human-Robot Interaction Nisarga Nilavadi, Andrey Rudenko, Timm Linder Abstract We introduce a unified approach to forecast the dynamics of human keypoints along with the motion trajectory based on a short sequence of input poses. While many studies address either full-body pose prediction or motion trajectory prediction, only a few attempt to merge them. We propose a motion transformation technique to simultaneously predict full-body pose and trajectory key-points in a global coordinate frame. We utilize an off-the-shelf 3D human pose estimation module, a graph attention network to encode the skeleton structure, and a compact, non-autoregressive transformer suitable for real-time motion prediction for human-robot interaction and human-aware navigation. We introduce a human navigation dataset “DARKO” with specific focus on navigational activities that are relevant for human-aware mobile robot navigation. We perform extensive evaluation on Human3.6M, CMU-Mocap, and our DARKO dataset. In comparison to prior work, we show that our approach is compact, real-time, and accurate in predicting human navigation motion across all datasets. Result animations, our dataset, and code will be available at https://nisarganc.github.io/UPTor-page/ I Introduction Human motion forecasting and activity recognition is a critical component for social robots and autonomous systems that need to operate and interact with people in domestic and industrial environments [ 1 ] . Accurate trajectory prediction in crowded environments has strong impact on the effectiveness and safety of a mobile robot, allowing smooth and unobtrusive navigation [ 2 ] . Further refining trajectory predictions with full-body poses provides complete information about human behaviour with many promising applications in human-robot interaction [ 3 , 4 ] , automated driving [ 5 ] , surveillance [ 6 ] and healthcare [ 7 ] . Trajectory Prediction [ 8 , 9 , 10 , 11 , 12 , 13 ] focuses on the coarse-level root joint trajectories of vehicles, pedestrians or other dynamic agents, dealing with broader movement patterns in a global coordinate space [ 1 ] . Full-body Pose Prediction , also referred to as Human Motion Prediction [ 14 , 15 , 16 , 17 , 18 ] , on the other hand, deals with the fine-grained prediction of 3D skeleton joints coordinates, relative to the fixed root joint (e.g. hip or torso) resulting in pose prediction in a local coordinate space [ 3 ] without considering the global translation. Historically speaking, the research in trajectory prediction and full-body pose prediction, with a few notable exceptions [ 19 , 3 , 20 ] , progressed independently and targeted distinct application scenarios. However, solving these tasks in a unified manner is advantageous for effective human-robot interaction. Incorporating the full-body pose features can refine dynamics modeling in trajectory prediction [ 21 , 1 ] . Similarly, full-body pose prediction in global coordinate space aids robot response in approach and handover applications, allowing to better plan the avoidance maneuvers in close proximity [ 22 ] . Prior art addressed the problem of pose and trajectory prediction as two separate tasks [ 23 , 16 ] or solved it in a decoupled manner with separate modules for each task [ 24 , 3 ] . Furthermore, only a handful of works [ 3 ] specifically focused on navigation activities, which are of critical interest for predictive planning of a mobile robot, over more diverse actions without distinct locomotion. This research gap is also reflected in the prior art datasets of full-body motion, which are often dominated by static activities such as bending, reaching, handing over, standing up, etc. Fig. 1: Human key points estimated from the robot’s perception stack and human motion prediction in global 3D coordinates from our UPTor model. In this work we develop an approach for Unified human Pose and Trajectory prediction (“UPTor”) for real-time and safe human-aware motion planning of our intralogistics robot 3 , see Fig. 1 . We encode the skeleton features with a graph attention network and utilize a non-autoregressive Transformer model [ 25 , 16 ] to process input pose sequences in 3D global coordinates. To support this form of input and output, we propose a motion transformation technique and train the transformer on sequences within the transformed coordinate space. We evaluate our method on the H3.6M [ 26 ] and CMU-Mocap [ 27 ] datasets, and contribute a novel DARKO dataset of 17 subjects performing 508 sequences of diverse navigation-related activities. Through qualitative and quantitative experiments, we demonstrate that our approach is compact, fast, and accurate in full-body pose and trajectory prediction of navigating humans for robotic applications. In summary, we make the following contributions: • Human motion prediction with global translation in global 3D coordinates using our motion transformation technique, which can be applied to any local pose prediction method to achieve the rare and unexplored unified task of pose and trajectory prediction; • A compact, accurate, and real-time human navigation prediction approach that is well-suited for robotics applications; • A human navigation dataset with diverse actors collected using the 3D perception stack of a mobile robot. II Related Work Many human motion prediction methods in the computer vision community focus on accurate root-relative pose predictions and excel at generating poses in a stochastic manner within local space [ 23 , 28 , 29 , 30 ] . Similarly, many trajectory prediction studies from autonomous systems community focus on predicting plausible trajectories in unstructured environment [ 9 , 10 ] . Predicting human poses and trajectories together to forecast complete human motion in global space is a relatively new but actively developing research direction [ 5 , 24 , 30 , 3 , 20 , 31 , 19 , 32 ] . Traditionally, RNNs were used to generate sequential predictions [ 33 , 34 ] . Yet, their inherent nature of processing data sequentially can be a bottleneck, especially in real-time applications. Aiming for a fast and accurate solution for applications on a mobile robot, in this work we explore a non-autoregressive variant of Transformer, which performs simultaneous decoding of the entire output sequence in a single step, as in the Pose Transformer (POTR) [ 16 ] . POTR is specifically designed for root-relative pose prediction and thus removes global translation from human motion sequences. STPOTR [ 3 ] extends POTR with a second transformer for trajectory prediction and introduces shared attention layers between the pose and trajectory transformers. Since STPOTR predicts poses and trajectory in a decoupled fashion, 3D human poses are pre-processed to separate the global movement from poses by subtracting pose dynamics joints from the root joint. Predicted pose and trajectory key-points are later concatenated to obtain the global pose over time. DeRPoF [ 24 ] also adopts a decoupled approach with an LSTM-based encoder-decoder for trajectory forecasting and a Variational Autoencoder (VAE) encoder-decoder to predict root-relative pose dynamics. DMMGAN [ 20 ] is an accurate but computationally-expensive model designed for predicting diverse human motions, including both trajectories and poses, using a Transformer-attention based network. Diversifying Latent Flows (Dlow) [ 23 ] aims to generate multiple hypotheses of human poses utilizing a pre-trained deep generative model. Our approach directly utilizes the 3D joint positions from an off-the-shelf 3D human pose estimation module [ 35 ] . We propose

Read the complete text