Continuous-Time SO(3) Forecasting with Savitzky–Golay Neural Controlled Differential Equations

Lennart Bastian 1,2* Mohammad Rashed 1,2* Nassir Navab 1,2 Tolga Birdal 3 1 Technical University of Munich 2 Munich Center of Machine Learning 3 Imperial College London · 2025

Read Continuous-Time SO(3) Forecasting with Savitzky–Golay Neural Controlled Differential Equations by Lennart Bastian 1,2* Mohammad Rashed 1,2* Nassir Navab 1,2 Tolga Birdal 3 1 Technical University of Munich 2 Munich Center of Machine Learning 3 Imperial College London (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06780.txt.

Introduction

Lennart Bastian and co-authors forecast how objects rotate, learning the motion as a smooth flow through time. Their paper appeared in 2025. Rotation tracking is core to vision and robotics, but sensors are noisy and sparse, motions are complex, and users want long forecasts. The team models rotation with neural controlled differential equations, a method that follows a guiding path, here smoothed with a classic Savitzky-Golay filter, while respecting the true geometry of three-dimensional rotations.

Unlike simpler methods that assume plain motion patterns, theirs learns a general hidden dynamical system behind the observed trajectory. On real-world data it forecast rotations more convincingly than existing approaches. The authors aim at robots and cameras that must predict where a turning object will face next.

Catalog record

CREATORLennart Bastian 1,2* Mohammad Rashed 1,2* Nassir Navab 1,2 Tolga Birdal 3 1 Technical University of Munich 2 Munich Center of Machine Learning 3 Imperial College London
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-06780

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Continuous-Time SO(3) Forecasting with Savitzky–Golay Neural Controlled Differential Equations Lennart Bastian 1,2* Mohammad Rashed 1,2* Nassir Navab 1,2 Tolga Birdal 3 1 Technical University of Munich 2 Munich Center of Machine Learning 3 Imperial College London Abstract Tracking and forecasting the rotation of objects is fundamental in computer vision and robotics, yet S ​ O ​ ( 3 ) SO(3) extrapolation remains challenging as (1) sensor observations can be noisy and sparse, (2) motion patterns can be governed by complex dynamics, and (3) application settings can demand long-term forecasting. This work proposes modeling continuous-time rotational object dynamics on S ​ O ​ ( 3 ) SO(3) using Neural Controlled Differential Equations guided by Savitzky-Golay paths. Unlike existing methods that rely on simplified motion assumptions, our method learns a general latent dynamical system of the underlying object trajectory while respecting the geometric structure of rotations. Experimental results on real-world data demonstrate compelling forecasting capabilities compared to existing approaches. † † ∗ Equal Contribution. 🖂 : {lennart.bastian,m.rashed}@tum.de 1 Introduction Predicting the 3D rotation of objects is essential for numerous robotics applications, including object manipulation, grasping, and autonomous navigation. Tracking scenarios frequently involve temporary occlusions and detection failures, making accurate forecasting of rotational motion important for maintaining consistent object states. A major challenge lies in representing the non-Euclidean structure of the rotation manifold \mathrm{SO}(3) , where even simple object motions can exhibit complex behavior Borisov and Mamaev 2018 . Unlike translational tracking in Euclidean space, rotational tracking must account for the non-linear nature of rotational motion governed by objects’ moment of inertia. This becomes challenging when tracking systems produce noisy measurements due to occlusions or detection failures Fan et al. 2025 ; Park et al. 2008 ; Gugushvili and Klaassen 2012 . Figure 1 : We propose Savitzy Golay neural Controlled Differential Equations (SG-nCDEs), learning a latent representation of \mathrm{SO}(3) dynamics with respect to a robust control path \mathbf{p}(t;\cdot) with coefficients \mathbf{\rho} . The control path is constructed directly on the manifold \mathrm{SO}(3) by filtering noisy input observations, enabling robust forecasting of future object orientations \mathbf{\hat{y}}_{k} . Our model derives the object dynamics directly from noisy pose estimates \mathbf{\tilde{x}}_{k}\in\mathrm{SO}(3) , without restrictive assumptions such as constant angular velocity. We propose Savitzy Golay neural Controlled Differential Equations (SG-nCDEs) for rotational forecasting by modeling object dynamics through a latent dynamical system that respects the manifold structure of \mathrm{SO}(3) . Unlike previous methods that rely on simplified assumptions like constant velocity or minimal torque Mur-Artal et al. 2015 , our approach can predict rotational trajectories for complex object dynamics from noisy, sparse observations. Our proposed method addresses existing limitations through a geometry-aware neural network that learns a continuous-time path in \mathrm{SO}(3) , enabling long-term forecasting under noisy conditions. This supports advancements across multiple domains from robotic manipulation Marturi et al. 2019 ; Wu et al. 2022 , autonomous navigation Ettinger et al. 2021 , and tracking with sensor fusion Fan et al. 2025 ; Bosse and Zlot 2009 . Our contributions include: • A general data-driven approach for modeling \mathrm{SO}(3) object dynamics by integrating a latent dynamical system with neural controlled differential equations (nCDEs). • Leveraging a robust integration control signal on the Lie group \mathrm{SO}(3) that handles noisy input states while respecting the geometric structure of rotations. • Experimental validation on real-world tracking data demonstrate superior performance, particularly in challenging scenarios with sensor noise and occlusions. 2 Related Works We review relevant literature on \mathrm{SO}(3) filtering for tracking, rotation representations, and trajectory forecasting methods. \mathrm{SO}(3) Filtering for Tracking Robustly filtering rotational measurements is essential for tracking applications from video stabilization Jia and Evans 2014 to robot localization Tang et al. 2019 ; Wong et al. 2020 ; Talbot et al. 2024 and sensor fusion Nirmal et al. 2016 ; Busam et al. 2017 . Filtering approaches using splines Kim et al. 1995 ; Lovegrove et al. 2013 ; Haarbach et al. 2018 must preserve manifold properties of \mathrm{SO}(3) to avoid numerical errors Busam et al. 2017 , but their solutions cannot be computed in closed form Sommer et al. 2020 . Without explicit assumptions, splines diverge near interpolation boundaries Persson et al. 2021 ; Sommer et al. 2020 , leading many tracking systems to extrapolate with constraints such as constant velocity or minimal torque Mur-Artal et al. 2015 . Savitzky-Golay filtering on \mathrm{SO}(3) offers an efficient alternative by regressing polynomials in the tangent space and mapping back to the manifold via the exponential map Jongeneel and Saccon 2022 . This enables simultaneous angular velocity and acceleration estimation, recently exemplified for hand tracking Fan et al. 2025 . Other data-driven methods like Gaussian Processes Lang et al. 2014 ; Lang and Hirche 2017 ; Lilge and Barfoot 2024 offer principled uncertainty quantification but have limited representational capacity compared to deep neural networks and require careful kernel design for \mathrm{SO}(3) . Rotation Representations Learning 3D rotations presents unique challenges. Classical approaches like Euler angles Kundu et al. 2018 or quaternions Zhao et al. 2020 suffer from singularities or topological constraints that hinder direct regression Zhou et al. 2019 ; Brégier 2021 ; Geist et al. 2024 and backpropagation Chen et al. 2022 ; Teed and Deng 2021 . This stems from \mathrm{SO}(3) not being homeomorphic to any subset of 4D Euclidean space Zhou et al. 2019 . Recent work demonstrates that these limitations significantly impact downstream tasks Zhou et al. 2019 ; Levinson et al. 2020 ; Geist et al. 2024 , motivating higher-dimensional representations like 6D Zhou et al. 2019 , 9D Levinson et al. 2020 , and 10D Peretroukhin et al. 2020 parameterizations with better numerical stability. Trajectory Forecasting Neural Ordinary Differential Equations (neural ODEs) solve initial value problems using neural networks Chen et al. 2018 , with extensions like GRU-ODEs Rubanova et al. 2019 ; De Brouwer et al. 2019 incorporating sequential data more effectively. Neural Controlled Differential Equations (Neural CDEs) Kidger et al. 2020 enable more flexible conditioning by integrating the hidden state with a smooth signal. Most forecasting approaches for tracking focus on predicting Euclidean position, with rotational forecasting receiving less attention despite its importance for complete 6D pose tracking. Some existing works Byravan and Fox 2017 address transformation prediction directly from point cloud sequences, but do not address longer time horizons. Others assume conservational motion which is inherently limiting Mason et al. 2023 . Our approach addresses these limitations by learning to forecast rotational trajectories directly from noisy pose data while respecting the geometric structure of \mathrm{SO}(3) , and avoiding restrictive assumptions on the underlying trajectory. 3 Preliminaries SO(3) Representation While positions and rotations of tracked objects each have three degrees of freedom, the space of rotations forms a non-commutative Lie group, making rotational tracking inherently more complex. We begin by introducing key background and definitions. Definition 1 (Special Orthogonal Group) . The special orthogonal group \mathrm{SO}(3) represents the set of all rotations about the origin in \mathbb{R}^{3} : \mathrm{SO}(3)=\left\{\mathbf{R}\in\mathbb{R}^{3\times 3}:\mathbf{R}^{\top}\mathbf{R}=\mathbf{I},\operatorname{det}(\mathbf{R})=1\right\} (1) The Lie algebra \mathfrak{so}(3) , the tangent space at the identity of \mathrm{SO}(3) , consists of real 3\times 3 skew-symmetric matrices. Definition 2 (Hat Operator) . Any \boldsymbol{\omega}\in\mathbb{R}^{3} can be associated with an element \hat{\boldsymbol{\omega}}\in\mathfrak{so}(3) via the hat operator: \displaystyle\hat{\boldsymbol{\omega}}=S(\boldsymbol{\omega})=\begin{pmatrix}0&-\omega_{3}&\omega_{2}\\ \omega_{3}&0&-\omega_{1}\\ -\omega_{2}&\omega_{1}&0\\ \end{pmatrix} (2) The time evolution of a

Read the complete text