Geometric Visual Servo Via Optimal Transport
Ethan Canzini, Simon Pope, Ashutosh Tiwari · 2025
Read Geometric Visual Servo Via Optimal Transport by Ethan Canzini, Simon Pope, Ashutosh Tiwari (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02768.txt.
Introduction
Ethan Canzini, Simon Pope, and Ashutosh Tiwari steer robot arms by camera through optimal transport, in work from 2025. Visual servoing moves a gripper from a start pose to a target pose on camera cues. Classic control leans on hand tuned feature picks and treats detected features as exact points.
Their law treats camera data as a probability spread over three dimensional pose space. Wasserstein distance between current and target spreads plays the role of shortest path error. A standard drive law plus gravity offset then chases that error down along curved flows. Pose based and image based servoing both fit inside the one frame. Tests from varied start poses show the law generalises across setups.
Catalog record
| CREATOR | Ethan Canzini, Simon Pope, Ashutosh Tiwari |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-02768
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Geometric Visual Servo Via Optimal Transport Ethan Canzini, Simon Pope, Ashutosh Tiwari Abstract When developing control laws for robotic systems, the principle factor when examining their performance is choosing inputs that allow smooth tracking to a reference input. In the context of robotic manipulation, this involves translating an object or end-effector from an initial pose to a target pose. Robotic manipulation control laws frequently use vision systems as an error generator to track features and produce control inputs. However, current control algorithms don’t take into account the probabilistic features that are extracted and instead rely on hand-tuned feature extraction methods. Furthermore, the target features can exist in a static pose thus allowing a combined pose and feature error for control generation. We present a geometric control law for the visual servoing problem for robotic manipulators. The input from the camera constitutes a probability measure on the 3-dimensional Special Euclidean task-space group, where the Wasserstein distance between the current and desired poses is analogous with the geometric geodesic. From this, we develop a controller that allows for both pose and image-based visual servoing by combining classical PD control with gravity compensation with error minimization through the use of geodesic flows on a 3-dimensional Special Euclidean group. We present our results on a set of test cases demonstrating the generalisation ability of our approach to a variety of initial positions. Keywords: Robot Control; Geometric Modelling; Optimal Control; System Dynamics 1 Introduction As the proliferation of robotics across a variety of fields increases, the development of feedback control laws remains a key research challenge. Many robotic platforms rely on vision systems to produce stable feedback laws [ 1 , 2 , 3 ] , as this sensing modality leverages a balance between cost, efficiency, and safety in a variety of environments and applications [ 4 ] . Connecting the vision input to a stable feedback law remains a key challenge in developing autonomous systems that are capable of dealing with uncertainty in their environmental observations. Visual servoing refers to a wide domain of control architectures that generate an error signal based on the tracking of image features of components. By relating the visual features to a desired pose, this form of control allows for the computation of error signals that incorporate either the pose or the camera features and track to the desired goal [ 5 ] . Visual servoing-based control remains a key part of the robotics control community, with applications ranging from manipulators [ 6 ] , underwater autonomous vehicles [ 7 ] and spacecraft orbital rendezvous [ 8 ] , where the interaction matrix \mathbf{L}_{e} can be tuned depending on the environment that the robot is operating in. However, these extracted features are purely based on camera parameters which can lead to noisy estimations of the camera pose and requires human and domain-specific knowledge to fine-tune which specific features are being tracked [ 9 , 10 ] . Furthermore, the use of visual servo control with system dynamics requires the use of inverse kinematics and coordinate-level proportional-derivative control which can infringe on performance guarantees [ 11 ] . Additionally, many system dynamics lose the benefits of the physical characteristics of the robotic platforms during the derivation process [ 12 ] . Generally speaking, visual servoing takes two different forms: • Pose-Based Visual Servo (PBVS) which relates the extracted features to a specific pose and computes the difference between the desired and current pose • Image-Based Visual Servo (IBVS) where the extracted features are tracked during control towards the desired feature set. These features are normally extracted using geometric representations of the images and therefore lack the the granularity and feature set that is required for high-accuracy tasks [ 13 ] . For vision-based feedback, the goal is to minimise the error e(t) defined as: e(t)=s(m(t),a)-s^{\ast}, (1) where m(t) represents the camera measurements and a are other intrinsic camera properties that are based on pre-defined photographic parameters. Note that, compared to most standard feedback control literature, the error is calculated in the reverse manner. For most approaches, including the one put forward in this work, the target camera features s^{\ast} are kept constant and motionless during operation. Visual servoing has a rich research history in the field of robot control, with applications ranging from terrestrial robotics in manufacturing environments [ 14 , 15 ] , to controlling underwater autonomous vehicles [ 7 ] , to endoscopic surgery [ 16 ] and the docking and manipulation of non-cooperative space satellites [ 8 , 17 ] . Initial work was focused on better ways to extract and track features during operation, with an emphasis on using RGB images to generate and track features that are present in the frame [ 18 , 19 ] . As the demand for more accurate feature tracking increased, multi-camera systems emerged as a way to improve the feature accuracy of the tracking system [ 20 ] . However, the lack of accurate distance estimation from monocular camera fusion remained a key problem. The proliferation and improvement in RGB-D cameras allowed the development of algorithms that utilise point clouds as features, allowing the leveraging of distance data to create a depth map of features [ 21 , 22 , 23 ] . The evolution of visual servoing control has led to its formulation as an optimal control problem. This has led to its use within model predictive control (MPC) frameworks, with further implementations looking to embed robust control to improve the accuracy under feature uncertainty [ 6 , 24 , 25 ] . Control barrier functions provide an additional level of robustness to the visual servo control [ 26 ] , but struggle to compute optimal feedback gains in high-dimensional environments. Machine learning (ML) methods seek to alleviate this curse of dimensionality with self-supervised learning and human demonstrations providing reference trajectories to generate optimal controls [ 27 , 28 ] . Recent advances in the use of depth camera technology have enabled the development of control laws that use depth inputs to generate point clouds of physical features. This proves beneficial when dealing with complex geometries that are difficult to define or when working in cluttered environments [ 22 , 29 ] . These point clouds can be defined as probability measures, whereby metrics can be evaluated across them to find displacements across spaces [ 30 , 31 ] . Optimal transport (OT) allows the computation of distance metrics between these measures [ 32 ] , and can be used to generate a variety of different motion paths for autonomous systems [ 33 ] . More recently, thanks to the landmark work by Benamou and Brenier [ 34 ] , OT can be formulated dynamically to generate flow maps that are represented by a set of partial differential equations. This has seen a prevalence in control theory applications, where it can be beneficial to represent states as probability measures and find the optimal coupling between them [ 35 ] . This approach can be seen as an energy-based control law [ 36 ] , and can be used for averaging control of particles [ 37 ] and for steering control of stochastic linear systems [ 38 , 39