UniStateDLO: Unified Generative State Estimation and Tracking of Deformable Linear Objects Under Occlusion for Constrained Manipulation
Kangchen Lv*, Mingrui Yu*, Shihefeng Wang, Xiangyang Ji, Xiang Li · 2025
Read UniStateDLO: Unified Generative State Estimation and Tracking of Deformable Linear Objects Under Occlusion for Constrained Manipulation by Kangchen Lv*, Mingrui Yu*, Shihefeng Wang, Xiangyang Ji, Xiang Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-17764.txt.
Introduction
Kangchen Lv and four co-authors give robots a way to keep track of a cable even when most of it is hidden. Ropes, wires, and hoses flop into endless shapes, and in a cramped workspace other objects block the camera's view. A robot that loses the cable's shape mid-task cannot plan its next grip.
UniStateDLO treats seeing as educated guessing: a diffusion model generates the full cable shape that best fits the visible fragments, for one frame and across a whole motion. It copes with self-hiding bends and multi-object pileups. Training used only synthetic scenes, yet the system transferred to real robots with no retraining. Plugged into a working manipulation loop, it kept control stable in clutter. Cable handling is a quiet bottleneck in factories and surgery alike.
Catalog record
| CREATOR | Kangchen Lv*, Mingrui Yu*, Shihefeng Wang, Xiangyang Ji, Xiang Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 87 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-17764
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
UniStateDLO: Unified Generative State Estimation and Tracking of Deformable Linear Objects Under Occlusion for Constrained Manipulation Kangchen Lv*, Mingrui Yu*, Shihefeng Wang, Xiangyang Ji, Xiang Li Abstract Perception of deformable linear objects (DLOs), such as cables, ropes, and wires, focuses on accurately and robustly estimating their 3-D states, which is the cornerstone for successful downstream manipulation. Although vision-based methods have been extensively explored, they remain highly vulnerable to occlusions that commonly arise in constrained manipulation environments due to surrounding obstacles, large and varying deformations, and limited viewpoints. Moreover, the high dimensionality of the state space, the lack of distinctive visual features, and the presence of sensor noises further compound the challenges of reliable DLO perception. To address these open issues, this paper presents UniStateDLO, the first complete DLO perception pipeline with deep-learning methods that achieves robust performance under severe occlusion, covering both single-frame state estimation and cross-frame state tracking from partial point clouds. Both tasks are formulated as conditional generative problems, leveraging the strong capability of diffusion models to capture the complex mapping between highly partial observations and high-dimensional DLO states. UniStateDLO effectively handles a wide range of occlusion patterns, including initial occlusion, self-occlusion, and occlusion caused by multiple objects. In addition, it exhibits strong data efficiency as the entire network is trained solely on a large-scale synthetic dataset, enabling zero-shot sim-to-real generalization without any real-world training data. Comprehensive simulation and real-world experiments demonstrate that UniStateDLO outperforms all state-of-the-art baselines in both estimation and tracking, producing globally smooth yet locally precise DLO state predictions in real time, even under substantial occlusions. Its integration as the front-end module in a closed-loop DLO manipulation system further demonstrates its ability to support stable feedback control in complex, constrained 3-D environments. The project page is available at https://unistatedlo.github.io/ . Fig. 1: We propose UniStateDLO , a novel unified perception framework for deformable linear objects (DLOs) that supports both single-frame state estimation and cross-frame tracking of DLOs under severe occlusions. Leveraging diffusion-based generative modeling, UniStateDLO reconstructs complete DLO configurations from even highly partial point clouds with strong accuracy, robustness and real-time performance. Trained entirely on synthetic data, it generalizes in a zero-shot manner to diverse real-world DLOs and provides a reliable perception front-end for constrained manipulation tasks. I Introduction Deformable linear objects (DLOs), including ropes, wires, and cables, are one-dimensional deformable structures that frequently appear in manufacturing, service, and surgical applications [ 65 , 59 , 15 ] . Unlike rigid objects, the shape of DLOs will vary along their length due to bending and deformation. Enabling robotic systems to automatically manipulate DLOs in tasks, such as shape control [ 61 , 34 ] , cable routing [ 27 , 32 ] , and knot tying [ 36 , 26 ] , fundamentally relies on accurate and real-time DLO state estimation, which serves as the cornerstone for closed-loop control. Although many approaches have been developed in recent years to improve DLO perception, the infinite-dimensional state space and frequent occlusions in constrained environments make it still a challenging research issue. First, the state space of DLO possesses nearly infinite degrees of freedom under deformation. In practice, its state is often simplistically represented by a discretized chain of uniformly distributed nodes [ 23 , 34 ] , yet this representation still results in hundreds or even thousands of dimensions. Moreover, occlusions caused by obstacles or self-interactions, particularly during manipulation in constrained environments, demand greater robustness in perception systems to reliably reconstruct the full DLO state from partial observations. A complete 3-D DLO state estimation pipeline can be roughly divided into three stages: segmentation (i.e., segmenting pixel-level DLO masks from scenes [ 5 , 4 , 62 ] ), detection (i.e., estimating the DLO state in a single frame [ 57 , 28 ] ), and tracking (i.e., capturing deformations across sequential frames [ 58 , 54 , 56 ] ). For single-frame estimation, existing approaches often begin by extracting DLO skeleton lines from 2-D images [ 28 , 29 ] or 3-D point clouds [ 55 , 64 ] , followed by merging disconnected segments through manually designed strategies. Some researchers also incorporate data-driven methods [ 33 ] to enhance the performance under occlusion or complex topology. For cross-frame tracking, a classical line of work formulates the problem as non-rigid point set registration with multiple geometric constraints [ 40 , 45 , 46 , 9 , 54 , 56 ] , where DLO nodes are modeled as centroids of a Gaussian Mixture Model (GMM), and the observed point cloud is treated as samples drawn from it. Some recent works also explore to perform DLO tracking with particle filtering [ 58 ] and 3-D Gaussian splatting techniques [ 12 ] . However, previous single-frame estimation approaches only exploit individual frames and neglect temporal continuity, whereas tracking approaches critically depend on accurate and robust initialization. In constrained manipulation scenarios, the common occurrence of long-term and large-scale occlusions further degrades their performance. To address these challenges, we introduce UniStateDLO, a unified framework that formulates both DLO state estimation and tracking under occlusions as a conditional generation task. Motivated by the remarkable capability of diffusion models [ 20 ] in learning complex probabilistic distributions, we hypothesize that they can effectively resolve the uncertainty of DLO nodes given partial observations. Intuitively, one could condition the diffusion model on a global embedding extracted from the DLO point cloud, which then samples 3-D node locations through iterative denoising. However, due to the weak visual distinctiveness of DLO point clouds, such global features often lack the fine-grained, node-wise geometric cues, which motivates conditioning the model on richer local features to fully leverage the potential of generative modeling. For DLO state estimation, we propose a novel two-branch network architecture with a diffusion-based fusion module to generate the final 3-D node predictions. Both branches share a PointNet++ [ 37 ] encoder but focus on complementary information: one leverages global features to achieve robustness under occlusions, while the other exploits local features to ensure precise node-wise estimation. By using the 3-D node predictions from both branches as per-node local conditions for the diffusion model, our approach can reconstruct occluded portions of the DLO while maintaining high local accuracy. Once the state estimation is done in the initial frame, inter-frame node motions are predicted iteratively based on the previous frame’s results to enable sequential tracking. K-nearest-neighbor-based feature aggregation module is employed to extract per-node features around last-frame nodes, which provide local conditions for the subsequent diffusion model in a manner consistent with the single-frame estimation framework. By leveraging generative modeling through diffusion models in both single-frame estimation and cross-frame tracking, UniStateDLO can imagine complete DLO configurations from even heavily occluded point clouds, delivering a robust perception module for precise and reliable DLO manipulation in constrained scenarios. Our model is trained on a large-scale synthetic dataset of 300K samples and generalizes to real-world DLOs with substantially different physical properties in a zero-shot manner. Across both simulation and real-world evaluations with diverse occlusion patterns, our approach consistently outperforms existing baselines. Its deployment as the real-time perception front-end in challenging shape control tasks within constrained environments, where multiple obstacles introduce large-scale, long-term occlusions and require continuous collision avoidance, further demonstrates its effectiveness.