Learning a General Model: Folding Clothing with Topological Dynamics

Yiming Liu, Lijun Han, Enlin Gu, Hesheng Wang, Senior Member, IEEE · 2025

Read Learning a General Model: Folding Clothing with Topological Dynamics by Yiming Liu, Lijun Han, Enlin Gu, Hesheng Wang, Senior Member, IEEE (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-20720.txt.

Introduction

Folding clothes is easy for people and hard for robots, since garments crumple into countless shapes. Yiming Liu and co-authors propose a general model for the task in this 2025 paper, built on the folding structure that stays visible as cloth moves. They turn that structure into a compact graph that stands for the garment's state.

Image segmentation sorts out which parts hide which, and keypoint detection anchors the graph. A graph neural network then learns how the graph evolves, and its predictions feed a control law through a deformation matrix. Tests with jackets show the system can read and fold complex clothing even when parts cover each other. The authors aim at one model that folds many garments, not one trick per item.

Catalog record

CREATORYiming Liu, Lijun Han, Enlin Gu, Hesheng Wang, Senior Member, IEEE
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-20720

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning a General Model: Folding Clothing with Topological Dynamics Yiming Liu, Lijun Han, Enlin Gu, Hesheng Wang, Senior Member, IEEE Abstract The high degrees of freedom and complex structure of garments present significant challenges for clothing manipulation. In this paper, we propose a general topological dynamics model to fold complex clothing. By utilizing the visible folding structure as the topological skeleton, we design a novel topological graph to represent the clothing state. This topological graph is low-dimensional and applied for complex clothing in various folding states. It indicates the constraints of clothing and enables predictions regarding clothing movement. To extract graphs from self-occlusion, we apply semantic segmentation to analyze the occlusion relationships and decompose the clothing structure. The decomposed structure is then combined with keypoint detection to generate the topological graph. To analyze the behavior of the topological graph, we employ an improved Graph Neural Network (GNN) to learn the general dynamics. The GNN model can predict the deformation of clothing and is employed to calculate the deformation Jacobi matrix for control. Experiments using jackets validate the algorithm’s effectiveness to recognize and fold complex clothing with self-occlusion. I Introduction Clothing manipulation is an intricate household task. Equipping robots with the capability to manipulate soft fabrics will enhance their intelligence and improve the quality of people’s lives. However, the inherent complexity of clothing presents challenges. As a pliable object, fabric possesses infinite degrees of freedom and often undergoes significant deformation with self-occlusion. Early approach [ 1 ] utilizes mesh-based model to describe the clothing state, whose efficiency is limited by complex dynamics. Furthermore, unlike simpler fabrics such as towels, clothing is not a flat 2D structure. Even when fully unfolded, it still retains Local overlap, further complicating the analysis of its behavior. Although recent research has made progress in tasks such as hanging [ 2 ] , unfolding [ 3 ] , and folding [ 4 ] , most methods are limited to simple cloth like towels or T-shirts. The geometries of these clothes are relatively rectangular and approximated as one-layer. Methods for handling complex clothing with long sleeves and other attached parts are still under development. For developing clothing manipulation algorithms, it is essential to understand how humans manipulate clothing. Instead of modeling the garment’s deformation precisely, we extract key features by analyzing the garment’s structure. By predicting the behavior of those topological features, we can develop a manipulation policy for clothing deformation tasks. This topology-based approach, which includes perception, modeling, and control of soft objects, has already been applied in the manipulation of deformable linear objects [ 5 ] . However, extracting topological features from clothing is more challenging, limiting further application. Fig. 1: We propose a new topology-based representation of clothing states. By characterizing clothing occlusions and keypoints, this representation can be extracted and generated from complex clothing in different folding states. A general GNN is applied to obtain the dynamics model of the topological graph. In this paper, we propose a topology-based method for clothing folding. By characterizing the topological skeleton of clothing, we can develop a topological representation for clothing folding (see Fig. 1 ). This representation reduces the feature dimension and adapts to complex garments in various folding states. Specifically, by combining keypoint detection with occlusive layer semantic segmentation, we present the process for generating topological graphs from different folding states and under self-occlusion conditions. Considering the sparsity of the 2D topology skeleton, we use Graph Neural Network (GNN) and improve the message propagation mechanism to learn the dynamics of clothing. An additional aggregation and update mechanism for nodes within the same region is introduced to enhance network efficiency and model the general dynamics of topological graphs. Based on the extraction of topological graphs and their general dynamics, a Jacobian-based controller is designed to execute the folding task in stages. In summary, the contributions of this paper are as follows: • We define a topological graph that can represent complex clothing in various folding states. By combining keypoint detection and occlusive layer semantic segmentation, this graph can be extracted from self-occluded garments. • We propose a general dynamics model for the topological graph to predict the deformation of clothing. This model enables the Jacobi-based control, which can be applied to fold complex clothing. • We developed the platform and conducted the experiments on folding jackets. The results validate the effectiveness of our perceiving and controlling method. II Related Work Clothing Feature Perception: The infinite dimensionality and self-occlusion of clothing present significant challenges for clothing feature perception. Traditionally, geometric features, such as corner points [ 6 ] and clothing seams [ 7 ] , have been employed in state perception. The application of these geometric features is limited by the lack of relationships between different features, making clothing modeling challenging. Leveraging the DeepFashion dataset [ 8 ] , approaches such as [ 9 ] and [ 10 ] detect the corresponding semantic keypoints and construct topological skeletons. However, under self-occlusion, the difficulty to identify clothing categories and the lack of the keypoints limit its application. Recently, the normalized canonical space is proposed to reconstruct and track the clothing meshes from point clouds [ 11 ] , [ 12 ] . These methods still rely on the priors of clothing categories and are only trained in simulation. In this paper, we propose a novel algorithm for keypoint detection and topological relationship generation that is independent of clothing categories. Cloth Manipulation: To achieve the desired cloth configuration, cloth manipulation research can be classified into two categories: model-free methods and model-based methods. Model-free methods employ reinforcement learning [ 13 ] , [ 14 ] or imitation learning [ 15 ] to learn the mapping functions from perception to action. However, These algorithms are tailored to specific tasks and struggle to generalize across different types of fabrics and operations. On the other hand, Model-based methods represent states using particles [ 16 ] or images [ 17 ] , employing analysis techniques such as MSD [ 18 ] and PBD [ 16 ] , or network architectures like LSTM [ 19 ] and GNN [ 20 ] to learn state transition relationships. Through optimization control, these methods can generate the optimal control policy. However, due to the high dimensionality of state representation, model-based methods require significant computational resources, limiting their applicability to fabrics with complex structures. In this paper, we propose a new model-based clothing folding method that leverages topological graphs with low dimension. III Proposed Method We study the task of folding complex garments. Initially, the clothing is allowed to be unfolded entirely while not being flattened. It exposes the overall structure but still exhibits wrinkles and may be self obstructed, with the sleeves’ position remaining uncertain. These conditions impose challenges on the detection of keypoints and the accurate matching of the clothing outline template. Fig. 2: Overview. (a) In the workspace, an RGB image observation \boldsymbol{o}_{t} is captured by the camera. (b) Semantic keypoints and occlusive layer semantic segmentation are extracted from the observation and matched to construct the topological graph, G_{t} . (c) During manipulation, the tracked keypoints are utilized to refine G_{t} via a GNN-based corrective model, while the new state, G_{t+1} is predicted by a GNN-based

Read the complete text