FCBV-Net: Category-Level Robotic Garment Smoothing via Feature-Conditioned Bimanual Value Prediction
Mohammed Daba, Jing Qiu · 2025
Read FCBV-Net: Category-Level Robotic Garment Smoothing via Feature-Conditioned Bimanual Value Prediction by Mohammed Daba, Jing Qiu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-05153.txt.
Introduction
Mohammed Daba and Jing Qiu teach a robot to smooth garments it has never seen before. Their 2025 paper presents FCBV-Net, a network that values two-handed smoothing moves from three-dimensional point clouds. Clothes vary widely within a type, and systems often memorize one item instead of learning the category.
The network leans on frozen geometric features learned beforehand, so its sense of shape stays steady across new garments. Only the action-scoring part trains for the task. In simulated tests, efficiency on unseen garments dropped just eleven percent, versus ninety-six percent for an image-based rival. Final coverage reached eighty-nine percent, beating a correspondence-based method with the same shape features.
Catalog record
| CREATOR | Mohammed Daba, Jing Qiu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-05153
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FCBV-Net: Category-Level Robotic Garment Smoothing via Feature-Conditioned Bimanual Value Prediction Mohammed Daba, Jing Qiu Abstract Category-level generalization for robotic garment manipulation, such as bimanual smoothing, remains a significant hurdle due to high dimensionality, complex dynamics, and intra-category variations. Current approaches often struggle, either overfitting with concurrently learned visual features for a specific instance or, despite Category-level perceptual generalization, failing to predict the value of synergistic bimanual actions. We propose the Feature-Conditioned bimanual Value Network (FCBV-Net), operating on 3D point clouds to specifically enhance category-level policy generalization for garment smoothing. FCBV-Net conditions bimanual action value prediction on pre-trained, frozen dense geometric features, ensuring robustness to intra-category garment variations. Trainable downstream components then learn a task-specific policy using these static features. In simulated GarmentLab experiments with the CLOTH3D dataset, FCBV-Net demonstrated superior category-level generalization. It exhibited only an 11.5% efficiency drop (Steps80) on unseen garments compared to 96.2% for a 2D image-based baseline, and achieved 89% final coverage, outperforming an 83% coverage from a 3D correspondence-based baseline that uses identical per-point geometric features but a fixed primitive. These results highlight that the decoupling of geometric understanding from bimanual action value learning enables better Category-level generalization. I Introduction Robotic manipulation of garments, crucial for applications such as assistive care and automated dressing [ 1 , 2 ] , presents significant challenges such as high deformability, complex dynamics, and near-infinite dimensional configuration spaces [ 3 ] . Furthermore, items within the same category can display substantial intra-category variations in shape, size, and material, making robust manipulation a difficult task [ 4 ] . A key step in many garment handling pipelines, such as preparing a shirt for a person to wear, involves transforming an arbitrarily crumpled garment into a smoothed, predictable state (see Fig. 1 ). Fig. 1: Garment Smoothing is a crucial step in many downstream garment manipulation tasks (e.g., folding, dressing, etc.). It involves transforming a garment from a random state into a known, structured configuration. The core problem this paper addresses is enabling robots to perform bimanual garment smoothing efficiently across a specific garment category, not just on specific instances seen during training, but with strong generalization to unseen garments within the same category . Existing approaches often struggle with this generalization. Methods that learn action-value functions directly can optimize bimanual coordination but may rely on learned state representations that overfit to training instances [ 5 , 6 , 7 ] . Conversely, while correspondence-based methods have good category-level generalization [ 4 ] , they typically use policy transfer and do not directly predict the outcome quality of executing a specific bimanual action primitive. To address these limitations, we propose the Feature-Conditioned Bimanual Value Network (FCBV-Net) . FCBV-Net operates directly on 3D point cloud and introduces a novel approach to bimanual action value prediction. The central hypothesis is that explicitly conditioning an action’s value on robust, pre-trained per-point geometric features can dramatically improve generalization. We use dense features that are pre-trained for structural understanding and then keep them frozen during policy learning. This strategy decouples geometric understanding from learning task-specific interaction values. As a result, the system can effectively learn to anticipate synergistic outcomes and generalize across unseen garment geometries within a category. This strategy aims to decouple the learning of fundamental geometric understanding from the learning of task-specific, interaction-aware action values. The main contributions of this work are: • The proposal of FCBV-Net, a novel architecture for category-level bimanual garment smoothing that conditions value prediction for actions using pre-trained, frozen per-point geometric features. • A demonstration, through simulation in the GarmentLab environment using the CLOTH3D dataset, that FCBV-Net achieves superior generalization performance on unseen ‘tops’ category garments compared to a state-of-the-art 2D image-based baseline and a 3D correspondence-based policy transfer method. II RELATED WORK II-A Garment State Representation Early approaches often used 2D image features like corners and wrinkles [ 8 , 9 ] . The adoption of 3D point clouds provided richer geometric input [ 4 , 7 ] . Beyond raw data, structured representations like skeletons or keypoints capture garment topology [ 10 , 11 ] , though sparse features may lack detail for precise control. Consequently, dense representations, including learned visual correspondences [ 12 , 13 ] often via self-supervised contrastive learning [ 4 ] , or dense visual affordances [ 12 ] , have become prominent. To leverage the detail of dense representations while ensuring generalization, FCBV-Net utilizes 3D point clouds processed into pre-trained, frozen per-point geometric features, providing a robust, deformation-invariant foundation that is crucial for informing downstream value prediction across varied garment instances. II-B Learning Manipulation Policies Early methods for garment smoothing included heuristics [ 14 , 15 ] or goal-conditioned learning [ 16 ] . Learning from Demonstration (LfD) frequently used to learn policies [ 17 ] , Yet, these techniques necessitate several interactions. Learning action-value functions (Q-functions) using self-supervised learning are increasingly applied to unfolding and smoothing [ 18 , 5 ] . These may predict value maps for single-point [ 18 ] or bimanual interactions [ 5 , 7 ] , with training combining supervised signals and self-exploration. To manage large action spaces, strategies include ranking sparse keypoint candidates [ 7 ] or learning pair correspondence by sampling [ 5 ] . To address the risk of overfitting while still optimizing for complex interactions, FCBV-Net learns a Q-function conditioned on pre-trained features. This decoupling of geometry from policy learning allows the network to find optimal synergistic actions without being confined to fixed primitives or overfitting to training instances. II-C Category-Level Generalization Many prior efforts were instance or task-specific [ 5 , 6 ] , with limited transferability. As generalization in this case is challenging due to intra-category variations in shape, size, material, and topology. Strategies for generalization include learning dense visual correspondences [ 17 ] , often via self-supervision on 3D model datasets [ 4 ] , and generalizable structural representations like skeletal models [ 10 ] . FCBV-Net targets improved category-level generalization by conditioning learned action-values on robust, pre-trained dense visual correspondences features. II-D Simulation for Garment Manipulation Simulation enables large-scale data collection for policy learning [ 17 , 18 , 19 ] . While general simulators like PyBullet [ 20 ] have been adapted, specialized deformable object simulators like SoftGym [ 21 ] , using NVIDIA FleX [ 22 ] under the hood are common. However, these engines are frequently exhibiting a significant sim-to-real gap as a result of the lack of thorough sim-to-real algorithm designs. Modern unified environments such as GarmentLab [ 23 ] offer multi-physics (PBD, FEM), GPU acceleration, sim-to-real algorithm avoidance, and leveraging Isaac Sim platforms [ 24 ] . To ensure high-fidelity physical interactions and build upon a platform designed to mitigate the sim-to-real gap, our work is implemented and validated entirely within the modern, unified GarmentLab environment. III PROBLEM STATEMENT Let the state of a garment at time t be represented by a 3D point cloud observation \mathcal{O}_{t}\in\mathbb{R}^{N\times 3} , where N is the number of points. The primary objective is to determine a sequence of actions \{a_{0},a_{1},\ldots,a_{k-1}\} to transit the garment from an arbitrary initial configuration \mathcal{O}_{0} to a smoothed, flattened state \mathcal{O}_{k} that maximizes our objective reward, see equation ( 4 ).