Not-So-Optimal Transport Flows for 3D Point Cloud Generation

Ka-Hei Hui, Chao Liu, Xiaohui Zeng, Chi-Wing Fu, Arash Vahdat · 2025

Read Not-So-Optimal Transport Flows for 3D Point Cloud Generation by Ka-Hei Hui, Chao Liu, Xiaohui Zeng, Chi-Wing Fu, Arash Vahdat (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12456.txt.

Introduction

Ka-Hei Hui, Chao Liu, Xiaohui Zeng, Chi-Wing Fu, and Arash Vahdat present a 2025 method for generating 3D point clouds, the scatter of dots that sketches out a shape. A key trait of such clouds is that shuffling the dots changes nothing, and a good generator must honor that. A recent family of flow models did so, but it scaled badly to large clouds.

The team traces the trouble to flow paths that must be straightened early in training, which makes the model hard to learn. Their fix precomputes a rough transport plan offline, giving cheap training pairs. A mixed pairing during training eases the target further. On a standard shape test set, the model beat prior diffusion and flow methods at both free generation and shape completion.

Catalog record

CREATORKa-Hei Hui, Chao Liu, Xiaohui Zeng, Chi-Wing Fu, Arash Vahdat
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-12456

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Not-So-Optimal Transport Flows for 3D Point Cloud Generation Ka-Hei Hui, Chao Liu, Xiaohui Zeng, Chi-Wing Fu, Arash Vahdat Abstract Learning generative models of 3D point clouds is one of the fundamental problems in 3D generative learning. One of the key properties of point clouds is their permutation invariance, i.e., changing the order of points in a point cloud does not change the shape they represent. In this paper, we analyze the recently proposed equivariant OT flows that learn permutation invariant generative models for point-based molecular data and we show that these models scale poorly on large point clouds. Also, we observe learning (equivariant) OT flows is generally challenging since straightening flow trajectories makes the learned flow model complex at the beginning of the trajectory. To remedy these, we propose not-so-optimal transport flow models that obtain an approximate OT by an offline OT precomputation, enabling an efficient construction of OT pairs for training. During training, we can additionally construct a hybrid coupling by combining our approximate OT and independent coupling to make the target flow models easier to learn. In an extensive empirical study, we show that our proposed model outperforms prior diffusion- and flow-based approaches on a wide range of unconditional generation and shape completion on the ShapeNet benchmark. 1 Introduction Generating 3D point clouds is one of the fundamental problems in 3D modeling with applications in shape generation, 3D reconstruction, 3D design, and perception for robotics and autonomous systems. Recently, diffusion models ( Sohl-Dickstein et al., 2015 ; Ho et al., 2020 ) and flow matching ( Lipman et al., 2022 ) have become the de facto frameworks for learning generative models for 3D point clouds. These frameworks often overlook 3D point cloud permutation invariance, implying the rearrangement of points does not change the shape that they represent. In closely related areas, equivariant optimal transport (OT) flows ( Klein et al., 2024 ; Song et al., 2024 ) have been recently developed for 3D molecules that can be considered as sets of 3D atom coordinates. These frameworks learn permutation invariant generative models, i.e. , all permutations of the set have the same likelihood under the learned generative distribution. They are trained using optimal transport between data and noise samples, yielding several key advantages including low sampling trajectory curvatures, low-variance training objectives, and fast sample generation ( Pooladian et al., 2023 ) . Albeit these theoretical advantages, our examination of these techniques for 3D point cloud generation reveals that they scale poorly for point cloud generation. This is mainly due to the fact that point clouds in practice consist of thousands of points whereas molecules are assumed to have tens of atoms in previous studies. Solving the sample-level OT mapping between a batch of training point clouds and noise samples is computationally expensive. Conversely, ignoring permutation invariance when solving batch-level OT ( Pooladian et al., 2023 ; Tong et al., 2023 ) fails to produce high-quality OT due to the excessive possible permutations of point clouds. Figure 1: Different coupling types between Gaussian noise (left) and point clouds (right), where coupled noise and surface points share the same color: (a) Independent Coupling randomly maps noises to point clouds. (b) Minibatch OT computes OT map in batches of noises and point clouds. (c) Equivariant OT follows the similar minibatch OT but aligns points via permutation. (d) Our approach precomputes dense OT on data and noise supersets, then subsamples it to couple point clouds with slightly perturbed noise. Note that only (c) and (d) can produce high-quality OT. In this paper, we propose a simple and scalable generative model for 3D point cloud generation using flow matching, coined as not-so-optimal transport flow matching , as shown in Fig 1 . We first propose an efficient way to obtain an approximate OT between point cloud and noise samples. Instead of searching for an optimal permutation between point cloud and noise samples online during training, which is computationally expensive, we show that we can precompute an OT between a dense point superset and a dense noise superset offline. Since subsampling a superset preserves the underlying shape, we can simply subsample the point superset and obtain corresponding noise from the precomputed OT to construct a batch of noise-data pairs for training the flow models. Figure 2: In the OT flow model, the vector field {\mathbf{v}}_{t}({\mathbf{x}}_{0}) admits a large change in its output with a small perturbation of {\mathbf{x}}_{0} at t{=}0 . We demonstrate that our approximate OT reduces the pairwise distance between data and noise significantly and benefits from the advantages of OT flows, e.g. , straightness of trajectories and fast sampling. However, a careful examination shows that learning (equivariant) OT flows is generally challenging since straightening flow trajectories makes the learned flows complex at the beginning of the trajectory. Intuitively, in the OT coupling, the flow model should be able to switch between different target point clouds (i.e., different modes in the data distribution) with small variations in their input, making the flow model have high Lipchitz (see Fig 2 ). To remedy this, we propose a simple approach to construct a less “optimal” hybrid coupling by blending our approximate OT and independent coupling used in the flow matching model. In particular, we suggest perturbing the noise samples obtained from our approximate OT with small Gaussian noise. While this remedy makes our mapping less optimal from the OT perspective, we show that it empirically shows two main advantages: First, the target flow model is less complex and the generated points clouds have high sample quality. Second, when reducing the number of inference steps, the generation quality still degrades slower than other competing techniques, indicating smoother trajectories. In summary, this paper makes the following contributions: (i) We show that existing OT approximations either scale poorly or produce low-quality OT for real-world point cloud generation. (ii) We show that albeit the nice theoretical advantages, equivariant OT flows have to learn a complex function with high Lipchitz at the beginning of the generation process. (iii) To tackle these issues, we propose a not-so-optimal transport flow matching approach that involves an offline superset OT precomputation and online random subsampling to obtain an approximate OT, and adds a small perturbation to the obtained noise during training. (iv) We empirically compare our method against diffusion models, flows, and OT flows on unconditional point cloud generation and shape completion on the ShapeNet benchmark. We show that our proposed model outperforms these frameworks for different sampling budgets over various competing baselines on the unconditional generative task. In addition, we show that we can obtain reasonable generation quality on the shape completion task in less than five steps, which is challenging for other approaches. 2 Related Works Score-based Generative Models. Recently, diffusion models ( Sohl-Dickstein et al., 2015 ; Ho et al., 2020 ; Song & Ermon, 2019 ) have gained popularity in generating data of various formats, especially images ( Rombach et al., 2022 ; Ramesh et al., 2022 ) and videos ( Blattmann et al., 2023 ; Brooks et al., 2024 ) . These generative models employ an iterative process to transform a known distribution, typically a Gaussian, into a desired data distribution. Song et al. (2020) demonstrate that these models

Read the complete text