MFM-point: Multi-scale Flow Matching for Point Cloud Generation

Petr Molodyk, Jaemoo Choi, David W. Romero, Ming-Yu Liu, Yongxin Chen, [3pt] Georgia Institute of Technology, NVIDIA {pmolodyk3, jchoi843, yongchen}@gatech.edu · 2025

Read MFM-point: Multi-scale Flow Matching for Point Cloud Generation by Petr Molodyk, Jaemoo Choi, David W. Romero, Ming-Yu Liu, Yongxin Chen, [3pt] Georgia Institute of Technology, NVIDIA {pmolodyk3, jchoi843, yongchen}@gatech.edu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-20041.txt.

Introduction

Petr Molodyk, Jaemoo Choi, David W Romero, Ming-Yu Liu, and Yongxin Chen present MFM-Point, a multi-scale flow matching method for generating point clouds. Direct point-based generators are cheap to train and simple, but they usually trail methods that work through latent features, meshes, or voxels. The authors aim to close that gap without losing the simplicity.

Generation proceeds from coarse to fine, which improves quality and scale at no extra training or inference cost. A structured downsampling and upsampling plan preserves the geometry of unordered point sets and keeps the resolutions aligned. In tests, MFM-Point leads the point-based methods and rivals the best representation-based ones, with strong results on multi-category and high-resolution tasks.

Catalog record

CREATORPetr Molodyk, Jaemoo Choi, David W. Romero, Ming-Yu Liu, Yongxin Chen, [3pt] Georgia Institute of Technology, NVIDIA {pmolodyk3, jchoi843, yongchen}@gatech.edu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-20041

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MFM-point: Multi-scale Flow Matching for Point Cloud Generation Petr Molodyk, Jaemoo Choi, David W. Romero, Ming-Yu Liu, Yongxin Chen, [3pt] Georgia Institute of Technology, NVIDIA {pmolodyk3, jchoi843, yongchen}@gatech.edu Abstract In recent years, point cloud generation has gained significant attention in 3D generative modeling. Among existing approaches, point-based methods directly generate point clouds without relying on other representations such as latent features, meshes, or voxels. These methods offer low training cost and algorithmic simplicity, but often underperform compared to representation-based approaches. In this paper, we propose MFM-Point , a multi-scale Flow Matching framework for point cloud generation that substantially improves the scalability and performance of point-based methods while preserving their simplicity and efficiency. Our multi-scale generation algorithm adopts a coarse-to-fine generation paradigm, enhancing generation quality and scalability without incurring additional training or inference overhead. A key challenge in developing such a multi-scale framework lies in preserving the geometric structure of unordered point clouds while ensuring smooth and consistent distributional transitions across resolutions. To address this, we introduce a structured downsampling and upsampling strategy that preserves geometry and maintains alignment between coarse and fine resolutions. Our experimental results demonstrate that MFM-Point achieves best-in-class performance among point-based methods and challenges the best representation-based methods. In particular, MFM-point demonstrates strong results in multi-category and high-resolution generation tasks. Figure 1 : Overview of the proposed multi-scale point cloud generation framework. The framework consists of two processes: (1) Training (shown in brown) and (2) Inference (shown in green). During training, we perform downsampling to obtain the coarse representation X^{k+1}_{e}=\text{Down}(X^{k}_{e}) , and train a flow model v^{k}_{\theta} to transport samples from the distribution of X^{k}_{e} at each stage k . During inference, we first sample points through the learned flow v^{k+1}_{\theta} from X^{k+1}_{s} to X^{k+1}_{e} , and then upsample the result to reconstruct X^{k}_{s} , the input for the next finer stage. We employ an equal-size K-means clustering strategy as the downsampling operator \text{Down}(\cdot) , followed by a carefully designed upsampling procedure that ensures distributional alignment between the upsampled samples and the corresponding fine-scale distribution. 1 Introduction 3D point cloud generation is a fundamental problem in 3D generative modeling with broad applications in shape generation, 3D reconstruction, and autonomous robotics. Among various modeling paradigms, point-based methods 49 ; 54 ; 17 ; 22 have been extensively studied for their ability to directly generate 3D points without relying on intermediate representations such as latents, meshes, or voxels. These point-based approaches are appealing due to their low training cost, as they avoid expensive 3D voxelization and complex encoder–decoder architectures. They also offer algorithmic simplicity, making them both easy to implement and computationally efficient. This simplicity further facilitates the study of geometric priors and generative behavior, enabling the analysis of modeling principles without the confounding influence of heavy architectural components ( 14 ; 15 ) . Over the years, a variety of point-based generative frameworks have been explored, ranging from variational autoencoders (VAEs) 23 ; 22 and generative adversarial networks (GANs) 9 ; 48 to, more recently, normalizing flows 51 ; 42 ; 26 , autoregressive models 44 ; 18 ; 27 , diffusion and Flow Matching (FM) approaches 54 ; 47 ; 49 ; 11 ; 17 . However, point-based methods often underperform compared to representation-based approaches 47 ; 53 ; 36 , which can better capture global structures at the cost of greater model complexity and computational overhead. To overcome these limitations, we propose MFM-Point, a novel multi-scale Flow Matching (FM) framework for 3D point cloud generation. While prior point-based methods primarily focused on improving pair-matching strategies and loss formulations 11 ; 17 , our work takes an orthogonal direction by enhancing scalability and performance through a multi-scale generation framework. Specifically, MFM-Point generates point clouds in several stages of increasing resolution in a coarse-to-fine manner, where an independent flow model is trained at each stage to generate samples at the corresponding resolution. This progressive generation strategy enables MFM-Point to efficiently handle high-resolution and multi-category tasks. Each flow model is trained using the flow matching objective, which reduces generation to a simple regression problem. Consequently, our training pipeline remains conceptually simple and computationally efficient, while maintaining strong expressivity to tackle complex generation tasks. A key challenge in our multi-scale design lies in constructing geometrically consistent coarse-to-fine generation trajectories, as illustrated in Figure 1 . This means that fine-grained stages should use the coarse-grained information learned from previous resolutions and further refine the point cloud by gradually increasing the fidelity while preserving the overall geometry learned in the early stages. We discover that this can be achieved by using a structured point cloud upsampling and downsampling algorithm that can transfer the information from the low resolution point cloud to the next model when switching between stages during inference. To acheve this, we derive the theoretical requirements placed on the upsampling and downsampling algorithms used to align the point clouds between consecutive stages, and develop an algorithm that satisfies these requirements. We establish the effectiveness of MFM-Point through extensive experiments on high-resolution and multi-category point cloud generation. We demonstrate that our framework is both highly scalable and capable of producing diverse high-fidelity samples. Furthermore, our comprehensive ablation studies show that these performance gains stem directly from the proposed multi-scale architecture, highlighting the benefits of our coarse-to-fine design in improving both quality and efficiency. Additionally, we demonstrate class-conditional point cloud generation to underscore the flexibility and generalizability of MFM-Point across different generative settings. Our main contributions are summarized as follows: • We propose MFM-Point, a scalable multi-scale Flow Matching framework for point cloud generation. Its coarse-to-fine generation paradigm enables generation in complex settings while preserving the algorithmic simplicity and efficiency of point-based models. • We derive the theoretical requirements for cross-stage alignment, providing a principled mechanism for consistent multi-scale generation. We offer a specific implementation by designing new downsampling and upsampling operators that preserve local geometry, thereby improving overall generation performance. • Our method achieves best-in-class performance among point-based methods and comparable results to that of the best representation-based approaches, with particularly strong results in multi-category and high-resolution scenarios. Extensive ablation and further experiments validate the effectiveness and generalizability of our framework. 2 Background Flow Matching Flow matching models ( 29 ; 30 ; 31 ) aim to learn a velocity field v:[0,T]\times\mathbb{R}^{d}\rightarrow\mathbb{R}^{d} that transports a given source distribution \mu to a target distribution \nu through an ordinary differential equation (ODE). Given any joint distribution \gamma\in\Pi(\mu,\nu) , where \Pi is the joint distribution of distributions \mu and \nu , the flow matching model ( 29 ) minimizes a simple regression objective: \mathbb{E}_{(x_{0},x_{1})\sim\gamma}\left[\|v_{\theta}(t,x_{t})-(x_{1}-x_{0})\|^{2}\right], (1) where x_{t}=(1-t)x_{0}+tx_{1} for t\in[0,1] . Under mild regularity conditions, the optimal velocity field v_{\theta}(t,\cdot) transports the source distribution \mu to the target \nu via the following ODE dynamics: \vskip-5.0pt\dot{X}_{t}=v_{\theta}(t,X_{t}),\hskip 10.00002ptX_{0}\sim\mu. (2) In generative modeling, we typically set the source distribution as a standard Gaussian, \mu=\mathcal{N}(0,I) , and the target distribution as the data distribution, \nu=p_{\text{data}} . Then, the probability of x_{t} given x_{1}\sim p_{\text{data}} is given by: x_{t}=tx_{1}+(1-t)n\sim\mathcal{N}(tx_{1},(1-t)^{2}I), (3) where n\sim\mathcal{N}(0,I) . Similar to diffusion models ( 12 ; 43 ) , flow matching models rely on a simple regression objective, making them easy and stable to train. Multi-Scale Flow Matching for Image Generation. Recent progress in flow matching

Read the complete text