Mesh-RFT: Enhancing Mesh Generation via Fine-Grained Reinforcement Fine-Tuning
Jian Liu Jing Xu Song Guo Jing Li Jingfeng Guo Jiaao Yu, Haohan Weng Biwen Lei Xianghui Yang Zhuo Chen Fangqi Zhu, Tao Han Chunchao Guo · 2025
Read Mesh-RFT: Enhancing Mesh Generation via Fine-Grained Reinforcement Fine-Tuning by Jian Liu Jing Xu Song Guo Jing Li Jingfeng Guo Jiaao Yu, Haohan Weng Biwen Lei Xianghui Yang Zhuo Chen Fangqi Zhu, Tao Han Chunchao Guo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16761.txt.
Introduction
Jian Liu and co-authors present Mesh-RFT, a fine-tuning method that polishes computer-generated 3D meshes face by face, in a research paper from 2025. Pre-trained mesh makers inherit data biases and emit flawed shapes. Broad reward signals miss local defects like stray edges.
Their method masks low-quality faces and teaches the model to prefer better ones, region by region. A scoring system grades shape soundness and mesh regularity at whole-object and single-face levels. In tests, the method cut a key shape error by about a quarter over the base model. It also beat whole-object tuning. The work shows local feedback can fix local flaws.
Catalog record
| CREATOR | Jian Liu Jing Xu Song Guo Jing Li Jingfeng Guo Jiaao Yu, Haohan Weng Biwen Lei Xianghui Yang Zhuo Chen Fangqi Zhu, Tao Han Chunchao Guo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-16761
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Mesh-RFT: Enhancing Mesh Generation via Fine-Grained Reinforcement Fine-Tuning Jian Liu Jing Xu Song Guo Jing Li Jingfeng Guo Jiaao Yu, Haohan Weng Biwen Lei Xianghui Yang Zhuo Chen Fangqi Zhu, Tao Han Chunchao Guo Abstract Existing pretrained models for 3D mesh generation often suffer from data biases and produce low-quality results, while global reinforcement learning (RL) methods rely on object-level rewards that struggle to capture local structure details. To address these challenges, we present Mesh-RFT , a novel fine-grained reinforcement fine-tuning framework that employs Masked Direct Preference Optimization (M-DPO) to enable localized refinement via quality-aware face masking. To facilitate efficient quality evaluation, we introduce an objective topology-aware scoring system to evaluate geometric integrity and topological regularity at both object and face levels through two metrics: Boundary Edge Ratio (BER) and Topology Score (TS). By integrating these metrics into a fine-grained RL strategy, Mesh-RFT becomes the first method to optimize mesh quality at the granularity of individual faces, resolving localized errors while preserving global coherence. Experiment results show that our M-DPO approach reduces Hausdorff Distance (HD) by 24.6% and improves Topology Score (TS) by 3.8% over pre-trained models, while outperforming global DPO methods with a 17.4% HD reduction and 4.9% TS gain. These results demonstrate Mesh-RFT’s ability to improve geometric integrity and topological regularity, achieving new state-of-the-art performance in production-ready mesh generation. Figure 1: Representative High-Fidelity Mesh Generation by Mesh-RFT. Gallery of meshes generated from point clouds, demonstrating intricate geometric detail and artist-like aesthetic quality. 1 Introduction 3D polygonal meshes serve as the foundational representation for digital assets in industries such as gaming, film, and product design. Despite their ubiquity, high-quality, topologically optimized meshes—essential for downstream tasks like editing, rigging, and animation—are still predominantly handcrafted by skilled artists. Recent advances in generative models have enabled automated mesh synthesis, significantly reducing the time and expertise required to produce production-ready 3D assets. This democratization of mesh generation broadens access to 3D content creation, empowering non-experts to produce geometrically precise and artistically viable models for applications ranging from immersive media to industrial design. Existing 3D generative models often use intermediate representations like voxels wu2016learning ; wang2017cnn , point clouds luo2021diffusion ; jun2023shap ; qi2017pointnet , or implicit fields chen2019learning ; park2019deepsdf . While these avoid direct mesh generation complexities, post-processing (e.g., Marching Cubes lorensen1998marching ) often introduces topological issues and smoothing. Native mesh generation nash2020polygen is more direct, with recent work using autoregressive models and neural compression (e.g., VQ-VAE siddiqui2024meshgpt ; chen2024meshanything ; weng2024pivotmesh ) or geometric serialization tokenizers (e.g., chen2025meshxl ; chen2024meshanythingv2 ; tang2024edgerunner ; lionar2025treemeshgpt ; weng2024scaling ) for sequence-based generation. However, long sequences for high-resolution meshes can cause structural ambiguities and hallucinations (inconsistent edges, non-manifold vertices, distortions, holes), deviating from geometric constraints or artistic intent, ultimately leading to results that may not align with human aesthetic preferences or intended design. Though truncated training hao2024meshtron helps, autoregressive methods still lack stable generation and high fidelity. Recently, reinforcement learning ( schulman2017proximal ; shao2024deepseekmath ) has emerged as a compelling approach for aligning mesh generation more closely with human preferences. For example, DeepMesh zhao2025deepmesh leverages Direct Preference Optimization (DPO) rafailov2023direct , a simple yet effective preference alignment technique that has also found utility in various other domains she2024mapo ; liu2024enhancing ; zhou2024aligning . Nevertheless, directly applying reinforcement fine-tuning to mesh generation using this method encounters two primary challenges. Firstly, objectively quantifying mesh quality is difficult. DeepMesh relies on manual annotation of preference pairs, which is expensive, time-consuming, introduces subjective bias, and limits the training data to only 5,000 samples, hindering generalization. Secondly, its use of global reward signals fails to capture the local topological variations inherent in 3D meshes. As illustrated in Figure 2 , high-quality and low-quality structures often coexist within a single mesh, leading to training noise due to this mismatch in supervision. Figure 2: High-quality, artist-like structures often coexist with messy, low-quality regions within the same mesh. To overcome these limitations, we introduce Mesh-RFT , a novel framework that combines Masked Direct Preference Optimization (M-DPO) with fine-grained mesh quality evaluation for both global and localized refinement. Unlike prior work using subjective global rewards as supervision signals zhao2025deepmesh , we employ a topology-aware scoring system with automated metrics-Boundary Edge Ratio (BER) and Topology Score (TS)-to objectively evaluate mesh quality at both object and face levels, circumventing the laborious manual annotation efforts. Mesh-RFT further employs a localized optimization mechanism utilizing M-DPO and quality-aware masks to specifically refine defective regions, thereby addressing the coarse supervision of global rewards. Extensive experiments across diverse meshes demonstrate Mesh-RFT’s superior performance, achieving significant improvements over both the pretrain baseline (24.6% HD reduction, 3.8% TS improvement) and global DPO (17.4% HD reduction, 4.9% TS improvement), establishing a new benchmark for accuracy and fidelity in generative mesh modeling. In summary, our contributions are as follows: • We introduce the first fine-grained reinforcement fine-tuning framework, that integrates Masked Direct Preference Optimization (M-DPO) with fine-grained mesh quality evaluation. • We devise an objective topology-aware scoring system for evaluating mesh quality, eliminating dependency on manual annotation and addressing subjectivity and scalability limitations. • We propose a novel localized alignment mechanism that optimizes deficient regions geometrically and topologically via quality-aware masks, bridging the gap between global and local supervision. • Experiments demonstrate that our method achieves state-of-the-art performance in high-fidelity 3D mesh generation. 2 Related work 2.1 3D Generation via Alternative Representations Many 3D generative models avoid direct mesh modeling by using intermediate representations like voxels, point clouds, or implicit fields. Early voxel methods wu2016learning ; wang2017cnn using grids faced memory issues. Point cloud methods luo2021diffusion ; jun2023shap ; hu2024rangeldm ; xu2024pointllm with networks like PointNet qi2017pointnet ; qi2017pointnet++ struggle with consistency and detail. Implicit fields, especially neural fields chen2019learning ; park2019deepsdf ; zheng2023locally ; cheng2023sdfusion , offer efficient representations. These include score distillation with 2D diffusion models poole2022dreamfusion ; chen2023fantasia3d ; lin2023magic3d ; wang2023prolificdreamer ; tang2023dreamgaussian ; yi2024gaussiandreamer and 3D Transformer models like LRM hong2023lrm ; xu2023imagerewardlearningevaluatinghuman ; wang2024crm ; xu2024instantmesh ; tang2024lgm , alongside recent latent diffusion methods zhao2023michelangelo ; zhang2024clay ; wu2024direct3d ; li2024craftsman ; zhao2025hunyuan3d ; he2025triposf ; li2025triposg ; li2025step1x that have demonstrated good scalability and performance. However, these approaches often rely on post-processing via Marching Cubes lorensen1998marching , which can cause topological issues, smoothing, and artifacts. 2.2 Native Mesh Generation While neural shape representations such as implicit fields have been extensively studied, native mesh generation is an emerging area of research. Early approaches leveraging surface patches xu2024grm or mesh graphs dai2019scan2mesh often suffered from quality limitations. Diffusion-based methods alliegro2023polydiff ; he2025meshcraft have seen limited exploration in this domain, potentially due to inherent difficulties in directly processing meshes. PolyGen nash2020polygen demonstrated promise by autoregressively generating mesh vertices and faces. MeshGPT siddiqui2024meshgpt encoded meshes into quantized tokens using VQ-VAE van2017neural for autoregressive generation. Subsequently, MeshXL chen2025meshxl proposed a one-stage autoregressive model operating on coordinate-level mesh sequences. Various tokenization techniques chen2024meshanythingv2 ; tang2024edgerunner ; weng2024scaling ; wang2025nautilus and efficient training strategies hao2024meshtron ; wang2025iflame have been explored to address the challenges of long sequences in high-resolution generation; however, achieving stable and high-fidelity results remains a significant hurdle. 2.3 Reinforcement Learning for Mesh Generation Reinforcement Learning (RL) yuan2023rrhf has gained traction for 3D generation ye2024dreamreward using human feedback. Reinforcement Learning from Human