KaoLRM: Repurposing Pre-trained Large Reconstruction Models for Parametric 3D Face Reconstruction
Qingtian Zhu, Xu Cao, Zhixiang Wang, Yinqiang Zheng, Takafumi Taketomi · 2026
Read KaoLRM: Repurposing Pre-trained Large Reconstruction Models for Parametric 3D Face Reconstruction by Qingtian Zhu, Xu Cao, Zhixiang Wang, Yinqiang Zheng, Takafumi Taketomi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12736.txt.
Introduction
Qingtian Zhu and four co authors rebuild faces in three dimensions from a single photo. Their system is called KaoLRM. It repurposes a large reconstruction model, retargeting its learned prior for faces. Parametric face models, called 3D morphable models, describe a face with a compact, interpretable parameter set.
Regressors for those models often lack consistency across viewpoints, though. The same face seen from a new angle may yield a different reconstruction. A strong learned prior from a large model could stabilize those estimates. Single view face reconstruction serves avatars, effects, and biometrics. Better view consistency would make all of those uses more trustworthy.
Catalog record
| CREATOR | Qingtian Zhu, Xu Cao, Zhixiang Wang, Yinqiang Zheng, Takafumi Taketomi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-12736
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
KaoLRM: Repurposing Pre-trained Large Reconstruction Models for Parametric 3D Face Reconstruction Qingtian Zhu, Xu Cao, Zhixiang Wang, Yinqiang Zheng, Takafumi Taketomi Abstract We propose KaoLRM to re-target the learned prior of the Large Reconstruction Model (LRM) for parametric 3D face reconstruction from single-view images. Parametric 3D Morphable Models (3DMMs) have been widely used for facial reconstruction due to their compact and interpretable parameterization, yet existing 3DMM regressors often exhibit poor consistency across varying viewpoints. To address this, we harness the pre-trained 3D prior of LRM and incorporate FLAME-based 2D Gaussian Splatting into LRM’s rendering pipeline. Specifically, KaoLRM projects LRM’s pre-trained triplane features into the FLAME parameter space to recover geometry, and models appearance via 2D Gaussian primitives that are tightly coupled to the FLAME mesh. The rich prior enables the FLAME regressor to be aware of the 3D structure, leading to accurate and robust reconstructions under self-occlusions and diverse viewpoints. Experiments on both controlled and in-the-wild benchmarks demonstrate that KaoLRM achieves superior reconstruction accuracy and cross-view consistency, while existing methods remain sensitive to viewpoint variations. The code is released at https://github.com/CyberAgentAILab/KaoLRM . Figure 1 : KaoLRM predicts FLAME parameters from a single-view facial image and yields more faithful and consistent predictions for the same subject captured from different viewpoints. We validate the consistency qualitatively through visual inspection of the reconstructed FLAME meshes and quantitatively by measuring the variance of the FLAME parameters across the views. The methods for reference are DECA 12 and SMIRK 38 . 1 Introduction Inferring parametric 3D Morphable Models (3DMMs) from single-view facial images is a long-standing challenge in computer vision and graphics. Due to the compact and interpretable parametric representation, 3DMMs provide intuitive control over identity, expression, and pose, making them crucial for applications such as animation, virtual reality, and telepresence 47 . Training single-view 3DMM regressors with annotated parameters or explicit 3D supervision requires paired data, which are often expensive to obtain. To alleviate the reliance on such annotations, many existing methods 12 ; 7 ; 38 adopt self-supervised, analysis-by-synthesis training paradigms, where the reconstructed 3DMM is re-rendered from the input viewpoint and compared with the input images using a photometric loss. However, this paradigm lacks cross-view supervision, making the prediction sensitive to viewpoint changes at test time. Without cross-view constraints, the model relies primarily on 2D visual features to regress 3DMM parameters, rather than inferring the 3D structure in the image. Consequently, when viewpoint changes and key facial components become self-occluded, the reconstruction deteriorates rapidly, as shown in Fig. 1 . Recent advances in 3D foundation models 18 ; 51 ; 48 have marked a paradigm shift in 3D vision. Rather than relying heavily on inductive biases and task-specific architectures, these models exploit massive amounts of diverse 3D data and high-capacity neural networks to learn general-purpose priors for 3D reconstruction. For example, LRMs 18 reconstruct implicit 3D representations ( e.g . , triplane-based radiance fields 32 ; 5 ) with cross-view photometric supervisions that are robust to viewpoint changes and missing features. However, these implicit representations are not directly suitable for downstream applications that require structured, parameterized control, leaving a gap between general-purpose reconstruction models and task-specific parametric modeling. To address this discrepancy, we propose KaoLRM 1 1 1 “Kao” is derived from the Japanese word for “face”. to adapt the pre-trained prior of LRM for parametric face reconstruction. Figure 2 illustrates our key motivation. Leveraging LRM’s priors provides two main benefits. First, the rich 3D prior encoded in LRM captures 3D structure in images more effectively, improving the robustness to invisible facial components under viewpoint changes. Second, pre-training on a large-scale multi-view general object dataset reduces the reliance on large-scale multi-view facial datasets. Specifically, we incorporate a FLAME decoder into LRM to project the pre-trained triplane features into the FLAME parameter space, enabling parametric facial geometry prediction. To support cross-view photometric supervision, we model appearance using surface-bounded 2D Gaussian primitives derived from the FLAME mesh, and introduce a binding loss to enforce tight coupling between geometry and appearance. This coupling prevents the model from overfitting to appearance while ignoring geometric accuracy. Experimental results demonstrate that KaoLRM achieves superior reconstruction accuracy and improved cross-view consistency compared to prior methods, validating the effectiveness of transferring LRM’s pre-trained prior from general objects to parametric face models. Figure 2 : Motivation of KaoLRM. Instead of learning 3DMM regressors from scratch using multi-view face datasets that are expensive to scale, we leverage the learned 3D prior of LRM 18 ( e.g . , tri-plane features), allowing KaoLRM to be trained with a moderate amount of data. In summary, our contributions are threefold: • We introduce KaoLRM, a framework that re-targets the pre-trained knowledge of LRM for parametric face reconstruction. • We bridge the gap between LRM’s implicit 3D representations and FLAME parameters by integrating a parametric analysis-by-synthesis pathway into LRM’s rendering framework. • KaoLRM outperforms previous methods regarding reconstruction quality and view robustness while maintaining competitive performance in in-the-wild scenarios. 2 Related Work 2.1 Large Reconstruction Model As the prevailing trend in both neural network architectures and dataset sizes continues to move towards larger scales, one notable outcome in the specific domain of 3D reconstruction is the emergence of the Large Reconstruction Model (LRM) 18 . The underlying philosophy of LRM is to incorporate as little inductive bias as possible, e.g . , the camera geometry usually adopted for unprojection, so that the model can learn representations and mappings directly from data, potentially surpassing the constraints of traditional, hand-crafted geometric assumptions. As a result, LRM can regress the triplane-based radiance field directly from a single image in a completely feed-forward manner. Following LRM, a number of methods have emerged that further investigate and expand upon different facets of this paradigm. LGM 42 and GS-LRM 59 shift the target of regression from radiance fields to pixel-aligned 3D Gaussians; PF-LRM 50 studies the power of LRM in joint pose and shape prediction; MeshLRM 54 re-targets a NeRF LRM to directly generate meshes with higher quality; LRM-Zero 57 relieves the data reliance of LRM by employing synthetic data for training; LVSM 20 further extends the paradigm to novel view synthesis. Specifically regarding LRM approaches to digital human avatars, we observe the emergence of methods such as Human-LRM 55 and FaceLift 30 , both of which rely on multi-view diffusion models to generate pseudo multi-view inputs and employ LRM-based models to perform reconstruction. However, we observe that these methods produce generic 3D geometric representations, such as point clouds or meshes, which are not readily applicable to downstream tasks like animation, where parametric models such as SMPL 28 or FLAME 26 are typically preferred. In this paper, we opt to explore the possibility of re-targeting LRM to a compact 3D representation, namely a parametrized 3DMM, to achieve a more reasonable balance between generalizability and practicality. 2.2 3DMM Regression 3DMM 11 refers to parametric and statistical models of 3D face shape (and optionally appearance) built from real 3D scans. The original 3DMM 2 is a typical example where Principal Component Analysis (PCA) is applied to a dataset of registered 3D faces to model variations in both shape and texture. Since then, many specialized 3DMMs have been developed, e.g . , BFM 35 ,