Registration-Free Learnable Multi-View Capture of Faces in Dense Semantic Correspondence
Panagiotis P. Filntisis George Retsinas Radek Daneček, Vanessa Sklyarova Petros Maragos Timo Bolkart · 2026
Read Registration-Free Learnable Multi-View Capture of Faces in Dense Semantic Correspondence by Panagiotis P. Filntisis George Retsinas Radek Daneček, Vanessa Sklyarova Petros Maragos Timo Bolkart (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01450.txt.
Introduction
Panagiotis P Filntisis and five co-authors predict 3D face meshes from several camera views at once, with every point on the mesh meaning the same facial spot across people. Automated methods exist, but they learn from training faces that were registered by slow hand pipelines, the very cost they aim to cut.
Their framework, MOCHI, trains with no registered faces at all. A pseudo-linear solver keeps the mesh shape consistent, and dense keypoints from a landmark tool trained only on synthetic faces guide the match. The team also swap the usual point-to-surface loss for pointmap and normal losses, which train more smoothly and rebuild faces more cleanly in this setting. A short test-time tune of the network closes much of the gap to hand pipelines, and final scores beat the classic labor-heavy route on both accuracy and looks.
Catalog record
| CREATOR | Panagiotis P. Filntisis George Retsinas Radek Daneček, Vanessa Sklyarova Petros Maragos Timo Bolkart |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.01450
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Registration-Free Learnable Multi-View Capture of Faces in Dense Semantic Correspondence Panagiotis P. Filntisis George Retsinas Radek Daneček, Vanessa Sklyarova Petros Maragos Timo Bolkart Abstract Recent frameworks like ToFu and TEMPEH provide an automated alternative to classical registration pipelines by predicting 3D meshes in dense semantic correspondence directly from calibrated multi-view images. However, these learning-based methods rely on the slow, manual registration pipelines they aim to replace for their training supervision. We overcome this limitation with MOCHI ( M ulti-view O ptimizable C orrespondence of H eads from I mages), a multi-view 3D face prediction framework trained without requiring registered training data. MOCHI eliminates the registration data dependency by enforcing topological consistency through a pseudo-linear inverse kinematic solver. Semantic alignment is guided by dense keypoints from a 2D landmark predictor trained exclusively on synthetic data. Our analysis further reveals that standard point-to-surface distances induce training instabilities and visual artifacts in registration-free settings. We propose pointmap- and normal-based losses instead, which provide smoother gradients and superior reconstruction fidelity. Finally, we introduce a test-time optimization scheme that refines network weights over a few dozen iterations. This approach bridges the gap between feed-forward efficiency and iterative optimization precision, allowing MOCHI to outperform traditional labor-intensive pipelines in both reconstruction accuracy and visual quality. Code and model are public at: https://filby89.github.io/mochi . Figure 1 : MOCHI : Registration-free multi-view face capture in dense correspondence. Given calibrated multi-view images MOCHI predicts a canonical FLAME-topology mesh without needing precomputed registrations during training . When a target scan is available at test-time, an optional brief TTO of the model sharpens details and outperforms classical, manually labor-heavy registration pipelines . 2 2 footnotetext: The research work of P. P. Filntisis and P. Maragos was supported by the project “Applied Research for Autonomous Robotic Systems” (MIS 5200632) which is implemented within the framework of the National Recovery and Resilience Plan “Greece 2.0” (Measure: 16618- Basic and Applied Research) and is funded by the European Union- NextGenerationEU. 1 Introduction 3D face reconstruction in dense correspondence – that is, producing meshes with a consistent topology across identities and expressions at the highest possible quality – is a central problem in computer vision. It has direct impact on a number of industries such as entertainment (VFX in movies, games), future forms of communication (teleconferencing in AR, VR, Metaverse), healthcare (such as cosmetic surgery pre-visualization), and perhaps future 3D embodied AI assistants. Ideally, a face reconstruction method should not only recover accurate geometry but also enforce a shared topology across individuals and frames, enabling semantic consistency and compatibility with downstream tasks such as animation and editing. Despite tremendous progress in face modeling and reconstruction 21 ; 75 , the highest-fidelity systems still rely on multi-view stereo (MVS) capture 26 ; 67 followed by non-rigid registration to bring scans into a predefined topology 48 ; 11 ; 59 . This registration step requires iterative optimization and often manual verification or correction by human experts, making it one of the major bottlenecks for scalable 3D face capture. Recent learning-based systems aim to simplify this process by predicting meshes in a fixed topology directly from calibrated multi-view images 9 ; 46 ; 49 (see Fig. 2 ). While such methods greatly reduce processing time, they still require precomputed registrations during training , and preparing such data remains time-consuming and labor-intensive, and a classical registration pipeline must be in place to provide the training supervision. To remove the dependency on precomputed registrations for training, we present MOCHI, a registration-free approach for multi-view 3D face reconstruction in dense correspondence which allows training directly on raw scans. First, a pseudo-linear inverse kinematic solver 70 recovers 3DMM parameters from the network’s vertex outputs. These parameters are then passed through the parametric model to produce a topology-consistent mesh, which is used to regularize the network’s free-form predictions via vertex- and edge-level losses. Second, we observe that the traditionally used loss to train with scans, i.e., the point-to-surface distance, produces noisy gradients due to discrete closest point assignments thus often leading to self-intersections and other artifacts if the mesh is not already tightly aligned to the scan. We instead swap this out for differentiable rendering losses: pointmap and surface-normal losses 81 . These losses provide smoother gradients and train the model more reliably. Finally, because direct geometric errors often fail to capture semantically salient perceptual effects, we additionally supervise with 2D landmarks predicted by a model trained purely on synthetic data. At inference, we also introduce a test-time optimization (TTO) stage that performs a brief per-instance optimization of the model parameters, leading to registrations more accurate than the classical manually intensive registration pipeline. As a result, MOCHI can register a new multi-view dataset from scratch, using only the raw scans and calibrated images, effectively minimizing the manual effort needed (Fig. 2 ). Our contributions can be summarized as follows: (1) To our knowledge, the first dense-correspondence multi-view face reconstruction system that does not require precomputed registrations for training, yet surpasses registration-requiring methods such as TEMPEH 9 in reconstruction accuracy. (2) A stable training objective based on pointmap and surface-normal losses, reducing artifacts and improving model training compared to scan-to-mesh supervision. (3) A lightweight test-time optimization (TTO) of our base model, which when coupled with an existing scan outperforms even classical manually-intensive registration pipelines. Figure 2 : Traditional capturing systems (right) rely on a multi-stage approach involving MVS to produce a head scan, followed by registration that requires manual supervision. Recent learning-based systems (left) simplify the workflow but still require GT registrations from traditional systems for training (red arrow). MOCHI relies only on the fully automatic parts (green arrow), outperforming previous learning-based methods despite removing the need for expensive manual registrations. When coupled with TTO, MOCHI surpasses even the classic manual registration pipeline. 2 Related Work Recovering 3D faces from images has advanced dramatically since the seminal work of Blanz and Vetter 8 . We review the most relevant lines of work below. Scan Registration. Traditional high-fidelity face capture begins with obtaining a 3D scan, most commonly via multi-view stereo (MVS) 22 ; 26 ; 34 ; 43 ; 80 , though laser scanning 44 , photometric stereo 31 ; 56 , and learning-based methods 42 ; 38 ; 92 ; 81 ; 83 are also used; a comprehensive survey 82 covers these in detail. The resulting scans have unstructured topology and may contain holes or artifacts in difficult regions (e.g. lips, eyes, facial hair), and must therefore be registered onto a common topology to be useful in production. A common approach is to first fit a 3DMM to the scan 1 ; 7 ; 47 , providing a coarse initialization that is then refined with non-rigid ICP 6 to capture finer detail 48 ; 87 ; 14 ; 91 ; 59 ; 30 ; 12 ; 10 ; 45 ; 58 ; 65 . Other methods skip the model initialization and apply non-rigid ICP directly to deform a template to the scan 4 ; 13 ; 5 . Some works collapse scanning and registration into a single joint stage 27 ; 63 . Learning-based approaches train neural networks to map unstructured raw scans directly to a shared template topology, through mechanisms such as surface-to-surface