LIA-X: Interpretable Latent Portrait Animator

Yaohui Wang Di Yang Xinyuan Chen François Brémond Yu Qiao Antitza Dantcheva · 2025

Read LIA-X: Interpretable Latent Portrait Animator by Yaohui Wang Di Yang Xinyuan Chen François Brémond Yu Qiao Antitza Dantcheva (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-09959.txt.

Introduction

Yaohui Wang and co-authors present LIA-X, a portrait animator whose inner workings stay readable, in a 2025 computer vision paper. The system takes the facial motion from a driving video and applies it to a still portrait. It casts motion transfer as movement through a learned latent space, steered by a sparse motion dictionary that splits facial dynamics into separate, nameable factors.

That readability supports an edit-first strategy: users can adjust pose and expression in the portrait before the motion is rendered, closing the gap with the driver. A version with about a billion parameters trained on large datasets beat earlier methods on standard reenactment tests. The design also suits guided editing and three-dimensional-aware video work.

Catalog record

CREATORYaohui Wang Di Yang Xinyuan Chen François Brémond Yu Qiao Antitza Dantcheva
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-09959

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LIA-X: Interpretable Latent Portrait Animator Yaohui Wang Di Yang Xinyuan Chen François Brémond Yu Qiao Antitza Dantcheva Abstract We introduce LIA-X, a novel interpretable portrait animator designed to transfer facial dynamics from a driving video to a source portrait with fine-grained control. LIA-X is an autoencoder that models motion transfer as a linear navigation of motion codes in latent space. Crucially, it incorporates a novel Sparse Motion Dictionary that enables the model to disentangle facial dynamics into interpretable factors. Deviating from previous ’warp-render’ approaches, the interpretability of the Sparse Motion Dictionary allows LIA-X to support a highly controllable ’edit-warp-render’ strategy, enabling precise manipulation of fine-grained facial semantics in the source portrait. This helps to narrow initial differences with the driving video in terms of pose and expression. Moreover, we demonstrate the scalability of LIA-X by successfully training a large-scale model with approximately 1 billion parameters on extensive datasets. Experimental results show that our proposed method outperforms previous approaches in both self-reenactment and cross-reenactment tasks across several benchmarks. Additionally, the interpretable and controllable nature of LIA-X supports practical applications such as fine-grained, user-guided image and video editing, as well as 3D-aware portrait video manipulation. Figure 1 : Portrait Animation. We show results of three portraits animated by LIA-X using the video of Gal Gadot (top right). Given a source portrait ( 1^{\text{st}} column), LIA-X allows for editing the source portrait based on the learned semantics to align it with the initial driving frame in terms of head pose and facial expression ( 2^{\text{nd}} column). The animated sequences ( 2^{\text{nd}} - 5^{\text{th}} columns) are then obtained by applying motion transfer process on the edited portraits. † † *Corresponding author. 1 Introduction With the remarkable development of deep generative models [ 13 , 32 , 10 ] , techniques for video generation have been largely advanced. Portrait animation, a domain-specific video generation task that aims to transfer facial dynamics from a driving video to a portrait image, has also received increasing attention due to its wide applications in entertainment, e-education, and digital human creation. Towards accurately transferring facial dynamics, one straightforward strategy is to leverage pre-computed explicit representations such as facial landmarks [ 39 , 5 , 21 ] , 3DMM [ 19 , 6 ] , optical flows [ 17 , 23 ] and dense poses [ 50 ] as motion guidance. Such an approach has been widely used in both GAN-based and recent diffusion-based methods [ 5 , 21 ] . However, the generated quality of these methods relies heavily on the performance of the off-the-shelf feature extractors, which strongly restricts their usability in more challenging, real-world scenarios. Self-supervised learning-based techniques have also been proposed to tackle this problem. Previous methods explored learning either explicit structures such as 2D/3D keypoints [ 28 , 29 , 58 , 40 , 11 ] or implicit motion codes [ 48 , 45 ] in an end-to-end manner to model facial dynamics. They typically follow a ‘warp-render’ strategy to conduct portrait animation based on the computed optical-flow fields. While these methods have achieved promising results in both motion transferring and identity preservation, their performance still drops significantly when large variations exist between the source and driving data in terms of head poses and facial expressions. Towards addressing this issue, we introduce LIA-X, a novel framework that learns interpretable motion semantics to enable a highly controllable ‘edit-warp-render’ animation strategy. Deviating from the standard ‘warp-render’ pipeline, LIA-X allows users to first leverage the learned motion representations to align source portrait with the initial driving frame before applying the final motion transfer (see Fig. 1 ). Specifically, LIA-X is designed as a self-supervised autoencoder that does not rely on any explicit structure representations. Inspired by sparse dictionary coding [ 24 ] , we incorporates a novel Sparse Motion Dictionary - a set of motion vectors in autoencoder to capture the underlying motion distribution. The sparsity constraint encourages the model to use a minimal set of these motion vectors to reconstruct the training images, endowing the motion vectors with enhanced interpretability compared to the dense motion dictionary proposed in prior work [ 48 , 45 ] . The animation process is then modeled as a linear navigation of these motion codes to produce optical-flow fields for warping the source portrait. Interestingly, we found that these interpretable motion vectors can also be directly leveraged at the inference stage to manipulate source portrait, supporting fine-grained edits of facial attributes (e.g., eyes and mouth) as well as 3D-aware transformation (e.g., yaw, pitch and roll) in both image and video domains. In addition, we have analyzed the scalability of proposed LIA-X framework and successfully train large-scale models with up to 1 billion parameters using a diverse mixture of public and internal talking head datasets. Experimental results demonstrate that scaling up the LIA-X significantly improves performance across several benchmarks. Given the faster inference speed of autoencoder-based models compared to diffusion-based approaches, we believe the scalable and interpretable design of LIA-X can serve as a valuable complement to current state-of-the-art generative models, enabling efficient and highly controllable video generation. In summary, the key contributions of this work are: • We propose a novel portrait animator, LIA-X, an autoencoder that incorporates an interpretable Sparse Motion Dictionary to enable controllable portrait animation via an ‘edit-warp-render’ strategy. • We analyze the scalability of the LIA-X and demonstrate that our design can be scaled up to achieve superior performance. • Extensive experiments show that LIA-X outperforms state-of-the-art methods across several datasets, while also supporting a various of applications such as image and video editing, and 3D-aware portrait manipulation. Figure 2 : Overview of LIA-X. LIA-X consists of an encoder E , a generator G that includes an optical-flow generator G_{f} and a rendering network G_{r} . LIA-X is trained using a self-supervised learning strategy. To obtain an interpretable motion dictionary D_{m} , a sparsity constraint is incorporated into the training objective, enrouraging the network to use a minimal number of vectors in D_{m} to reconstruct each driving image. 2 Related Work Portrait animation has seen significant advancements in recent years, driven by the remarkable progress in deep generative models for video generation [ 36 , 26 , 35 , 43 , 42 , 41 , 44 , 9 , 2 , 53 , 31 , 34 , 47 , 8 , 7 , 12 , 30 , 14 , 20 , 46 , 1 , 22 , 51 , 3 , 56 ] . Previous methods have explored learning structure representations either based on conditional generation approaches [ 4 , 38 , 55 , 39 , 52 , 58 ] relying on off-the-shelf extractors, or self-supervised learning strategies to learn representations such as 2D/3D keypoints [ 28 , 40 , 11 , 58 ] , motion regions [ 29 ] , and depth maps [ 16 ] in an end-to-end manner. Self-attention mechanism [ 54 ] has also been studied to improve cross-identity generation quality. More recent techniques [ 48 , 45 ] have proposed to model motion transfer as a linear navigation of learned motion codes, which has proven effective in implicitly capturing 2D and 3D representations to capture complex facial dynamics. Diffusion-based models [ 49 ] have

Read the complete text