A Mutual Information Perspective on Multiple Latent Variable Generative Models for Positive View Generation
Dario Serez [email protected], Marco Cristani [email protected], Alessio Del Bue [email protected], Vittorio Murino [email protected], Pietro Morerio [email protected] · 2025
Read A Mutual Information Perspective on Multiple Latent Variable Generative Models for Positive View Generation by Dario Serez [email protected], Marco Cristani [email protected], Alessio Del Bue [email protected], Vittorio Murino [email protected], Pietro Morerio [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-13718.txt.
Introduction
Dario Serez, Marco Cristani, Alessio Del Bue, Vittorio Murino, and Pietro Morerio analyzed in 2025 how multiple latent variables share the work in image generation models. Such models build an image step by step, from global traits to fine local detail, as in well-known face and image generators. How much each latent variable truly contributes was mostly a matter of observation, not measurement.
Their frame uses mutual information, a measure of shared signal, to quantify each variable's contribution. The analysis shows current models often underuse some of their latent variables. From that base they build a method to make synthetic data for contrastive self-supervised learning, using the layered structure on purpose. The catalog creator field appends email addresses; the head byline governs. Readers get a meter for a hidden part of generative models, plus a use for it.
Catalog record
| CREATOR | Dario Serez [email protected], Marco Cristani [email protected], Alessio Del Bue [email protected], Vittorio Murino [email protected], Pietro Morerio [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-13718
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Mutual Information Perspective on Multiple Latent Variable Generative Models for Positive View Generation Dario Serez [email protected], Marco Cristani [email protected], Alessio Del Bue [email protected], Vittorio Murino [email protected], Pietro Morerio [email protected] Abstract In image generation, Multiple Latent Variable Generative Models (MLVGMs) employ multiple latent variables to gradually shape the final images, from global characteristics to finer and local details ( e.g . , StyleGAN, NVAE), emerging as powerful tools for diverse applications. Yet their generative dynamics remain only empirically observed, without a systematic understanding of each latent variable’s impact. In this work, we propose a novel framework that quantifies the contribution of each latent variable using Mutual Information (MI) as a metric. Our analysis reveals that current MLVGMs often underutilize some latent variables, and provides actionable insights for their use in downstream applications. With this foundation, we introduce a method for generating synthetic data for Self-Supervised Contrastive Representation Learning (SSCRL). By leveraging the hierarchical and disentangled variables of MLVGMs, our approach produces diverse and semantically meaningful views without the need for real image data. Additionally, we introduce a Continuous Sampling (CS) strategy, where the generator dynamically creates new samples during SSCRL training, greatly increasing data variability. Our comprehensive experiments demonstrate the effectiveness of these contributions, showing that MLVGMs’ generated views compete on par with or even surpass views generated from real data. This work establishes a principled approach to understanding and exploiting MLVGMs, advancing both generative modeling and self-supervised learning. Code and pre-trained models at: https://github.com/SerezD/mi_ml_gen . (a) (b) Figure 1 : (a) Multiple Latent Variable Generative Models utilize multiple latent variables (here \mathbf{Z}_{1},\mathbf{Z}_{2},\mathbf{Z}_{3} ), which are sequentially introduced at different layers of the generative network to produce high-quality images. (b) The base image (left), generated using points \mathbf{z}_{1},\mathbf{z}_{2},\mathbf{z}_{3} , can be selectively modified by altering individual latents ( \mathbf{z}_{1} to \hat{\mathbf{z}}_{1} , \mathbf{z}_{2} to \hat{\mathbf{z}}_{2} , or \mathbf{z}_{3} to \hat{\mathbf{z}}_{3} ). Each latent affects the final image differently, at first influencing broader, global attributes and later refining finer, local details (darker to lighter shading in the figure). 1 Introduction Latent Variable Generative Models (LVGMs), such as Variational Autoencoders (VAEs) ( Kingma & Welling, 2014 ; Rezende et al., 2014 ) and Generative Adversarial Networks (GANs) ( Goodfellow et al., 2014 ) , are foundational approaches for image generation. Given a random variable \mathbf{X}\in\mathcal{X} , representing high-dimensional pictures in pixel space, LVGMs aim to approximate the underlying data distribution p(\mathbf{X}) . To achieve this, they learn a parameterized generator g(\mathbf{z};\theta)=\mathbf{x} , where \mathbf{Z}\in\mathcal{Z} denotes a latent variable sampled from a simpler and known distribution in the \mathcal{Z} latent space. A key objective of the learning process is to ensure that the generator is continuous, such that neighboring latent points \mathbf{z}^{\prime} and \mathbf{z}^{\prime\prime} are mapped to perceptually similar outputs \mathbf{x}^{\prime} and \mathbf{x}^{\prime\prime} . This regularization of the latent space allows LVGMs to generate novel content and meaningfully interpolate latent features ( Radford et al., 2016 ; Higgins et al., 2017 ) . Over the years, advancements in latent generative modeling have focused on the use of multiple latent variables, rather than a single latent code ( Vahdat & Kautz, 2020 ; Karras et al., 2019 ; Karras et al., 2020 ; Karras et al., 2021 ; Sauer et al., 2022 ) . By incorporating latent variables at different layers of the network (see Figure 1(a) ), these Multiple Latent Variable Generative Models (MLVGMs) offer a hierarchical structure where early latent codes influence broad, global features and later codes refine finer, local details ( Figure 1(b) ). The resulting architecture enhances control over the image synthesis process, enabling the generation of high-resolution images with richer detail and improved precision. The ability to disentangle global and local features in image generation not only improves the visual quality of generated images but also widens the application scope of these models. For instance, the StyleGAN architecture ( Karras et al., 2019 ) has demonstrated exceptional performance in image editing ( Alaluf et al., 2022 ; Pehlivan et al., 2023 ) , manipulation ( Tov et al., 2021 ) , and translation ( Richardson et al., 2021 ) . Additionally, recent studies have shown that MLVGMs can serve as effective foundation models for tasks such as adversarial purification ( Serez et al., 2025 ) . Collectively, these findings highlight the versatility of MLVGMs, showcasing their utility not only in creative and generative domains but also as pre-trained models for broader applications. Nevertheless, existing research primarily leverages the “global-to-local” behavior of MLVGMs as an empirical tool, applying it across diverse tasks without delving into the mechanics of latent variable utilization. In other terms, these approaches assume that earlier latent variables shape coarse image attributes while later ones refine fine details, but they do so without formally analyzing how each latent variable contributes to image generation. As a result, the internal dynamics of MLVGMs remain poorly understood. To address this gap, we propose a novel approach that establishes a direct relationship between feature distances in each latent space ( \mathbf{Z}_{1},\mathbf{Z}_{2},\dots,\mathbf{Z}_{n} ) and mutual information (MI) shifts in the shared image space \mathbf{X} . The key insight is that producing an equivalent MI shift in the output space requires increasingly larger perturbations ( \mu_{i} ) as we move deeper into the generative hierarchy—i.e., from \mathbf{Z}_{1} to \mathbf{Z}_{n} . This quantitatively confirms the global-to-local pattern and reveals how influence diminishes across successive latent variables (see Figure 2 ). Figure 2 : Illustration of our findings on the “global-to-local” behavior in MLVGMs. On the left, perturbations are applied to each latent variable independently ( \mathbf{Z}_{1}\to\hat{\mathbf{Z}}_{1} , \mathbf{Z}_{2}\to\hat{\mathbf{Z}}_{2} , and \mathbf{Z}_{3}\to\hat{\mathbf{Z}}_{3} ), with the average perturbation magnitude increasing across latent spaces ( \mu_{1}<\mu_{2}<\mu_{3} ). On the right, each perturbed latent variable is used to generate modified images ( \mathbf{X}^{\prime} , \mathbf{X}^{\prime\prime} , and \mathbf{X}^{\prime\prime\prime} ) from the original \mathbf{X} . Notably, the increasing perturbation magnitude in the latent space maintains approximately equal Mutual Information shifts in the image space: I(\mathbf{X};\mathbf{X}^{\prime})\approx I(\mathbf{X};\mathbf{X}^{\prime\prime})\approx I(\mathbf{X};\mathbf{X}^{\prime\prime\prime}) . This provides the first quantitative measure of the “global-to-local” property, where earlier latents affect global features and later latents refine local details. By grounding this analysis in information theory, our approach moves beyond intuition and provides a principled framework for understanding latent variable roles in MLVGMs. It enables direct comparisons between latent spaces and exposes inefficiencies in how models allocate representational power. Notably, we reveal the underutilization of later-stage latents in all tested models, opening to further architectural improvements and enabling better strategies for leveraging MLVGMs in downstream tasks. With this understanding, we propose a novel application of MLVGMs in Self-Supervised Contrastive Representation Learning (SSCRL). In SSCRL, feature extractors, or encoders f(\mathbf{x};\phi) with parameters \phi , are trained to represent data by contrasting positive and negative views. Positive views are semantically similar images, encouraged to have close representations in the latent space, while negative views correspond to unrelated data points that are forced to have distant representations. Therefore, we propose to leverage the different impacts of multiple latent variables in MLVGMs to manipulate specific features and generate positive views. This approach enables the training of SSCRL encoders without relying on real data , demonstrating the potential of MLVGMs as pre-trained models for producing high-quality synthetic images tailored for representation learning.