Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiability

Hoang-Son Nguyen and Xiao Fu School of Electrical Engineering and Computer Science Oregon State University {nguyhoa3, xiao.fu}@oregonstate.edu · 2025

Read Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiability by Hoang-Son Nguyen and Xiao Fu School of Electrical Engineering and Computer Science Oregon State University {nguyhoa3, xiao.fu}@oregonstate.edu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17040.txt.

Introduction

Hoang-Son Nguyen and Xiao Fu study a hard problem in machine learning in this 2025 paper. Often a system records mixed signals, and the hidden parts that made them stay unknown. The authors ask when those hidden parts can be found again from the mixed data alone. Their setting drops several limits that earlier work had needed.

The pair present a method called Diverse Influence Component Analysis. It uses the shape of how each hidden part affects the observed data. A rule that rewards varied influence lets the method pick out the parts. The paper shows the parts can be found without extra help signals. Readers gain a new route to clean, split views of mixed data.

Catalog record

CREATORHoang-Son Nguyen and Xiao Fu School of Electrical Engineering and Computer Science Oregon State University {nguyhoa3, xiao.fu}@oregonstate.edu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE129 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-17040

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiability Hoang-Son Nguyen and Xiao Fu School of Electrical Engineering and Computer Science Oregon State University {nguyhoa3, xiao.fu}@oregonstate.edu Abstract Latent component identification from unknown nonlinear mixtures is a foundational challenge in machine learning, with applications in tasks such as disentangled representation learning and causal inference. Prior work in nonlinear independent component analysis (nICA) has shown that auxiliary signals—such as weak supervision—can support identifiability of conditionally independent latent components. More recent approaches explore structural assumptions, e.g., sparsity in the Jacobian of the mixing function, to relax such requirements. In this work, we introduce Diverse Influence Component Analysis (DICA), a framework that exploits the convex geometry of the mixing function’s Jacobian. We propose a Jacobian Volume Maximization (J-VolMax) criterion, which enables latent component identification by encouraging diversity in their influence on the observed variables. Under reasonable conditions, this approach achieves identifiability without relying on auxiliary information, latent component independence, or Jacobian sparsity assumptions. These results extend the scope of identifiability analysis and offer a complementary perspective to existing methods. 1 Introduction Nonlinear mixture model identification (NMMI) seeks to uncover latent components transformed by unknown nonlinear functions. A typical data model of interest in the context of NMMI is as follows: \displaystyle\boldsymbol{x}=\boldsymbol{f}(\boldsymbol{s}),~\boldsymbol{s}\in\mathbb{R}^{d},~\boldsymbol{x}\in\mathbb{R}^{m}, (1) where \boldsymbol{s}=(s_{1},\ldots,s_{d})\sim p({\boldsymbol{s}}) is a random vector following a certain distribution p({\boldsymbol{s}}) with support {\cal S} , \boldsymbol{f}:\mathbb{R}^{d}\rightarrow\mathbb{R}^{m} is an unknown, nonlinear mixing function, \boldsymbol{x}=(x_{1},\ldots,x_{m})\in\mathbb{R}^{m} is the observed data, and s_{i} for i\in[d] and x_{j} for j\in[m] represent the i th latent component and the j th observed feature, respectively. The function \boldsymbol{f} is modeled a diffeomorphism that maps a latent variable to a d -dimensional Riemannian manifold \mathcal{X} embedded in \mathbb{R}^{m} , where m\geq d [ 1 , 2 , 3 ] . Given the observations (samples) of \boldsymbol{x} , NMMI amounts to recovering \boldsymbol{s} and \boldsymbol{f} up to certain acceptable or inconsequential ambiguities. NMMI and variants play a fundamental role in understanding many machine learning tasks, e.g., latent disentanglement [ 1 , 4 ] , causal representation learning [ 5 , 6 ] , object-centric learning [ 2 ] , and self-supervised learning [ 7 , 8 ] . The NMMI task is clearly ill-posed—in general, an infinite number of different (\boldsymbol{f},\boldsymbol{s}) could be found from observations of \boldsymbol{x} under ( 1 ). Hence, establishing identifiability of \boldsymbol{f} and \boldsymbol{s} becomes a central topic in NMMI. This identifiability challenge was extensively studied under the umbrella of nonlinear independent component analysis (nICA) [ 9 , 10 , 11 , 12 ] . A key take-home point is that nICA poses a much more challenging identification problem relative to ICA (where \boldsymbol{f} is a linear system). That is, even if s_{1},\ldots,s_{d} are statistically independent, the model in ( 1 ) is not identifiable [ 9 ] . In recent years, much progress has been made in understanding identifiability of ( 1 ). The line of work [ 10 , 11 , 12 , 1 ] showed that if s_{1},\ldots,s_{d} are conditionally independent given a certain auxiliary variable \boldsymbol{u} (which can be understood as additional side information), then \boldsymbol{f} and \boldsymbol{s} can be recovered to reasonable extents. Another line of work tackles identifiability by assuming that the mixing function \boldsymbol{f} is structured other than completely unknown. For example, the works [ 13 , 14 , 15 , 16 ] make an explicit structural assumption that \boldsymbol{f} is a post-nonlinear mixing function, the work [ 17 ] assumes that \boldsymbol{f} is conformal, and the more recent work [ 18 ] assumes that \boldsymbol{f} is piecewise linear. Using these structures, the auxiliary information can be circumvented for establishing identifiability. More recent advances propose to exploit implicit structures (other than explicit structures like post-nonlinearity) of \boldsymbol{f} . Notably, the works [ 19 , 3 , 20 ] utilize structures defined over the Jacobian of \boldsymbol{f} for identifying the model ( 1 ). In particular, it was shown that as long as the Jacobian of \boldsymbol{f} follows certain sparsity patterns, then the model ( 1 ) is identifiable [ 3 , 21 , 2 ] . Imposing structural constraints on the Jacobian of \boldsymbol{f} is arguably less restrictive compared to assuming explicit parameterization of \boldsymbol{f} . Structures of Jacobian reflect how the latent variables s_{1},\ldots,s_{d} affect the observed features x_{1},\ldots,x_{m} , making the related assumptions admit intuitive physical meaning. Using structured Jacobian to model such influences is also advocated by several causal representation learning paradigms (see, e.g., independent mechanism analysis (IMA) [ 20 ] under the independent causal mechanism (ICM) principle [ 22 ] ), and other frameworks, e.g, object-centric representation learning [ 2 , 23 , 24 ] . Open Question. The advancements have been encouraging, yet understanding to NMMI identifiability remains to be deepened. In particular, the Jacobian sparsity-based approaches, e.g., [ 19 , 3 , 21 , 2 ] , provided intriguing ways of establishing identifiability of the model in ( 1 )—without using auxiliary variables, latent component independence, or relatively restrictive structural constraints of \boldsymbol{f} . However, the Jacobian sparsity assumptions in these works essentially assume that the observed features are only generated from a subset of s_{1},\ldots,s_{d} , which sometimes may not hold. It is tempting to circumvent using such strict sparsity-based assumptions in NMMI. Contributions. In this work, we propose to make use of an alternative condition of \boldsymbol{f} ’s Jacobian for NMMI. We leverage the fact that, as long as the influences of \boldsymbol{s} imposed on each x_{j} for j=1,\ldots,m are sufficiently diverse , \boldsymbol{f} ’s Jacobian exhibits an interesting convex geometry. This geometry is similar to the classical “ sufficiently scattered condition (SSC)” in the structured matrix factorization (SMF) literature [ 25 , 26 , 27 , 28 , 29 ] and, particularly, the more recent developments in polytopic matrix factorization (PMF) [ 30 ] (also see [ 31 , 32 , 33 ] ). As a consequence, taking intuition from volume-regularized SMF [ 25 , 34 , 35 , 30 , 32 , 31 , 36 , 33 ] , we show that fitting the data model in ( 1 ) together with maximizing the learned \boldsymbol{f} ’s Jacobian volume provably recovers \boldsymbol{f} and \boldsymbol{s} up to acceptable ambiguities. The proposed Jacobian volume maximization (J-VolMax) approach does not rely on auxiliary variables or statistical independence. More notably, an \boldsymbol{f} with a dense Jacobian can still be provably identified under our formulation. These advantages make our identifiability results applicable to a wide range of scenarios that are not covered by the existing literature, substantially expanding the understanding of model identifiability under ( 1 ). We tested the proposed approach on synthetic data and in a single-cell transcriptomics application. The results corroborate with our NMMI theory. Notation. Please refer to Appendix A.1 for details. 2 Background From ICA to nICA. ICA is arguably one of the most influential latent component identification approaches. Classic ICA [ 37 , 38 ] assumes that \boldsymbol{x}=\boldsymbol{A}\boldsymbol{s} with a nonsingular \boldsymbol{A} and that \displaystyle\textstyle p(\boldsymbol{s})=\prod_{i=1}^{d}p_{i}(s_{i}), (2) where at most one s_{i} is a Gaussian variable. Then, the identifiability of (\boldsymbol{A},\boldsymbol{s}) up to permutation and scaling ambiguities can be established by finding a linear filter to output independent estimates. Unfortunately, the nICA model, i.e.,

Read the complete text