Interpretable Visualizations of Data Spaces for Classification Problems

Christian Jorgensen, Arthur Y. Lin, Rhushil Vasavada, Rose K. Cersonsky · 2025

Read Interpretable Visualizations of Data Spaces for Classification Problems by Christian Jorgensen, Arthur Y. Lin, Rhushil Vasavada, Rose K. Cersonsky (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-05861.txt.

Introduction

Christian Jorgensen and co-authors present a way to draw maps of data that show how a classification model draws its boundaries, in work from 2025. Models sort items into classes with high success, which means some view of the data makes the boundary simple. Standard plotting tools rarely reveal that view.

Their method mixes supervised and unsupervised ideas to build a map a person can read both by eye and by numbers. They show the method on a chemical problem, judging whether compounds harm nerves. The map exposes the boundary the model uses for that task. Although the example comes from chemistry, the authors say the approach suits any field where people need to open up classification models and inspect their logic.

Catalog record

CREATORChristian Jorgensen, Arthur Y. Lin, Rhushil Vasavada, Rose K. Cersonsky
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-05861

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Interpretable Visualizations of Data Spaces for Classification Problems Christian Jorgensen, Arthur Y. Lin, Rhushil Vasavada, Rose K. Cersonsky Abstract How do classification models “see” our data? Based on their success in delineating behaviors, there must be some lens through which it is easy to see the boundary between classes; however, our current set of visualization techniques makes this prospect difficult. In this work, we propose a hybrid supervised-unsupervised technique distinctly suited to visualizing the decision boundaries determined by classification problems. This method provides a human-interpretable map that can be analyzed qualitatively and quantitatively, which we demonstrate through several established examples from literature. While we discuss this method in the context of chemistry-driven problems, its application can be generalized across subfields for “unboxing” the operations of machine-learning classification models. I Introduction Over the past decades, machine learning (ML) has transformed computational chemistry, for example accelerating materials discovery [ 30 ] , improving the quality of molecular simulations [ 4 , 5 ] , and enabling reliable predictions of materials properties [ 37 ] . However, many traditional ML and deep learning methods function as “black boxes,” providing little insight into the underlying physics and chemistry of materials and materials processes. In chemical research, the true value of an ML model often goes beyond its ability to make accurate predictions; it also lies in its ability to reveal how the set of input parameters, often carefully chosen to capture the underlying chemistry, relate to the property of interest. Addressing this need for interpretability has driven the development of approaches specifically tailored to uncover structure-property relationships, a pursuit central to the field of Quantitative Structure-Property Relationship (QSPR) modeling. Many approach QSPR through human-interpretable visualization via unsupervised machine-learning methods. Principal Component Analysis (PCA) is the most standard method for mapping tasks [ 35 ] . PCA involves identifying a linear (or nonlinear, in the case of kernel PCA) combination of the features that maximizes the variance (i.e., information) retained. Beyond PCA, there exist a slue of other mapping techniques commonly used to qualitatively understand data spaces – t-distributed Stochastic Neighbor Embedding (t-SNE) [ 32 , 38 ] , Uniform Manifold Approximation and Projection (UMAP) [ 33 ] , and Isometric Mapping (ISOMAP) [ 47 ] focus on the preservation of the local, global, and geometric data structure, respectively, and Locally Linear Embedding (LLE) [ 41 ] and multi-dimensional scaling (MDS) [ 25 , 26 ] preserve distances between neighboring data points. While all of these methods excel at creating informative visualizations and identifying patterns in the structure of the data, the “P” (property) of “QSPR” is only spuriously accounted for, where any relationship between the visualized dimensions and the property of interest is purely coincidental. So, how do we determine which chemical or materials characteristics most influence the decisions made by our model? When these characteristics are explicitly included in our input vectors, feature importance algorithms are indispensable for such tasks. Tree-based methods, such as random forests, are most commonly associated with feature importance analysis, as they provide built-in “feature importance” measures, which are typically computed as the total reduction of either the Gini impurity (for classification) or the squared error (for regression) of each of the features [ 7 ] . SHAP (SHapley Additive exPlanations), another popular post-training analysis scheme, draws from game theory and involves assigning each feature a value (i.e., SHAP value) representing its contribution to a model’s prediction by considering all possible combinations of feature subsets [ 31 ] . Alternatively, many have measured the importance of different features by measuring the performance of feature subsets through iteratively adding or removing features [ 23 ] . These techniques can excel because they can identify the most relevant descriptors from a large feature set. Still, they may not be particularly informative when the features are heavily correlated or correspond to abstract concepts. Thus, while methods from unsupervised learning and supervised learning address aspects of QSPR, none can fully do so on their own. Consequently, this problem has motivated the extension of existing unsupervised learning algorithms to incorporate target information into their mapping strategies, with two beneficiaries of this direction having been UMAP and PCA. With UMAP, supervision can be added by intersecting the unsupervised UMAP graph constructed on the input features with a graph constructed on the targets. However, recent works have validated that because unsupervised UMAP is already designed for resolving clusters, adding supervision does not significantly alter performance [ 58 ] . Supervised PCA, on the other hand, involves modifying the principal components to instead identify directions in which the dependence between the input and output data is maximized [ 3 , 16 ] . More specifically, supervised PCA learns a linear (or kernel, in kernel supervised PCA) projection that maximizes an empirical approximation of the Hilbert-Schmidt independence criterion [ 18 ] , which itself is a function of the cross-covariance between the input and output data. While this formulation ensures that classes would be well-separated, its major shortcoming is that it does not explicitly preserve the variance of the input data. This can result in cases in which directions in the data that are correlated with the target due to collinearity are overrepresented in the map, despite the fact that these directions are likely noise. Concurrently, there has been an emergence of “visualization as interpretation” techniques that aim to provide a solution. Briefly, the vast majority of these methods aim to compute invertible mappings that represent the partitioning and clustering of data in higher-dimensional spaces [ 52 , 40 , 34 , 6 ] . For example, DeepView, the most notable of these methods, combines UMAP and Fisher distance for projection and then samples the resultant latent space on a grid to assess data boundaries [ 42 ] . However, computational cost and model complexity have remained large challenges for many of these methods, particularly those reliant on multi-layer architectures or grid-based assessments. And, methods based on high-nonlinear operations or architectures can result in topologically-bound latent spaces, where arbitrary points or pixels in a grid may not directly correspond to physically realizable data points [ 53 ] . We address these challenges by exploiting a family of methods, known as hybrid learning, that draws elements from both supervised and unsupervised learning. Some of these methods include principal components regression [ 24 ] , partial least squares regression [ 55 ] , cluster-wise regression [ 44 ] , continuum regression [ 45 ] , and principal covariates regression (PCovR) [ 12 ] . Of these methods, PCovR is particularly valuable for QSPR because it is directly formulated as a multi-objective problem in which the regression and dimensionality reduction tasks are optimized simultaneously. PCovR has broad applications within the fields of climate science [ 13 ] , economic forecasting [ 20 ] , psychology [ 50 ] , bioinformatics [ 10 , 49 , 46 ] , and molecular crystallization [ 9 ] . Similar to PCA, PCovR can also be used to incorporate nonlinear relationships through similarity kernels [ 21 ] , and can be used with regularization to create sparse solutions [ 13 , 49 ] . In addition, PCovR has been used to modify other techniques, such

Read the complete text