Unbiased mixed variables distance

Michel van de Velden · 2024

Read Unbiased mixed variables distance by Michel van de Velden (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00429.txt.

Description

Abstract Defining a distance in a mixed setting requires the quantification of observed differences of variables of different types and of variables that are measured on different scales. There exist several proposals for mixed variable distances, however, such distances tend to be “biased” towards specific variable types and measurement units.

Catalog record

CREATORMichel van de Velden
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-00429

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Unbiased mixed variables distance Michel van de Velden Abstract Defining a distance in a mixed setting requires the quantification of observed differences of variables of different types and of variables that are measured on different scales. There exist several proposals for mixed variable distances, however, such distances tend to be “biased” towards specific variable types and measurement units. That is, the variable types and scales influence the contribution of individual variables to the overall distance. In this paper, we define “unbiased” mixed variable distances for which the contributions of individual variables to the overall distance are not influenced by measurement types or scales. We define the relevant concepts to quantify such biases and we provide a general formulation that can be used to construct unbiased mixed variable distances. Keywords: Data visualization, distance-based methods, cluster analysis 1 Introduction In many fields of study, it is necessary to measure the distance or similarity between observations or data points. These measurements are crucial for various tasks, including data visualization, clustering and classification. A significant challenge arises when a dataset consists of mixed variables, that is, when the observed variables differ in type or scale ( Blanco-Mallo et al. 2023 ) . Mixed variable datasets can consist of both numerical variables (continuous or discrete) and categorical variables (binary, ordinal, or nominal). These datasets are often encountered in machine learning and scientific computing tasks ( Luo et al. 2024 ; Hermes et al. 2024 ) . In a purely numerical context distance can be based on the sizes of the observed differences. For categorical variables, the situation is more complex as one only observes whether categories are equivalent or not. However, it may be appropriate to quantify differences between categories of the same variable differently. For example, one could argue that the dissimilarity between the colors purple and blue is smaller than the dissimilarity between yellow and blue. Similarly, the number of categories could play a role in quantifying differences. If there are many categories, there are more ways to differ. Many proposals exist to quantify distance for categorical variables; see, e.g. Boriah et al. 2008 ; Šulc & Řezanková 2019 ; van et al. 2024 . The complexity arising from quantifying differences and the impact that these choices have in a clustering context when working with categorical data is thoroughly demonstrated by Amiri et al. 2018 . For mixed variables, the situation is even more complex, as this requires a method of calculating distance that can handle the diversity in variable types effectively, i.e., the aggregation over the variables should not be influenced by types or scales. One of the most recognized and widely used methods for calculating mixed variable distances is based on Gower’s general coefficient of similarity ( Gower 1971 ) . This measure, which we refer to as Gower’s distance, provides an elegant and simple way to measure the distance between observations based on measurement on numerical and categorical variables. Although alternative proposals that allow distance calculations in mixed variable contexts exist (see, e.g., Ahmad & Dey 2007 ; Huang 1998 ; van et al. 2019 ) , Gower’s proposal – implemented in the gower R package ( Loo 2017 ) and in the daisy function available through the cluster R package ( Maechler et al. 2023 ) – remains a popular choice. It has been observed that biases can arise when computing distances in a multivariate context. For example, in numerical data, measurement scales can influence distances. Similarly, for categorical variables, the number of categories can affect the calculation of distances. Additionally, in settings with mixed variable types, it is important to control the balance between these types. For example, the Gower’s distance is known to be biased toward the contribution of categorical variables ( Hennig & Liao 2013 ; Foss et al. 2016 ) . One way to overcome potential biases is to weigh the distances resulting from the numerical and categorical variables differently ( Huang 1998 ; Bello et al. 2021 ) . Alternatively, different ways to scale the numerical and categorical variables may be introduced ( Hennig & Liao 2013 ) . In this paper, we consider the problem of biased mixed variables distance. In particular, we propose a general formulation for multivariate mixed variable distances that can be used to construct unbiased mixed variable distances. To achieve this, we consider two properties: 1) multivariate additivity and 2) commensurability . Multivariate additivity means that the multivariate distance is the sum of univariate distances. The commensurability requirement ensures that the univariate contributions to the overall distance are commensurable (i.e., corresponding in size, extent, amount or degree). We show that existing mixed variable distances lead to biased distances, and we derive the magnitude of such biases. We evaluated several specific mixed variable distances and show that our new unbiased mixed variable distances provide a more objective starting point in an exploratory distance-based data analysis. This paper is organized as follows. After introducing the two desirable properties for multivariate mixed variable distances in Section 2 , we propose a general multivariate mixed variable distance formulation that can be used to construct mixed variable distances that satisfy these properties. In Sections 4 and 5 , we show how several common distance and scaling options – for numerical and categorical variables – violate the properties of Section 2 and hence may lead to biased mixed variable distances. In Section 6 we show, using our general distance formulation, how unbiased mixed variable distances can be constructed. Using a simulation study and an empirical application, we illustrate our findings in Sections 7 and 8 , and end the article with our conclusions. 2 Properties of a multivariate mixed variable distance A multivariate mixed variable distance requires the quantification and aggregation of distances for different variables. Aggregation concerns “joining” or the accumulation of variable-specific distances. For this aggregation, we only consider distances that are “additive”. We define multivariate additivity as the property that in a p- dimensional setting, the distance can be expressed as the sum of p distances. That is, Definition 2.1 (Multivariate Additivity) Let \mathbf{x}_{i}=\left(x_{i1},\dots,x_{ip}\right) denote a p- dimensional vector. A distance function d\left(\mathbf{x}_{i},\mathbf{x}_{l}\right) between observations i and l is multivariate additive if d\left(\mathbf{x}_{i},\mathbf{x}_{l}\right)=\sum_{j=1}^{p}d_{j}\left(\mathbf{x}_{i},\mathbf{x}_{l}\right), where d_{j}\left(\mathbf{x}_{i},\mathbf{x}_{l}\right) denotes the j- th variable specific distance. Note that the variable-specific distances d_{j} are functions of the p- dimensional vectors. These functions can be used to take into account associations between the p variables. The Manhattan distance for d\left(\mathbf{x}_{i},\mathbf{x}_{l}\right) satisfies the additivity property, whereas the Euclidean distance, which takes the square root of the sum of squared differences, does not. Euclidean distance implicitly assumes that the variables jointly span a space. It is then defined by considering the length of a straight line connecting the points in this high-dimensional space. In a multivariate context, the contribution per variable for the Euclidean distance decreases as more variables are included. By defining the overall multivariate distance d\left(\mathbf{x}_{i},\mathbf{x}_{l}\right) as the sum of variable-specific distances, each variable-specific distance d_{j}\left(\mathbf{x}_{i},\mathbf{x}_{l}\right) is treated equivalently. This requires these distances to be on similar or equivalent scales. To achieve this we introduce an additional property: commensurability. In particular, we define distances as commensurable if the mean variable-specific distance between observations is constant across variables. That

Read the complete text