Estimating Conditional Covariance between labels for Multilabel Data

Laurence A. F. Park, Jesse Read · 2025

Read Estimating Conditional Covariance between labels for Multilabel Data by Laurence A. F. Park, Jesse Read (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-18951.txt.

Introduction

Laurence Park and Jesse Read compare three models for measuring label links in multi label data in a 2025 paper. Labels in such data lean on the input traits, so raw label counts cannot show true dependence. A Probit model is the usual probe, yet it reads a transformed scale that may mislead.

Their test bed lets them watch each model at work on known covariance patterns. All three models read fixed and input driven links about as well, based on the strength of the link. All three also falsely claim input driven links when only fixed ones exist. The Probit model made the fewest errors of the three.

Catalog record

CREATORLaurence A. F. Park, Jesse Read
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-18951

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Estimating Conditional Covariance between labels for Multilabel Data Laurence A. F. Park, Jesse Read Abstract Multilabel data should be analysed for label dependence before applying multilabel models. Independence between multilabel data labels cannot be measured directly from the label values due to their dependence on the set of covariates x → \vec{x} , but can be measured by examining the conditional label covariance using a multivariate Probit model. Unfortunately, the multivariate Probit model provides an estimate of its copula covariance, and so might not be reliable in estimating constant covariance and dependent covariance. In this article, we compare three models (Multivariate Probit, Multivariate Bernoulli and Staged Logit) for estimating the constant and dependent multilabel conditional label covariance. We provide an experiment that allows us to observe each model’s measurement of conditional covariance. We found that all models measure constant and dependent covariance equally well, depending on the strength of the covariance, but the models all falsely detect that dependent covariance is present for data where constant covariance is present. Of the three models, the Multivariate Probit model had the lowest error rate. I Introduction Multilabel data consisting of a set of labels \vec{y}\in\{0,1\}^{L} with covariates \vec{x}\in\mathbb{R}^{M} appears in many domains of science (e.g. [ 1 , 2 , 3 ] ). Each label y_{i} in a multilabel data set is modelled as a function of the covariates \vec{x} , where the multilabel model provides the additional precision by also modelling the dependence between labels. If there is no dependence between some of the labels, the data can be split and modelled separately; if there is no dependence between all labels each label can be modelled separately using a binary classifier, providing a simpler and faster analysis. Therefore, before fitting a multilabel model, the multilabel data should be examined to determine the relationship between labels. It is a common misconception that a multilabel model is required if there is dependence between the labels, when in fact all labels will be dependent due to each label being a function of \vec{x} . Therefore, instead of checking for independence, the conditional independence of each label y_{i} conditioned on \vec{x} is required [ 4 ] . Previous studies have investigated the utility of the multivariate Probit model for assessing conditional independence of labels [ 5 ] . Unfortunately, initial investigations reveal that the multivariate Probit model is not measuring the label dependence directly [ 6 ] . In this article, we evaluate the utility of three analytical models for assessing the conditional covariance of labels in multilabel data (Multivariate Probit, Multivariate Bernoulli and Staged Logit). The models are presented and a controlled experiment is run to identify the ability of each model to detect conditional covariance between labels. The contributions of this article are: • A detailed description of the problem when using the multivariate Probit model for detecting label dependence (Section III-C ). • Presentation of two alternative models for assessing label dependence (Section IV ). • An experiment assessing each model’s ability to assess label dependence (Section V ). • An examination of each model’s approximations (Section VI ). Note that we are not proposing a new multilabel classification model. We are investigating the use of analytical models for detecting conditional covariance in multilabel data. The article will proceed as follows: Section II introduces multilabel classification and the need to measure conditional covariance in multilabel data, Section III describes the multivariate Probit model and its problems. Section IV presents two alternative models for measuring label dependence. Section V provides an experiment and the results from the three models. Finally, Section VI examines the models to provide insight into the experimental results. II Background Multilabel classification is a generalisation of Binary classification from one response to many responses, called labels. A multilabel classifier takes input vector \vec{x}\in\mathbb{R} and provides an associated set of L binary predictions \vec{y}\in\{0,1\}^{L} , one for each of the L labels. For example, the input data could be a vector representing an retinal image and the output be the absence or presence of a set of L diseases detected from the image. A multilabel classifier has a label space of size 2^{L} (the number of possible label states) and therefore can be mapped to a multiclass classification problem predicting one of the 2^{L} possible states. As L increases, the label space grows exponentially; for example for 50 labels, there are 1.1259\times 10^{15} possible solutions making the multilabel classification task difficult and time consuming when L is large. If we assume that all of the labels are independent, a multilabel classification problem over L labels can be split into a set of L binary classifiers. If the labels are not independent, we can take advantage of the dependence to improve the classification accuracy, but the complexity of the model increases. Numerous methods have been proposed to deal with the large label space [ 7 , 8 , 9 , 10 , 11 , 12 ] . For example, label powerset subset methods [ 13 , 14 ] , replace the powerset \{0,1\}^{L} with multiple subsets providing a smaller solution space. Classifier chain methods [ 15 ] use the conditional probability chain rule to traverse parts of the solution [ 16 ] , seek optimal chain structures [ 17 ] or use fixed structures [ 18 ] rather than a fully-cascaded chain. We can see that modelling sets of labels (rather than single labels) leads to increased classification accuracy, but it also leads to greater complexity. Therefore multilabel classification should only be used when there is dependence between labels to exploit. If we find that from a set of 20 labels (with solution space 2^{20}=1048576 ) the first 10 labels are independent of the second 10, the data can be split into two 10 label problems (with solution space 2\times 2^{10}=2048 ), which maintains the possible classification accuracy, but reduces the solution space by a factor of 512. Therefore, before performing multilabel classification, the data labels should be examined and dependencies be identified. But what exactly are we measuring to determine if there is dependence between labels? Two random variables are considered Independent if \displaystyle P(Y_{i},Y_{j})=P(Y_{i})P(Y_{j}) Assuming that each label y_{i} is a Bernoulli random variable Y_{i} with parameter p_{i} (such that P(Y_{i}=1)=p_{i} ), the independence equation becomes \displaystyle p_{ij}=p_{i}p_{j} where p_{ij}=P(Y_{i}=1,Y_{j}=1) . The covariance between two Bernoulli random variables is \displaystyle\rho_{ij} \displaystyle=\mathbb{E}[Y_{i}Y_{j}]-\mathbb{E}[Y_{i}]\mathbb{E}[Y_{j}] \displaystyle=p_{ij}-p_{i}p_{j} Therefore, if the covariance is zero the random variables are independent, if the covariance is not zero, the random variables are not independent. For multilabel data, all labels should have covariance, simply due to them all being dependent on \vec{x} . Therefore the independence between labels can be measured by the labels’ conditional covariance (the label covariance conditioned on \vec{x} ). Therefore, to determine if a pair of labels are dependent we are required to model \displaystyle\rho_{ij}=f(\vec{x}) If we find f(\vec{x})=0 , then the labels are independent. If f(\vec{x}) is constant, then the conditional covariance is independent of \vec{x} . otherwise it is dependent on \vec{x} . Analytical models have been proposed to measure conditional covariance between labels [ 5 , 19 ] , but they only examined constant conditional covariance (the conditional covariance might depend on

Read the complete text