Learning to Score
Yogev Kriger, Shai Fine · 2025
Read Learning to Score by Yogev Kriger, Shai Fine (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-14302.txt.
Introduction
Yogev Kriger and Shai Fine wrote a machine learning paper in two thousand twenty-five on learning to score when true labels are missing. Many tasks sit between labeled and unlabeled data. Here no target labels exist at all, but side information is at hand: facts that correlate with the labels or that constrain the data.
Their model joins three parts: learning a good representation, using the side information, and learning a distance rule. One use is in health care, where symptoms are known but disease stages lack clear criteria. A severity score can still be built. They show the system working on standard data sets and on patient records.
Catalog record
| CREATOR | Yogev Kriger, Shai Fine |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-14302
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning to Score Yogev Kriger, Shai Fine Abstract Common machine learning settings range from supervised tasks, where accurately labeled data is accessible, through semi-supervised and weakly-supervised tasks, where target labels are scant or noisy, to unsupervised tasks where labels are unobtainable. In this paper we study a scenario where the target labels are not available but additional related information is at hand. This information, referred to as Side Information, is either correlated with the unknown labels or imposes constraints on the feature space. We formulate the problem as an ensemble of three semantic components: representation learning, side information and metric learning. The proposed scoring model is advantageous for multiple use-cases. For example, in the healthcare domain it can be used to create a severity score for diseases where the symptoms are known but the criteria for the disease progression are not well defined. We demonstrate the utility of the suggested scoring system on well-known benchmark data-sets and bio-medical patient records. I Introduction Various data-driven methods have been developed over the years; supervised methods were devised for cases where ample labeled data is available. Semi-supervised methods were developed for cases where labels are well-defined but the labeled data is less available and difficult to acquire. Weak supervision has been suggested to handle cases where labels are either noisy, imprecise, or incomplete (i.e. weakly labeled data). Unsupervised methods were developed for situations where it is difficult or impossible to obtain labeled data. We introduce a model that aims to solve problems for which we do not observe any label, but labels do exist. The goal of the model is to create a score that would represent the true, unseen labels. This is achieved by utilizing characteristics of the instances and additional data, referred to as side information. Following the learning process, the model produces a score that is representative of the unseen labels. The proposed framework showcases its versatility by accommodating a wide spectrum of scenarios, ranging from cases where the side information offers minimal insight about the label, to cases where it is richly informative. This spectrum encapsulates two extremes - an unsupervised task, where the side information bears no correlation to the label, and a supervised task, where the side information is, in fact, the label itself. It is applicable for many problems in various fields: Consider the emergence of a novel, previously unidentified disease. The Medical community lacks the requisite knowledge to define a disease-severity score, not knowing which factors contribute to the disease prognosis. However, the experts could provide some reasonable assumptions. For example, observing a decrease in blood saturation is positively correlated with worse prognosis. In this case, saturation could be used as side information and combined with additional clinical records of the patients as model input, could be used to create a disease-severity score. Another possible application is scoring marketing leads using life-time value (an estimate of the revenue customers will generate) as the side information. The model can also be used for quality assurance, to score the yield of an assembly line using, as a side information, lot level production factors and SLAs (service-level agreement) along the manufacturing process. In the educational sector, the model can leverage related academic performance metrics as side information to enrich the understanding of students’ performance in other domains, paving the way for a more holistic and individualized approach to student assessment. We created such a generic model by combining several machine learning techniques. In a nutshell, our approach can be described as representation learning constrained by the side information. The model receives the side information in two possible forms: information that is related to the labels and information related to the underlying structure of the data that is incorporated using metric learning. We used a generative modeling approach to construct a latent space but strived to form a representation that is as discriminative as possible. The rest of the paper is organized as follows - next, we describe a few related works. In section II we provide details of the fundamental concepts we used for the development of the model. In section III we introduce and thoroughly explain the various components of the scoring model. In section IV we perform multiple ablation studies and demonstrate the contribution of each component to the model. Furthermore, we show the ability of the model to handle various scenarios, ranging from supervised to unsupervised. We then turn to illustrate how the scoring model can address real-life setting, by applying it to learn the well known UPDRS severity score for Parkinson Disease patients, using the motoric UPDRS as side information. We conclude in section V with some insights and suggestions for further research. I-A Related work To some extent, score learning is a novel task. In what follows, we present the most related tasks and models, and highlight the similarities and differences to score learning. I-A 1 Variational Information Bottleneck for Semi-Supervised Classification In their paper, Voloshynovskiy et al. [ 1 ] construct a new formulation of Information Bottleneck (IB) for semi-supervised classification and use a variational decomposition to convert it into a practically tractable setup with learnable parameters. Additionally, they suggest using learnable priors on the latent space of classifiers together with different regularizes. Our multi-task architecture can be viewed as dimensionality reduction with regularizes, similar to the architecture suggested in their work. Both works share concepts and components such as IB, reconstruction and classification space constraints. However, the setup is different - in our model there is no direct knowledge of the target or access to the target labels. Rather, knowledge of the target is provided indirectly, in the form of side information, and constraints over the feature space, which in turn are being utilized by the metric learning component. I-A 2 Learning using privileged information Learning using privileged information (LUPI) [ 2 ] is a machine learning paradigm where models are trained using both standard input data and additional ”privileged” information. This privileged information can include data that is not available during test time, such as detailed annotations of training data, and can be used to improve model performance on a wide range of tasks. For example, when the objective is to classify biopsy images to two categories - images that contain a cancerous tumor and images that do not. When the images are accompanied by pathologists reports, describing the images in a high level holistic manner, they could be used as ”privileged” information to find a good classification rule in the pixel space. The privileged information can be considered as a form of side information. However the LUPI setting differs from ours, since on one hand we don’t assume any access to the target labels, and on the other hand we may allow access to the side information during test time as well. I-A 3 Weak supervision Weak supervision [ 3 ] is a technique used in machine learning where models are trained using weakly labeled data, instead of a fully labeled data. Weakly labeled data refers to data that is not exhaustively labeled, such as data with labels that are noisy, imprecise, or incomplete. This approach can help overcome the limitations of traditional supervised learning that relies on large