Feature Weighting Improves Pool-Based Sequential Active Learning for Regression

Dongrui Wu · 2026

Read Feature Weighting Improves Pool-Based Sequential Active Learning for Regression by Dongrui Wu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-02019.txt.

Introduction

Wu improves sequential active learning for regression by weighting features. Active learning picks samples in sequence from a large unlabeled pool, under a labeling budget. Representativeness and diversity rest on sample distances, yet earlier methods ignore feature importance, giving bad picks. Wu proposes four single-task and three multi-task weighted approaches, using ridge coefficients from a labeled set. Tests show the gain almost always improves five methods in both, and the idea extends to stream based learning and classification.

Catalog record

CREATORDongrui Wu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70236
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.02019

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Feature Weighting Improves Pool-Based Sequential Active Learning for Regression Dongrui Wu Abstract Pool-based sequential active learning for regression (ALR) optimally selects a small number of samples sequentially from a large pool of unlabeled samples to label, so that a more accurate regression model can be constructed under a given labeling budget. Representativeness and diversity, which involve computing the distances among different samples, are important considerations in ALR. However, previous ALR approaches do not incorporate the importance of different features in inter-sample distance computation, resulting in inaccurate distances and hence sub-optimal sample selection. This paper proposes four feature weighted single-task ALR approaches and three feature weighted multi-task ALR approaches, where the ridge regression coefficients trained from a small amount of previously labeled samples are used to weight the corresponding features in inter-sample distance computation. Extensive experiments showed that this intuitive and easy-to-implement enhancement almost always improves the performance of five existing ALR approaches, in both single-task and multi-task regression problems. The feature weighting strategy may also be easily extended to stream-based ALR, and classification algorithms. I Introduction In many real-world machine learning problems, acquiring a large amount of unlabeled data is relatively easy, but obtaining their labels is challenging, as it may be: 1. Labor intensive . For example, in affective computing [ 1 , 2 ] , it is easy to collect affective input signals such as sound, images, videos, etc., but labelling each piece of such signals may require the aggregation of individual labels from tens or more evaluators, due to the uncertainty and subtleness of emotions: the VAM ( Vera am Mittag in German, Vera at Noon in English) corpus [ 3 ] used 6-17 evaluators for each speech utterance, the DEAP (Database for Emotion Analysis using Physiological Signals) dataset [ 4 ] used 14-16 assessors for each video clip, the SEED-IV (Shanghai Jiao Tong University Emotion EEG Dataset) dataset [ 5 ] used 44 evaluators for each film clip, the GAPED (Geneva Affective Picture Database) dataset [ 6 ] used 60 college students for each picture, and the IADS-2 (International Affective Digitized Sound, 2nd Edition) dataset [ 7 ] used over 110 assessors for each piece of sound. 2. Time consuming . For example, in fracturing based enhanced oil recovery in the oil and gas industry [ 8 ] , 180-day post-fracturing cumulative oil production is usually used as a performance measure. The fracturing parameters of an oil well, e.g., location, length of perforations, number of zones/holes, volumes of injected slurry/water/sand, etc., can be easily recorded; however, to get the groundtruth output (180-day post-fracturing cumulative oil production), one has to wait at least 180 days, which is very time consuming. 3. Expensive . Usually one needs to pay to recruit tens or hundreds of evaluators for the affective sound/images/videos, so there is considerable cost to label each piece of affective input. Fracturing an oil well and observing its 180-day cumulative oil production is even more expensive. So, an important and realistic question is: how to achieve the best possible learning performance from a small number of labeled samples? Different strategies have been proposed for this purpose, e.g., active learning [ 9 ] , transfer learning [ 10 , 11 ] , self-supervised learning [ 12 ] , their combinations [ 13 , 14 , 15 , 16 ] , and so on. This paper considers pool-based sequential active learning, which optimally and sequentially selects a small number of samples from a large pool of unlabeled samples to label, so that a more accurate machine learning model can be constructed under a given (small) labeling budget. This is a very common active learning scenario. For example, in affective computing mentioned above, a large amount of affective input signals are usually collected offline before recruiting assessors for evaluation. There have been many pool-based sequential active learning approaches proposed for classification problems [ 9 , 17 ] , e.g., uncertainty sampling, expected model change maximization, query-by-committee, etc. However, pool-based sequential active learning for regression (ALR) problem is much less studied. As an evidence, the recent deep active learning survey [ 17 ] published in 2025 includes 205 references, among which only one is on ALR. This paper focuses on this under-explored pool-based sequential ALR problem. Previous research [ 18 ] has proposed three important criteria to be considered in pool-based sequential ALR for selecting the next unlabeled sample to label: 1. Informativeness , i.e., the selected sample must contain rich information, which could be measured by the disagreement among multiple learners [ 19 ] , or expected model change [ 20 ] , etc. 2. Representativeness , i.e., the selected sample should represent a large population, instead of being an outlier. 3. Diversity , i.e., all selected samples together should scatter across the full input space, instead of occupying only a small local region. Based on these three criteria, several ALR approaches have been proposed, e.g., query-by-committee [ 19 ] , greedy sampling [ 21 ] , expected model change maximization [ 20 ] , representativeness-diversity (RD) [ 18 ] , greedy sampling in the input space (GSx) [ 22 ] , greedy sampling in the output space [ 22 ] , improved greedy sampling (iGS) in both input and output spaces [ 22 ] , graph-based ALR (GALR) [ 23 ] , and the multi-task iGS (MT-iGS) [ 24 ] . Many of them, e.g., RD, GSx, iGS, GALR and MT-iGS, need to compute the distances among different samples when calculating the representativeness and/or diversity measures. However, their distance computation does not consider the weights associated with the input features (which can be estimated from the small amount of labeled samples), resulting in inaccurate distances and hence sub-optimal performance. This paper proposes a simple yet effective remedy, showing that by incorporating the estimated feature weights in inter-sample distance computation, ALR performance can be consistently improved. Our main contributions are: 1. We propose four feature weighted (FW) single-task ALR approaches, i.e., FW-RD, FW-GSx, FW-GALR and FW-iGS, to improve RD, GSx, GALR and iGS, respectively. 2. We propose three feature weighted multi-task (FW-MT) ALR approaches, i.e., FW-MT-GSx, FW-MT-GALR and FW-MT-iGS, to improve GSx, GALR (both GSx and GALR can be used in both single task ALR and multi-task ALR) and MT-iGS, respectively. 3. We perform extensive experiments to demonstrate that each of the seven feature weighted ALR approaches outperformed its corresponding unweighted version, and feature weighting benefits both linear and nonlinear prediction models. The remainder of this paper is organized as follows: Section II introduces five existing pool-based sequential ALR approaches, i.e., RD, GSx, iGS, GALR and MT-iGS, which need to compute the distances among different samples. Section III proposes seven feature weighted ALR approaches, FW-RD, FW-GSx, FW-iGS, FW-GALR, FW-MT-GSx, FW-MT-iGS and FW-MT-ALR. Sections IV and V perform experiments to demonstrate the effectiveness and robustness of the single-task and multi-task feature weighted ALR approaches, respectively. Finally, Section VI draws conclusions and points out future research directions. II Related Work This section introduces five existing pool-based sequential ALR approaches, RD [ 18 ] , GSx [ 22 ] , iGS [ 22 ] , GALR [ 23 ] and MT-iGS [ 24 ] , which involve inter-sample distance computation in sample selection.

Read the complete text