Improving Performance in Classification Tasks with LCEN and the Weighted Focal Differentiable MCC Loss

Pedro Seber [email protected], Richard D. Braatz [email protected] · 2026

Read Improving Performance in Classification Tasks with LCEN and the Weighted Focal Differentiable MCC Loss by Pedro Seber [email protected], Richard D. Braatz [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-21252.txt.

Introduction

Pedro Seber and Richard Braatz extend the LCEN algorithm, previously limited to regression, to classification while keeping its interpretability. On four binary and multiclass datasets, LCEN beats most of ten rival model types on macro F1 and Matthews correlation, while dropping an average of 56 percent of input features. Features LCEN selects also lift other models when retrained on them. A weighted focal differentiable MCC loss trained on the same data wins throughout, scoring 4.9 percent higher on macro F1 than weighted cross-entropy.

Catalog record

CREATORPedro Seber [email protected], Richard D. Braatz [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60862
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.21252

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Improving Performance in Classification Tasks with LCEN and the Weighted Focal Differentiable MCC Loss Pedro Seber [email protected], Richard D. Braatz [email protected] Abstract The LASSO-Clip-EN (LCEN) algorithm was previously introduced for nonlinear, interpretable feature selection and machine learning. However, its design and use was limited to regression tasks. In this work, we create a modified version of the LCEN algorithm that is suitable for classification tasks and maintains its desirable properties, such as interpretability. This modified LCEN algorithm is evaluated on four widely used binary and multiclass classification datasets. In these experiments, LCEN is compared against 10 other model types and consistently reaches high test-set macro F 1 score and Matthews correlation coefficient (MCC) metrics, higher than that of the majority of investigated models. LCEN models for classification remain sparse, eliminating an average of 56% of all input features in the experiments performed. Furthermore, LCEN-selected features are used to retrain all models using the same data, leading to statistically significant performance improvements in three of the experiments and insignificant differences in the fourth when compared to using all features or other feature selection methods. Simultaneously, the weighted focal differentiable MCC (diffMCC) loss function is evaluated on the same datasets. Models trained with the diffMCC loss function are always the best-performing methods in these experiments, and reach test-set macro F 1 scores that are, on average, 4.9% higher and MCCs that are 8.5% higher than those obtained by models trained with the weighted cross-entropy loss. These results highlight the performance of LCEN as a feature selection and machine learning algorithm also for classification tasks, and how the diffMCC loss function can train very accurate models, surpassing the weighted cross-entropy loss in the tasks investigated. 1 Introduction Classification problems are ubiquitous in the sciences, engineering, and other important areas. Many machine learning models for classification exist, each with different capabilities and desirable properties. Examples of desirable properties include interpretability (as defined by 27 , “how an output y=f(X) was predicted for a given input X — that is, provide f(\cdot) in a form readily understandable to humans”) and sparsity (“a model that uses few input features, particularly relative to the total number of features available.” ( 27 ) ). Interpretability is an essential property for a model to be trusted in critical or sensitive applications ( 13 ) , which are widely found in many fields. If an interpretable model is not used, interpretability may be obtained indirectly through a posteriori methods, such as Shapley values ( 29 ) or LIME ( 22 ) . These methods can be useful and increase the trustworthiness of models, but previous works have found that they are unreliable ( 24 ) , a problem exacerbated by “shortcuts” taken by some models ( 18 ; 23 ) . These “shortcuts” apparently increase predictive capabilities but are unrelated to the actual problem, leading to issues with predictive performance generalization. Feature selection is one technique to improve the sparsity of models, which, in turn, can improve interpretability to humans if the method is interpretable. Feature selection involves determining the most important features among those available. There are many methods for feature selection, which can be broadly classified into wrapper, filter, and embedded methods. A summary of feature selection methods for regression tasks, some of which also function for classification, can be found in the Introduction section of 27 . The works of 8 , 17 , and 16 provide comprehensive reviews on feature selection methods for different classification tasks (including, but not limited to, text mining or image-based classification). Recently, the LASSO-Clip-EN (LCEN) algorithm has been introduced for nonlinear, interpretable feature selection and machine learning in regression tasks ( 27 ) . LCEN had strong feature selection capabilities (comparable to that of thresholded EN, but LCEN was 10.3-fold faster) and machine learning performance (LCEN had low test RMSEs in multiple tested datasets and was among the best methods tested) ( 27 ) . However, an important limitation of 27 (noted in that work) is that it considered only regression tasks. Nevertheless, an analogous algorithm can be created and used for nonlinear, interpretable feature selection and machine learning in classification tasks. In opposition to what has been described above, there are also scenarios in which the priority is the maximization or minimization of a performance metric or metrics (such as the mean squared error (MSE), F 1 score, or precision), even at the cost of interpretability or other desirable properties. Artificial neural networks (ANNs) tend to dominate these scenarios due to their high potential for accurate modeling, in part because ANNs are universal function approximators ( 6 ; 14 ; 15 ) . As noted by 28 , in classification tasks, there is a discrepancy between what is optimized and the classification metrics. Deep learning models for classification are typically trained with the (weighted) cross-entropy (CE) loss, which aims to minimize the cross-entropy between the model’s predictions and the labels from data. However, classification models are then evaluated based on other classification metrics such as the F 1 score or the Matthews correlation coefficient (MCC). This discrepancy between the loss function and the target metric can mean that the model with the lowest cross-entropy is not necessarily the model with the best metrics. Classification metrics are typically not differentiable, so they cannot be directly used as a loss function, but a differentiable variant of the MCC was used by 28 to create the weighted focal differentiable MCC loss function (diffMCC). In contrast, this issue does not occur in regression tasks because the loss function and evaluation metric are typically the MSE, and most evaluation metrics are a function of y-\hat{y} , which is optimized by the MSE loss. In this work, a modification of the LCEN algorithm for classification tasks is also described and evaluated. This modified algorithm is tested on a wide range of classification problems, including binary and multiclass tasks related to the physical, biological, and social sciences. As done in 27 , ablated and variant LCEN algorithms for classification are tested to show that LCEN, and not other algorithms, is also optimal for the tasks investigated. In the four real datasets tested in this work, LCEN reaches higher macro F 1 scores and MCCs than the majority of machine learning models evaluated; its performance is only 7.3% lower in macro F 1 score and 15.0% lower in MCC than the best model of each dataset. LCEN is consistently among the top models in the datasets tested, something only MLPs can also achieve, and LCEN can even surpass these deep learning models in some cases. This high performance occurs at no cost to LCEN’s interpretability or sparsity. LCEN consistently created sparse models, eliminating an average of 56% of all input features and more than 50% of the input features on 3/4 of the tested datasets. Furthermore, LCEN’s feature selection capabilities are also tested by retraining every model on a dataset containing only features selected by LCEN. The average performance (in terms of macro F 1 scores and MCCs) of all machine learning models tested is improved in 3/4 of the tested datasets and unchanged in the last one when compared to the results obtained with all features or

Read the complete text