Self-Error Adjustment: Theory and Practice of Balancing Individual Performance and Diversity in Ensemble Learning
Rui Zou · 2025
Read Self-Error Adjustment: Theory and Practice of Balancing Individual Performance and Diversity in Ensemble Learning by Rui Zou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-04948.txt.
Introduction
Rui Zou proposes a new frame for ensemble learning called Self-Error Adjustment in two thousand twenty-five. Ensembles join guesses from many models to lift scores. The hard part is the trade off between each model's skill and the spread across models. Past tricks use chance or a fixed penalty, with loose bounds and a narrow range.
The frame splits ensemble error into self error terms and spread terms. A dial in the loss sets how much each part counts. That gives finer control and a wider range of change. Zou proves tighter bounds than past work. Tests on public data sets for class and value tasks beat the base methods.
Catalog record
| CREATOR | Rui Zou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-04948
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Self-Error Adjustment: Theory and Practice of Balancing Individual Performance and Diversity in Ensemble Learning Rui Zou Abstract Ensemble learning boosts performance by aggregating predictions from multiple base learners. A core challenge is balancing individual learner accuracy with diversity. Traditional methods like Bagging and Boosting promote diversity through randomness but lack precise control over the accuracy-diversity trade-off. Negative Correlation Learning (NCL) introduces a penalty to manage this trade-off but suffers from loose theoretical bounds and limited adjustment range. To overcome these limitations, we propose a novel framework called Self-Error Adjustment (SEA), which decomposes ensemble errors into two distinct components: individual performance terms, representing the self-error of each base learner, and diversity terms, reflecting interactions among learners. This decomposition allows us to introduce an adjustable parameter into the loss function, offering precise control over the contribution of each component, thus enabling finer regulation of ensemble performance. Compared to NCL and its variants, SEA provides a broader range of effective adjustments and more consistent changes in diversity. Furthermore, we establish tighter theoretical bounds for adjustable ensemble methods and validate them through empirical experiments. Experimental results on several public regression and classification datasets demonstrate that SEA consistently outperforms baseline methods across all tasks. Ablation studies confirm that SEA offers more flexible adjustment capabilities and superior performance in fine-tuning strategies. I Introduction Ensemble learning is a fundamental technique in machine learning, known for achieving superior performance by aggregating the predictions of multiple base learners [ 1 , 2 ] . This approach has demonstrated notable success across a wide range of fields, including image recognition [ 3 ] , speech recognition [ 4 ] , natural language processing [ 5 ] , healthcare [ 6 ] , environmental monitoring [ 7 ] , and large language models [ 8 ] . One of the central challenges in constructing high-performance ensemble models is balancing the accuracy of individual learners with the diversity among them, which has been a core topic in ensemble learning theory [ 9 , 10 ] . Diversity among base learners plays a critical role in enhancing ensemble performance [ 11 ] , as increased diversity helps mitigate overfitting and improve the generalization capabilities of the ensemble model. To further understand the role of diversity in ensemble learning, various error decomposition frameworks have been proposed. These frameworks break down ensemble error into components representing individual performance and diversity. For instance, [ 12 ] decomposed generalization error into an error term and an ambiguity term, while [ 13 ] introduced the bias-variance-covariance decomposition. Building on these concepts, [ 14 ] connected different decomposition methods and enhanced the Negative Correlation Learning (NCL) approach. More recently, works such as [ 15 ] and [ 16 ] utilized probabilistic and statistical theories to decompose ensemble error through multiplicative and additive rules, emphasizing the importance of diversity in human-machine collaboration. Traditional ensemble methods like Bagging [ 17 ] , Boosting [ 18 ] , and Random Forests [ 19 ] foster diversity indirectly by introducing randomness into the data or model space. However, these methods control diversity implicitly, making it challenging to achieve a precise balance between individual performance and diversity [ 1 ] . Negative Correlation Learning (NCL) [ 20 ] addresses this issue by introducing a penalty term into the loss function to explicitly regulate the trade-off between individual performance and diversity. While NCL has shown practical success, it has also led to several improved variants, such as Adaptive Negative Correlation Learning [ 21 ] , Regularized Negative Correlation Learning [ 22 ] , and Deep Negative Correlation Learning [ 23 , 24 ] . However, despite its success, NCL-based methods have certain limitations. First, their theoretical boundary conditions are relatively loose, leading to a large theoretical bound. Second, in practical applications, the range for effective adjustment is constrained, with substantial gaps between the achieved performance and the theoretical bound. Lastly, the adjustment strategy does not result in uniform changes in diversity, potentially causing the model to miss critical performance points. Moreover, most existing ensemble error decomposition theories are derived from a statistical framework, which lacks an intuitive interpretation of the training process. To address these issues, we propose a novel framework called Self-Error Adjustment (SEA). SEA decomposes the ensemble error from the perspective of training, breaking it into two components: individual performance terms representing the self-error of each base learner, and diversity terms capturing the interactions among learners. Based on this decomposition, we introduce an adjustable parameter into the loss function, enabling precise control over the ratio between individual performance and diversity. This allows for more flexible adjustment of ensemble performance. Through theoretical analysis, we derive strict bounds for general adjustable ensemble methods under the SEA framework. In traditional settings, the boundary condition is based on the “optimizability of the loss function”, typically ensured by the positive definiteness of the Hessian matrix [ 14 ] . While this condition is usually equivalent to “training can reduce ensemble error” for most tasks, it may not always hold in the context of adjustable ensemble learning. By considering both conditions, we establish a tighter theoretical bound than that of conventional methods. Figure 1 illustrates the relationship between the traditional theoretical bound (Original Bound), our proposed bound (Proposed Bound), and the actual bound (Real Bound) as the ensemble size M increases. As the figure shows, the actual bound aligns more closely with our proposed bound than with the traditional theoretical bound. Fig. 1: A comparison of the traditional theoretical bound (Original Bound), the proposed theoretical bound (Proposed Bound), and the actual bound (Real Bound). Adjustable parameters are incremented by 0.1, leading to an approximate estimate of the theoretical bound, which causes the actual bound to exhibit a step-like structure. In addition, we theoretically demonstrate that SEA provides a broader range of effective adjustments and more consistent diversity changes compared to NCL and its enhanced variant, NCL ∗ [ 14 ] . This conclusion is further supported by our ablation experiments. In these experiments, we compare SEA with several mainstream ensemble methods across multiple public regression and classification datasets. The results consistently show that SEA achieves superior performance in all tasks. Our main contributions are as follows: (1) To the best of our knowledge, we are the first to decompose ensemble error from the training perspective and propose the SEA framework, which effectively balances individual performance and diversity. (2) We conduct a comprehensive analysis of the SEA framework, deriving tighter theoretical bounds for adjustable ensemble methods and validating them experimentally. This leads to improved model training efficiency. (3) We theoretically and empirically show that the advantages of SEA lie in its broader adjustment range and more uniform changes in diversity. (4) We evaluate SEA on multiple regression and classification datasets, and the experimental results demonstrate that SEA consistently outperforms baseline methods in both task types. The remainder of this paper is organized as follows: Section II reviews the related work; Section III details the SEA framework and provides the theoretical analysis; Section IV presents the experimental results and discussions; Section V concludes the paper and outlines future work. II Related Work Ensemble learning, which improves generalization and robustness by combining multiple base learners, is a key area of research in