Quantification of Uncertainties in Probabilistic Deep Neural Network by Implementing Boosting of Variational Inference
Pavia Bera, Sanjukta Bhanja · 2025
Read Quantification of Uncertainties in Probabilistic Deep Neural Network by Implementing Boosting of Variational Inference by Pavia Bera, Sanjukta Bhanja (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13909.txt.
Introduction
Pavia Bera and Sanjukta Bhanja propose Boosted Bayesian Neural Networks, a way to sharpen uncertainty estimates in neural networks. Their 2025 paper starts from variational inference, which replaces fixed weights with weight distributions. The standard approach leans on a single density, which can miss the true shape of the posterior and hurt performance.
Boosting builds the approximation as a mixture of densities, added step by step, so the family can bend closer to the real posterior. Tests showed about five percent higher accuracy than conventional networks, with better uncertainty scores. The cost is more computation. The authors pitch the trade as worth it in high stakes fields like medical diagnosis, where a missed case is costly.
Catalog record
| CREATOR | Pavia Bera, Sanjukta Bhanja |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-13909
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Quantification of Uncertainties in Probabilistic Deep Neural Network by Implementing Boosting of Variational Inference Pavia Bera, Sanjukta Bhanja Abstract Modern neural network architectures have achieved remarkable accuracies but remain highly dependent on their training data, often lacking interpretability in their learned mappings. While effective on large datasets, they tend to overfit on smaller ones. Probabilistic neural networks, such as those utilizing variational inference, address this limitation by incorporating uncertainty estimation through weight distributions rather than point estimates. However, standard variational inference often relies on a single-density approximation, which can lead to poor posterior estimates and hinder model performance. We propose Boosted Bayesian Neural Networks (BBNN) , a novel approach that enhances neural network weight distribution approximations using Boosting Variational Inference (BVI). By iteratively constructing a mixture of densities, BVI expands the approximating family, enabling a more expressive posterior that leads to improved generalization and uncertainty estimation. While this approach increases computational complexity, it significantly enhances accuracy—an essential tradeoff, particularly in high-stakes applications such as medical diagnostics, where false negatives can have severe consequences. Our experimental results demonstrate that BBNN achieves ∼ \sim 5% higher accuracy compared to conventional neural networks while providing superior uncertainty quantification. This improvement highlights the effectiveness of leveraging a mixture-based variational family to better approximate the posterior distribution, ultimately advancing probabilistic deep learning. August 24, 2026 K eywords Neural Network \cdot Bayesian Statistics \cdot Variational Inference \cdot Boosting of Variational Inference \cdot Bayes Backpropagation 1 Introduction The classical neural network was first proposed by McCullough and Walter Pitts in 1943 [ 1 ] , marking a significant milestone in the development of artificial intelligence. These early models laid the foundation for modern neural networks, which have since evolved into deep learning architectures that achieve state-of-the-art performance across various domains [ 2 , 3 ] . However, despite their success, deep neural networks (DNNs) remain heavily reliant on large amounts of labeled training data, leading to challenges in generalization, particularly in data-scarce environments [ 4 ] . One of the fundamental issues with conventional DNNs is their inability to quantify uncertainty in predictions. Standard neural networks employ deterministic weight estimates, making them susceptible to overfitting and poorly calibrated confidence estimates [ 5 , 6 ] . This limitation is particularly problematic in high-stakes applications such as medical diagnosis [ 7 ] , financial modeling [ 8 ] , and autonomous systems [ 9 ] , where uncertainty estimation is crucial for decision-making. Bayesian Neural Networks (BNNs) address this issue by treating weights as probability distributions rather than fixed values, thereby incorporating an inherent mechanism for uncertainty quantification [ 10 ] . Variational inference (VI) has emerged as a scalable approach to approximating Bayesian posterior distributions in neural networks. However, traditional VI techniques often rely on simple variational families that fail to capture complex, multimodal posteriors [ 11 ] . This shortcoming can lead to suboptimal generalization, as the approximated posterior lacks the flexibility to model uncertainty effectively. Furthermore, standard Bayesian methods such as Markov Chain Monte Carlo (MCMC) can be computationally expensive, limiting their applicability to large-scale deep learning models [ 12 ] . To address these limitations, we propose Boosted Bayesian Neural Networks (BBNNs) , which leverage Boosting Variational Inference (BVI) to enhance weight distribution approximations. Unlike conventional variational methods that use a single variational density, BVI iteratively constructs a mixture of densities, enabling a more expressive and accurate posterior representation [ 8 ] . This approach provides a flexible alternative to traditional inference techniques, allowing neural networks to capture a broader range of uncertainties while maintaining computational efficiency [ 13 ] . BBNNs not only provide superior uncertainty quantification but also demonstrate improved generalization in limited-data settings. Traditional deep learning models require vast amounts of labeled data to generalize effectively, whereas BBNNs, through their probabilistic framework, are inherently more robust to data scarcity [ 14 ] . This makes them particularly valuable in biomedical applications, fraud detection [ 15 ] , and adversarially robust learning systems [ 16 ] . Moreover, Bayesian learning methods have been shown to mitigate adversarial vulnerabilities by incorporating uncertainty awareness in decision-making [ 17 ] . Beyond uncertainty quantification, BBNNs offer several computational advantages. By integrating boosting techniques into variational inference, BBNNs strike a balance between model complexity and computational efficiency [ 18 ] . Unlike standard BNNs, which often require extensive MCMC sampling [ 12 ] , BVI enables a more scalable and computationally feasible approximation of the posterior. Recent advancements in Bayesian deep learning, such as Gaussian processes [ 14 ] and Bayesian dropout [ 17 ] , further highlight the practical relevance of our proposed approach. Another key advantage of BBNNs is their adaptability to dynamic environments. Traditional deep learning models often struggle with distribution shifts and non-stationary data. However, BBNNs dynamically update their uncertainty estimates, making them well-suited for real-time decision-making applications, such as robotics [ 19 ] , edge AI [ 20 ] , and federated learning [ 15 ] . The ability to continuously refine posterior approximations ensures that BBNNs remain robust even in evolving data landscapes. BBNNs also contribute to the broader field of Bayesian optimization. Bayesian optimization techniques, which rely on probabilistic models to guide the search for optimal hyperparameters, have been widely applied in deep learning [ 21 , 22 ] . Our method enhances Bayesian optimization by incorporating a more accurate uncertainty model, improving sample efficiency and convergence rates. This makes BBNNs a powerful tool for tasks requiring adaptive learning, such as automated machine learning (AutoML) and self-improving AI systems. Moreover, BBNNs can be integrated with Bayesian active learning strategies to improve data efficiency. Active learning, which involves selecting the most informative samples for labeling, benefits from accurate uncertainty estimates. By leveraging the uncertainty-aware properties of BBNNs, active learning models can significantly reduce the amount of labeled data required for training while maintaining high predictive performance [ 23 ] . This is particularly useful in domains where data annotation is expensive, such as medical imaging and genomics. Furthermore, Bayesian learning has gained traction in neuroscience and cognitive modeling. The brain is often hypothesized to function as a Bayesian inference engine, continuously updating beliefs based on incoming sensory data [ 24 ] . By adopting a probabilistic learning framework, BBNNs align more closely with biological neural systems, making them valuable for modeling human decision-making and perception. Additionally, Bayesian methods have shown promise in reinforcement learning [ 25 ] , where they enable more efficient exploration strategies and adaptive behavior in robotic systems. Scaling Bayesian inference to deep learning remains an ongoing challenge due to the high-dimensional nature of modern neural networks. Techniques such as stochastic optimization and local reparameterization tricks [ 26 ] have improved the scalability of Bayesian methods, but computational efficiency remains a key concern. Future research in Bayesian deep learning will likely focus on developing more efficient inference techniques, potentially leveraging quantum computing and neuromorphic hardware to accelerate probabilistic computations [ 27 ] . In this work, we present a comprehensive analysis of BBNNs, demonstrating their superiority in terms of predictive accuracy and uncertainty quantification. Our experimental results indicate that BBNNs achieve \sim 5% higher accuracy compared to conventional neural networks while maintaining significantly better-calibrated uncertainty estimates. This