Disentangling Uncertainties by Learning Compressed Data Representation

Zhiyu An, Zhibo Hou, Wan Du · 2025

Read Disentangling Uncertainties by Learning Compressed Data Representation by Zhiyu An, Zhibo Hou, Wan Du (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-15801.txt.

Introduction

Zhiyu An, Zhibo Hou, and Wan Du separate two kinds of doubt in learned models of moving systems, in a 2025 study. One kind stems from the system's own randomness; the other from lack of data. Splitting them matters for safe control, smart exploration, and policy transfer. Older tools are either slow or sloppy at the split. Their model learns a compressed code of the data spread, then samples outputs with a Langevin method.

The model can predict any output shape, not just bell curves. On a mixed test set it flagged the two doubt kinds with high scores. It also handled multi-peak outputs where rivals fail. Theory shows it beats bin-based compression on memory and compute cost. Code is public.

Catalog record

CREATORZhiyu An, Zhibo Hou, Wan Du
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-15801

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Disentangling Uncertainties by Learning Compressed Data Representation Zhiyu An, Zhibo Hou, Wan Du Abstract We study aleatoric and epistemic uncertainty estimation in a learned regressive system dynamics model. Disentangling aleatoric uncertainty (the inherent randomness of the system) from epistemic uncertainty (the lack of data) is crucial for downstream tasks such as risk-aware control and reinforcement learning, efficient exploration, and robust policy transfer. While existing approaches like Gaussian Processes, Bayesian networks, and model ensembles are widely adopted, they suffer from either high computational complexity or inaccurate uncertainty estimation. To address these limitations, we propose the Compressed Data Representation Model (CDRM), a framework that learns a neural network encoding of the data distribution and enables direct sampling from the output distribution. Our approach incorporates a novel inference procedure based on Langevin dynamics sampling, allowing CDRM to predict arbitrary output distributions rather than being constrained to a Gaussian prior. Theoretical analysis provides the conditions where CDRM achieves better memory and computational complexity compared to bin-based compression methods. Empirical evaluations show that CDRM demonstrates a superior capability to identify aleatoric and epistemic uncertainties separately, achieving AUROCs of 0.8876 and 0.9981 on a single test set containing a mixture of both uncertainties. Qualitative results further show that CDRM’s capability extends to datasets with multimodal output distributions, a challenging scenario where existing methods consistently fail. Code and supplementary materials are available at https://github.com/ryeii/CDRM . 1 Introduction The disentanglement of epistemic and aleatoric uncertainties plays a crucial role in safe control and reinforcement learning Brunke et al. (2022) . One notable example is the problem of efficient exploration with limited resources. Consider a robot operating with a finite power supply for world exploration. To optimize its power usage, the robot must selectively choose actions that generate the most informative data for reducing its world model’s uncertainty. This process is known as curiosity-driven exploration Pathak et al. (2017) . However, model uncertainty can stem from two distinct sources: epistemic uncertainty , which arises from a lack of data (e.g., the unknown furniture arrangement in an unexplored room) and is reducible through additional data collection; and aleatoric uncertainty , which results from inherent environmental randomness (e.g., the outcome of a dice throw) and remains irreducible regardless of additional data (see Hüllermeier and Waegeman (2021) for a review). This distinction becomes critical for resource-efficient exploration, as it would be wasteful for the robot to expend its limited power gathering data in hopes of predicting an inherently random event. States with irreducible aleatoric uncertainty are called stochastic traps Shyam et al. (2019) or noisy TVs Schmidhuber (1991) , with the latter term derived from the observation that a naively curious agent could dwell on the unpredictability of a noisy TV screen. Several existing works aim to avoid the noisy TV problem through uncertainty estimation using Deep Ensemble Lakshminarayanan et al. (2017) ; Burda et al. (2018) ; Pathak et al. (2019) , Gaussian Process (GP) Wachi et al. (2018) , or learning a separate model to predict uncertainty Mavor-Parker et al. (2022) , but they either cannot accurately disentangle the two types of uncertainties or have high computational complexity. Aside from efficient exploration, uncertainty estimation is also important for ensuring the safety of reinforcement learning agents Brunke et al. (2022) . For example, the variance output by a GP system dynamics model is used as an estimation of epistemic uncertainty, which is used to filter potentially unsafe actions An et al. (2023) ; Zhao et al. (2023) . These works also suffer from inaccurate estimation and high computational complexity. Traditional uncertainty estimation for regressive models has strong baselines such as GP Duvenaud (2014) , MC Dropout Gal and Ghahramani (2016) , Bayesian Network Blundell et al. (2015) , and Deep Ensemble Lakshminarayanan et al. (2017) . However, they suffer from the aforementioned disadvantages. Deep Evidential Learning (DER) Amini et al. (2020) uses neural networks to predict the parameters of the output distribution. To get an accurate prediction, it requires the output distribution type to be known, while in real-world datasets such distribution types are often unknown. Chan et al. (2024) proposes a hyper-network that produces the parameters of multiple downstream diffusion networks and uses the downstream networks as an ensemble for prediction. This method shows empirical improvement compared to Deep Ensemble but suffers from higher computational complexity due to more model forward passes. On the other hand, density-based methods Sun et al. (2023) ; Yoon and Kim (2024) ; Bui and Liu (2024) have shown promising results in out-of-distribution (OOD) detection or novelty detection for classification tasks. These methods estimate the data density of the dataset at a given input; the data density is then used to decide whether the input is OOD. Such methods have not yet been extensively adopted and tested in regressive tasks. In this paper, we explore applying data distribution representation learning to improve uncertainty estimation for a system dynamics regression task. We follow recent work on Energy-Based Models (EBM) Du et al. (2020) for their efficient training process, using Langevin dynamics sampling to train a neural network that contains a compressed representation of the dataset, named Compressed Data Representation Model ( CDRM ). The input for CDRM is a (state, action, next-state) tuple, and the output is a scalar indicating if this tuple is present in the dataset. Through this process, CDRM learns a compressed representation of the dataset. We then designed an efficient inference procedure using Langevin sampling to query the CDRM and obtain estimated next-state distributions for given (state, action) pairs. The learned data representation enables CDRM to sample from arbitrary next-state distributions, rather than being constrained to a Gaussian prior. Empirically, we observe that CDRM accurately estimates the next-state distribution even when it has multiple modes. While multimodal next-state distributions do not conform to the assumption of a single underlying dynamics function, a multimodal distribution more accurately captures the true distribution of inherently random real-world events, such as a dice throw, while keeping the state-space continuous. An overview of the training and inference procedures of CDRM is shown in Figure 1 . Our contributions are summarized as follows: • We propose Compressed Data Representation Model (CDRM), a framework to estimate epistemic and aleatoric uncertainty for a system dynamics regression task using a learned compressed data representation. • We design an efficient inference procedure using Langevin dynamics sampling to query CDRM for prediction and uncertainty estimates. • We provide theoretical analysis on the memory and computational complexities of CDRM and show the condition where CDRM surpasses bin-based compression. • We benchmark CDRM with competitive baseline methods on dedicated dataset for disentangling aleatoric and epistemic uncertainties. Quantitative and qualitative results demonstrated CDRM’s advantage at uncertainty disentanglement and multimodal distribution prediction. 2 Preliminaries 2.1 Problem Definition Given system dynamics \mathbf{s}_{t+1}=f(\mathbf{s}_{t},\mathbf{a}) where \mathbf{s}_{t} is the current state, \mathbf{a} is the action, and \mathbf{s}_{t+1} is the next state, we consider estimating aleatoric and epistemic uncertainties of a given input (\mathbf{s}_{t},\mathbf{a}) using a transition dataset \mathcal{D}:\{(\mathbf{s}_{t},\mathbf{a},\mathbf{s}_{t+1})\}_{N} . Here, we describe the definition of uncertainties: Aleatoric Uncertainty (AU) arises from the inherent randomness of a system that is irreducible by additional data. Existing works Chan et al. (2024) often model AU as a Gaussian distribution, with a real-world example being

Read the complete text