Uncertainty-driven Embedding Convolution
Sungjun Lim, Kangjun Noh, Youngjun Choi, Heeyoung Lee, Kyungwoo Song · 2025
Read Uncertainty-driven Embedding Convolution by Sungjun Lim, Kangjun Noh, Youngjun Choi, Heeyoung Lee, Kyungwoo Song (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-20718.txt.
Introduction
Sungjun Lim and four co-authors present a way to combine several text embedding models while accounting for each one's uncertainty, in a 2025 paper. Embeddings, the number lists that represent text for search and other tasks, vary in quality by model, and no single model wins everywhere. Standard ensembles ignore how unsure each model is about its output.
Their method, Uncertainty-driven Embedding Convolution, first turns fixed embeddings into probabilistic ones, then weights each model by its uncertainty when blending. A new similarity score also folds uncertainty straight into comparisons. Across many benchmarks, the approach improved both accuracy and robustness, which the authors attribute to principled handling of what each model does not know.
Catalog record
| CREATOR | Sungjun Lim, Kangjun Noh, Youngjun Choi, Heeyoung Lee, Kyungwoo Song |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-20718
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Uncertainty-driven Embedding Convolution Sungjun Lim, Kangjun Noh, Youngjun Choi, Heeyoung Lee, Kyungwoo Song Abstract Text embeddings are essential components in modern NLP pipelines. While numerous embedding models have been proposed, their performance varies across domains, and no single model consistently excels across all tasks. This variability motivates the use of ensemble techniques to combine complementary strengths. However, most existing ensemble methods operate on deterministic embeddings and fail to account for model-specific uncertainty, limiting their robustness and reliability in downstream applications. To address these limitations, we propose Uncertainty-driven Embedding Convolution (UEC) . UEC first transforms deterministic embeddings into probabilistic ones in a post-hoc manner. It then computes adaptive ensemble weights based on embedding uncertainty, grounded in a Bayes-optimal solution under a surrogate loss. Additionally, UEC introduces an uncertainty-aware similarity function that directly incorporates uncertainty into similarity scoring. Extensive experiments on retrieval, classification, and semantic similarity benchmarks demonstrate that UEC consistently improves both performance and robustness by leveraging principled uncertainty modeling. 1 1 1 Code is available at: https://github.com/MLAI-Yonsei/UEC 1 Introduction Embeddings are core building blocks in modern NLP, capturing semantic meaning for words, sentences, and documents. They support tasks like similarity [ 1 ] , retrieval [ 2 ] , QA [ 3 ] , and classification [ 4 ] . Numerous embedding models [ 3 , 5 , 6 ] have emerged with diverse architectures and training objectives, but their performance varies across tasks and domains. No single model excels universally; instead, models offer complementary strengths. This motivates combining multiple embeddings to leverage their diverse capabilities. Figure 1 : Comparison of deterministic embedding ensemble and UEC. Deterministic averaging (left) uniformly average embeddings without considering their reliability, often leading to suboptimal decisions. In this example, both candidate embeddings contribute equally, resulting in an incorrect retrieval. In contrast, the proposed UEC (right) adjusts weights based on the uncertainty, assigning higher importance to the more reliable candidate and successfully retrieving the correct answer. One intuitive strategy is to ensemble multiple embedding models directly in the representation space. Model merging techniques [ 7 , 8 , 9 ] assume shared architectures and initialization. However, embeddings are typically used in a decoupled manner: fixed representations are extracted in advance and stored for downstream tasks such as retrieval or classification [ 10 , 11 ] . Following this, combining embeddings at the representation level is more flexible and widely applicable. However, most existing ensemble approaches rely on deterministic aggregation methods such as uniform averaging, which fail to consider the reliability or uncertainty of individual embeddings [ 12 , 13 ] . Without modeling uncertainty, these methods treat all embeddings as equally reliable, which can lead to suboptimal or unstable performance, especially when some models are poorly calibrated or mismatched to the target task (Figure 1 ). To address these limitations, we propose Uncertainty-driven Embedding Convolution (UEC) , a framework for combining embeddings in a principled, uncertainty-aware manner. UEC consists of three key components. First, it converts pre-trained deterministic embeddings into probabilistic embeddings in a post-hoc fashion, allowing each embedding to represent both its mean and uncertainty. Second, it computes adaptive ensemble weights based on estimated uncertainty, down-weighting less reliable embeddings. This weighting strategy is grounded in a Bayes-optimal solution under a surrogate loss. Third, UEC introduces an uncertainty-aware similarity function that incorporates both distance and variance into the similarity score, enhancing robustness in similarity-driven tasks. In summary, our key contributions are as follows: • We propose a post-hoc mechanism to transform pre-trained deterministic embeddings into probabilistic representations, thereby enabling inherent uncertainty quantification for diverse existing embedding models. • We introduce the UEC framework, which adaptively combines multiple embeddings by weighting their contributions based on estimated uncertainties, a method theoretically grounded as a Bayes-optimal solution under surrogate loss minimization. • We develop an uncertainty-aware similarity function that explicitly incorporates embedding variance into the similarity scoring process, enhancing robustness and reliability in various similarity-driven downstream tasks. 2 Related Works 2.1 Embedding Ensemble Embedding ensemble methods aim to improve performance and robustness by combining multiple pretrained models. Shuang et al. [ 14 ] propose CDWE, which handles polysemy by generating multiple word prototypes through deconvolution. Fang et al. [ 13 ] enhance knowledge-aware embeddings by re-weighting knowledge-graph edges. Liu et al. [ 15 ] fuse BERT variants using cross-attention guided by pseudo-labels. Carlsson et al. [ 16 ] distill multiple encoders into one by matching their averaged outputs. Khan et al. [ 17 ] explore fusion strategies for domain-specific BERTs, yielding minor improvements. Despite these advances, most approaches rely on deterministic embeddings, limiting their ability to model uncertainty. In contrast, our method ensembles probabilistic embeddings, capturing both distributional information and model diversity through principled uncertainty estimation. 2.2 Probabilistic Embedding Probabilistic embeddings represent entities as distributions to capture uncertainty. In NLP, early work such as Gaussian [ 18 ] , Gaussian mixture [ 19 ] , and Bayesian word embeddings [ 20 ] modeled word-level uncertainty. On the other hand, Sen2Pro [ 21 ] extended this to sentences using pretrained models, but requires fine-tuning, lacks post-hoc conversion, and does not support uncertainty-aware fusion or scalable similarity for retrieval. In vision and multimodal tasks, probabilistic embeddings have shown success in face recognition [ 22 ] , medical imaging [ 23 ] , and vision-language models [ 24 , 25 , 26 ] . Our framework enables post-hoc uncertainty-aware convolution and similarity estimation, improving performance in tasks like retrieval and clustering. 3 Uncertainty-driven Embedding Convolution We propose Uncertainty-driven Embedding Convolution (UEC) , a principled framework for combining multiple embedding models by modeling predictive uncertainty. UEC consists of three stages (Figure 2 ): (1) post-hoc conversion of deterministic embeddings into probabilistic ones via Laplace approximation (LA) (Section 3.1 ), (2) Gaussian convolution with uncertainty-aware weights (Section 3.2 ), and (3) uncertainty-aware similarity scoring (Section 3.3 ). This enables robust and adaptive ensemble without manual tuning. Figure 2 : Overview of the UEC framework: UEC first transforms deterministic embeddings from multiple encoder models into probabilistic representations using Laplace approximation. These probabilistic embeddings are then adaptively combined by computing uncertainty-driven ensemble coefficients based on per-dimension variances. Finally, similarity is measured using a uncertainty-aware metric that accounts for both the mean and uncertainty of the ensembled embedding. 3.1 Post-hoc Probabilistic Embedding Model We first transform deterministic embeddings into probabilistic embeddings using LA [ 27 , 28 ] . This allows each embedding to express both mean and uncertainty, which we later exploit for adaptive model combination and similarity estimation. Laplace Approximation LA approximates the posterior distribution of model parameters w as a Gaussian centered at the maximum a posteriori (MAP) estimate. We denote the training dataset as \mathcal{D}=\{(x,y)\} , where x and y are the input and corresponding output, respectively. The posterior is given by Bayes’ theorem: p(w|\mathcal{D})=\frac{p(\mathcal{D}|w)p(w)}{p(\mathcal{D})}, where p(\mathcal{D}|w) is the likelihood, p(w) is the prior, and p(\mathcal{D}) is the evidence. Since this posterior is generally intractable, LA approximates p(w|\mathcal{D}) with a Gaussian centered at \hat{w} (the MAP) and precision matrix given by the Hessian H_{\hat{w}} : \displaystyle-\log p(w|\mathcal{D}) \displaystyle\approx-\log p(\hat{w}|\mathcal{D})+\frac{1}{2}(w-\hat{w})^{\top}H_{\hat{w}}(w-\hat{w}), \displaystyle H_{\hat{w}} \displaystyle=\nabla^{2}[-\log p(\mathcal{D}|w)-\log p(w)]\Big|_{w=\hat{w}}. This approximates posterior p(w|\mathcal{D})\approx\mathcal{N}(\hat{w},H^{-1}_{\hat{w}}) . Gaussian Embedding Generation Since computing the full Hessian is often impractical for large models, recent work shows that applying diagonal LA to only the