An Analysis of Posterior Collapse, Parameterization and Initialization in Variational Deep Gaussian Processes

Francisco Javier Sáez-Maldonado [email protected] of Computer Science and Artificial IntelligenceUniversidad de GranadaGranada, Spain, Juan Maroñas [email protected] of Quantitative MethodsCUNEF UniversidadMadrid, SpainDaniel Hernández-Lobato [email protected] of Computer Science andCentro de Investigación Avanzada en Física FundamentalUniversidad Autónoma de MadridMadrid, Spain · 2026

Read An Analysis of Posterior Collapse, Parameterization and Initialization in Variational Deep Gaussian Processes by Francisco Javier Sáez-Maldonado [email protected] of Computer Science and Artificial IntelligenceUniversidad de GranadaGranada, Spain, Juan Maroñas [email protected] of Quantitative MethodsCUNEF UniversidadMadrid, SpainDaniel Hernández-Lobato [email protected] of Computer Science andCentro de Investigación Avanzada en Física FundamentalUniversidad Autónoma de MadridMadrid, Spain (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25882.txt.

Introduction

Francisco Javier Sáez-Maldonado, Juan Maroñas, and Daniel Hernández-Lobato study a failure mode in deep Gaussian processes, layered probabilistic models prized for strong predictions. Fitting such models relies on variational inference, an approximation scheme that can collapse: the fitted posterior falls back to the prior, and the model explains the data as pure noise. Collapse is a known nuisance, but its causes inside these layered models were murky.

Links emerge between collapse, the Double Stochastic training algorithm, and the linear prior mean used in nearly every layer. Sáez-Maldonado and colleagues show the linear mean helps not by dodging a pathology of very deep models, as many believed. It helps by making the optimization better conditioned at the start. A zero-mean model initialized to mimic that start trains just as well, so the prior can be chosen for modelling reasons instead of training convenience.

Catalog record

CREATORFrancisco Javier Sáez-Maldonado [email protected] of Computer Science and Artificial IntelligenceUniversidad de GranadaGranada, Spain, Juan Maroñas [email protected] of Quantitative MethodsCUNEF UniversidadMadrid, SpainDaniel Hernández-Lobato [email protected] of Computer Science andCentro de Investigación Avanzada en Física FundamentalUniversidad Autónoma de MadridMadrid, Spain
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE258 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.25882

Browse the full library · About the Tower of Babel