Quantifying Multimodal Imbalance: A GMM-Guided Adaptive Loss for Audio-Visual Learning

Zhaocheng Liu, Zhiwen Yu, Xiaoqing Liu · 2025

Read Quantifying Multimodal Imbalance: A GMM-Guided Adaptive Loss for Audio-Visual Learning by Zhaocheng Liu, Zhiwen Yu, Xiaoqing Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21797.txt.

Description

Zhiwen Yu, Zhaocheng Liu, Xiaoqing Liu, Huanqiang Zeng, and C. L. Philip Chen address a quiet failure in systems that learn from several kinds of input at once. One input stream, such as sound or video, often learns faster and drowns out the weaker stream. Fixed rules for rebalancing ignore that some individual samples are far more skewed than others.

The team first defines a gap measure between what each single-stream branch predicts, and finds samples naturally split into balanced and skewed groups. A statistical mixture model separates the groups softly, and training then penalizes misalignment hardest on skewed samples while letting balanced ones focus on fusion. The method beat current leading baselines, and extra training on the clean balanced subset also worked as a data-cleaning step.

Catalog record

CREATORZhaocheng Liu, Zhiwen Yu, Xiaoqing Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-21797

Browse the full library · About the Tower of Babel