Soft Clustering Anchors for Self-Supervised Speech Representation Learning in Joint Embedding Prediction Architectures
Georgios Ioannides 1,3,7 Adrian Kieback 3,∗ Judah Goldfeder 4,∗ Linsey Pang 5 Aman Chadha 3,6 Aaron Elkins 3 Yann LeCun 2 Ravid Shwartz-Ziv 2 1 Carnegie Mellon University 2 New York University 3 James Silberrad Brown Center for AI 4 Columbia University 5 Northeastern University 6 Stanford University · 2026
Read Soft Clustering Anchors for Self-Supervised Speech Representation Learning in Joint Embedding Prediction Architectures by Georgios Ioannides 1,3,7 Adrian Kieback 3,∗ Judah Goldfeder 4,∗ Linsey Pang 5 Aman Chadha 3,6 Aaron Elkins 3 Yann LeCun 2 Ravid Shwartz-Ziv 2 1 Carnegie Mellon University 2 New York University 3 James Silberrad Brown Center for AI 4 Columbia University 5 Northeastern University 6 Stanford University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-09040.txt.
Description
Self-supervised speech models in the JEPA family learn by prediction, but without grounding their representations can collapse. Georgios Ioannides and seven co-authors propose GMM-Anchored JEPA: fit a Gaussian mixture model once on log-mel spectrograms, then use its frozen soft guesses as side goals all through training. A decaying schedule lets the anchor dominate early, then hands over to the JEPA objective.
Unlike HuBERT and WavLM, no iterative re-clustering is needed, and soft assignments beat hard ones. Trained on roughly 50,000 hours of speech, the model cuts word error rate to 28.68 percent from 33.22, while lifting emotion recognition and slot filling. Cluster usage also evens out dramatically, reaching up to 98 percent entropy against 31 percent for the baseline.
Catalog record
| CREATOR | Georgios Ioannides 1,3,7 Adrian Kieback 3,∗ Judah Goldfeder 4,∗ Linsey Pang 5 Aman Chadha 3,6 Aaron Elkins 3 Yann LeCun 2 Ravid Shwartz-Ziv 2 1 Carnegie Mellon University 2 New York University 3 James Silberrad Brown Center for AI 4 Columbia University 5 Northeastern University 6 Stanford University |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-09040