CultureMERT: Continual Pre-Training for Cross-Cultural Music Representation Learning
2025
Read CultureMERT: Continual Pre-Training for Cross-Cultural Music Representation Learning (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-17818.txt.
Description
The authors of CultureMERT present a music foundation model adapted for cross-cultural representation learning, published in 2025. Recent music models have improved audio representation learning. Yet they still work unevenly across diverse musical traditions. Their model starts from an existing foundation model and adapts it to several cultures at once.
A two-stage continual pre-training strategy uses learning-rate re-warming and re-decaying. That keeps adaptation stable on limited computing budgets. Training on a few hundred hours of Greek, Turkish, and Indian music improved tagging scores on non-Western tasks. The gains beat prior state of the art by several points, with minimal forgetting on Western tests. Merging separately adapted models in weight space, called task arithmetic, performed on par. The multi-cultural model was best overall across traditions.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-17818