CultureMERT: Continual Pre-Training for Cross-Cultural Music Representation Learning

2025

Read CultureMERT: Continual Pre-Training for Cross-Cultural Music Representation Learning (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-17818.txt.

Description

The authors of CultureMERT present a music foundation model adapted for cross-cultural representation learning, published in 2025. Recent music models have improved audio representation learning. Yet they still work unevenly across diverse musical traditions. Their model starts from an existing foundation model and adapts it to several cultures at once.

A two-stage continual pre-training strategy uses learning-rate re-warming and re-decaying. That keeps adaptation stable on limited computing budgets. Training on a few hundred hours of Greek, Turkish, and Indian music improved tagging scores on non-Western tasks. The gains beat prior state of the art by several points, with minimal forgetting on Western tests. Merging separately adapted models in weight space, called task arithmetic, performed on par. The multi-cultural model was best overall across traditions.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-17818

Browse the full library · About the Tower of Babel