Can Impressions of Music be Extracted from Thumbnail Images?

Takashi Harada, Takehiro Motomitsu, Katsuhiko Hayashi, Yusuke Sakai, Hidetaka Kamigaito · 2025

Read Can Impressions of Music be Extracted from Thumbnail Images? by Takashi Harada, Takehiro Motomitsu, Katsuhiko Hayashi, Yusuke Sakai, Hidetaka Kamigaito (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-02511.txt.

Description

Takashi Harada and four co-authors ask whether the feelings a song creates can be read from its thumbnail image. Music search and music-making systems increasingly take plain-language requests, but large sets of songs paired with written captions are rare. Facts about when a song fits, or what mood it stirs, are especially hard to get from audio alone.

The team proposes drawing such details from thumbnail images instead. Using their method, they built a dataset of about 360,000 captions that include mood and listening-situation details. Human judges rated the captions and found the approach worked. The team then trained a music retrieval model on the new dataset and showed it performed well on search tasks. The work offers a way to grow caption data for music systems without costly hand labeling.

Catalog record

CREATORTakashi Harada, Takehiro Motomitsu, Katsuhiko Hayashi, Yusuke Sakai, Hidetaka Kamigaito
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-02511

Browse the full library · About the Tower of Babel