FinMTEB: Finance Massive Text Embedding Benchmark

Yixuan Tang, Yi Yang · 2025

Read FinMTEB: Finance Massive Text Embedding Benchmark by Yixuan Tang, Yi Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10990.txt.

Description

Yixuan Tang and Yi Yang benchmark text embedding models on finance in a 2025 paper. Embedding models turn text into vectors for search and retrieval, and general leaderboards may not predict how they handle money documents. Their benchmark gathers sixty-four data sets across seven tasks, in Chinese and English, from news, annual reports, ESG filings, regulations, and earnings calls.

Fifteen models were evaluated, including one the authors adapted for finance using synthetic training data built from personas. Three findings stand out. General benchmark scores barely predicted finance performance. Finance-adapted models consistently beat their general parents. And on one similarity task, a simple word-count method beat the dense embeddings outright, exposing a real weakness. The paper gives financial NLP a proper yardstick and a warning: domain fit must be proven, not assumed from general rankings.

Catalog record

CREATORYixuan Tang, Yi Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10990

Browse the full library · About the Tower of Babel