FinMTEB: Finance Massive Text Embedding Benchmark
Yixuan Tang, Yi Yang · 2025
Read FinMTEB: Finance Massive Text Embedding Benchmark by Yixuan Tang, Yi Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10990.txt.
Description
Yixuan Tang and Yi Yang benchmark text embedding models on finance in a 2025 paper. Embedding models turn text into vectors for search and retrieval, and general leaderboards may not predict how they handle money documents. Their benchmark gathers sixty-four data sets across seven tasks, in Chinese and English, from news, annual reports, ESG filings, regulations, and earnings calls.
Fifteen models were evaluated, including one the authors adapted for finance using synthetic training data built from personas. Three findings stand out. General benchmark scores barely predicted finance performance. Finance-adapted models consistently beat their general parents. And on one similarity task, a simple word-count method beat the dense embeddings outright, exposing a real weakness. The paper gives financial NLP a proper yardstick and a warning: domain fit must be proven, not assumed from general rankings.
Catalog record
| CREATOR | Yixuan Tang, Yi Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-10990