STEB: Style Text Embedding Benchmark

Rafael Rivera Soto, Anna Wegmann, Cristina Aggazzotti · 2026

Read STEB: Style Text Embedding Benchmark by Rafael Rivera Soto, Anna Wegmann, Cristina Aggazzotti (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-31741.txt.

Description

Rivera Soto, Wegmann and Aggazzotti introduce STEB, a benchmark meant to standardize how style embeddings are judged. Text embeddings usually capture meaning, and big benchmarks reward that skill. Style, meaning how something is written rather than what it says, has been tested in scattered ways, with each study picking its own tasks.

STEB gathers 96 datasets across seven languages. Tasks include authorship verification and retrieval, AI-text detection and probing of linguistic features. The authors find that semantic embeddings consistently fail on stylistic tasks. No single style embedding wins across all tasks either. They release the benchmark code openly so later models can be compared on the same footing.

Catalog record

CREATORRafael Rivera Soto, Anna Wegmann, Cristina Aggazzotti
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE93 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.31741

Browse the full library · About the Tower of Babel