Probing BERT for German Compound Semantics
Filip Miletić Aaron Schmid Sabine Schulte im Walde · 2025
Read Probing BERT for German Compound Semantics by Filip Miletić Aaron Schmid Sabine Schulte im Walde (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14130.txt.
Description
Filip Miletić, Aaron Schmid, and Sabine Schulte im Walde (2025) probe whether BERT, a widely used language model, understands German noun compounds. German glues words into long compounds far more freely than English, and knowing a compound's meaning often requires judging how transparent it is: whether its sense follows from its parts. The team tests a German BERT model by asking whether the model's internal representations can predict rated transparency scores for 868 compounds with human gold-standard judgments.
They vary which tokens they read out, which layers of the network they tap, and whether the model treats capital letters as meaningful. As in earlier English studies, the signal is easiest to recover from the early layers. But the best German results clearly trail the English ones. The authors argue the task is harder in German because compounding is so productive there, leaving compounds and their parts more ambiguous.
Catalog record
| CREATOR | Filip Miletić Aaron Schmid Sabine Schulte im Walde |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-14130