Evaluating the Effectiveness of Linguistic Knowledge in Pretrained Language Models: A Case Study of Universal Dependencies
Wenxi Li · 2025
Read Evaluating the Effectiveness of Linguistic Knowledge in Pretrained Language Models: A Case Study of Universal Dependencies by Wenxi Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04887.txt.
Description
Wenxi Li tests whether adding grammar structure from Universal Dependencies helps pretrained language models tell true paraphrases from traps, in work from twenty twenty-five. The framework labels sentence grammar the same way across languages. It is widely used, yet its real payoff inside modern models was rarely measured.
Li folded the dependency structure into pretrained models and ran a cross lingual adversarial paraphrase task. Accuracy and F1 scores rose clearly on average. The gains shrank the gap to giant language models in some language pairs and passed them in others. A grammar based similarity score between each language and English tracked model success in that language. The author sees strong promise for the framework in fresh tasks.
Catalog record
| CREATOR | Wenxi Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-04887