A Preliminary Study of Large Language Models for Multilingual Vulnerability Detection
Junji Yu, Honglin Shu, Michael Fu, Dong Wang, Chakkrit Tantithamthavorn, Yasutaka Kamei, Junjie Chen · 2025
Read A Preliminary Study of Large Language Models for Multilingual Vulnerability Detection by Junji Yu, Honglin Shu, Michael Fu, Dong Wang, Chakkrit Tantithamthavorn, Yasutaka Kamei, Junjie Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-07376.txt.
Description
Junji Yu and co-authors test how well language models find security flaws in code written in many programming languages, in a 2025 study. Older deep learning detectors were trained per language, which limits real-world use. Large language models promise one tool for all languages. The team benchmarked pre-trained models and top large models across seven popular languages.
A pre-trained code model, CodeT5P, won overall, above all at catching the most critical flaws. The big general models did not dominate as expected. The authors discuss where large models might still help in practice and call the study a first step. Their results give future cross-language detection work a measured baseline to beat.
Catalog record
| CREATOR | Junji Yu, Honglin Shu, Michael Fu, Dong Wang, Chakkrit Tantithamthavorn, Yasutaka Kamei, Junjie Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-07376