Evaluating Large Language Models for Multilingual Vulnerability Detection at Dual Granularities
Honglin Shu, Michael Fu, Junji Yu, Dong Wang 🖂, Chakkrit Tantithamthavorn, Junjie Chen, Yasutaka Kamei · 2025
Read Evaluating Large Language Models for Multilingual Vulnerability Detection at Dual Granularities by Honglin Shu, Michael Fu, Junji Yu, Dong Wang 🖂, Chakkrit Tantithamthavorn, Junjie Chen, Yasutaka Kamei (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07503.txt.
Description
Honglin Shu and co-authors present a 2025 study on how well language models find security flaws in code across many programming languages. Most past work stuck to one or two languages and judged whole functions only. Real flaws, though, hide in single lines and in every language a team uses.
The team built a test bed from over thirty thousand real patches that fixed flaws, spanning seven languages. Models were scored at two levels: spotting a flawed function, and pinpointing the flawed line. A leading large model, sharpened with instruction tuning and a few examples, beat all rivals, including a code-specific model. The study maps where each model type stands for multilingual security screening.
Catalog record
| CREATOR | Honglin Shu, Michael Fu, Junji Yu, Dong Wang 🖂, Chakkrit Tantithamthavorn, Junjie Chen, Yasutaka Kamei |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 137 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-07503