MHRC-Bench: A Multilingual Hardware Repository-Level Code Completion Benchmark
Qingyun Zou, Jiahao Cui, Nuo Chen, Bingsheng He, Weng-Fai Wong · 2026
Read MHRC-Bench: A Multilingual Hardware Repository-Level Code Completion Benchmark by Qingyun Zou, Jiahao Cui, Nuo Chen, Bingsheng He, Weng-Fai Wong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03708.txt.
Description
Qingyun Zou and four co-authors present MHRC-Bench, the first benchmark for repository-level code completion in hardware description languages. Language models complete software code well, but chip design code was largely ignored by existing benchmarks, even though hardware projects also span many interdependent files. The benchmark includes separate training and evaluation parts and covers three major hardware coding styles.
Each completion target carries labels about code structure and hardware meaning, derived from syntax tree analysis. The team ran a broad evaluation of current models on the evaluation set and analyzed the results. A shared hardware benchmark matters because chip teams cannot judge coding assistants using software tests alone; the languages, idioms, and failure costs of hardware design are quite different.
Catalog record
| CREATOR | Qingyun Zou, Jiahao Cui, Nuo Chen, Bingsheng He, Weng-Fai Wong |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-03708