CPP-UT-Bench: Can LLMs Write Complex Unit Tests in C++?
Vaishnavi Bhargava, Rajat Ghosh, Debojyoti Dutta · 2024
Read CPP-UT-Bench: Can LLMs Write Complex Unit Tests in C++? by Vaishnavi Bhargava, Rajat Ghosh, Debojyoti Dutta (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02735.txt.
Description
Bhargava, Ghosh, and Dutta built a test set in twenty twenty-four to gauge if chat models can write hard unit tests for C++ code. Their set holds more than two thousand code and test pairs from fourteen open code bases. Those bases span nine fields, from learning tools to log and store code.
They try the set with in-text learning, light tune, and full tune of chat models. Tuned models beat their base forms in most tests by a large gap. The set's build snags and tune lessons are shared in full. Weights of the tuned models are shared for more work.
Catalog record
| CREATOR | Vaishnavi Bhargava, Rajat Ghosh, Debojyoti Dutta |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-02735