CPP-UT-Bench: Can LLMs Write Complex Unit Tests in C++?

Vaishnavi Bhargava, Rajat Ghosh, Debojyoti Dutta · 2024

Read CPP-UT-Bench: Can LLMs Write Complex Unit Tests in C++? by Vaishnavi Bhargava, Rajat Ghosh, Debojyoti Dutta (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02735.txt.

Description

Bhargava, Ghosh, and Dutta built a test set in twenty twenty-four to gauge if chat models can write hard unit tests for C++ code. Their set holds more than two thousand code and test pairs from fourteen open code bases. Those bases span nine fields, from learning tools to log and store code.

They try the set with in-text learning, light tune, and full tune of chat models. Tuned models beat their base forms in most tests by a large gap. The set's build snags and tune lessons are shared in full. Weights of the tuned models are shared for more work.

Catalog record

CREATORVaishnavi Bhargava, Rajat Ghosh, Debojyoti Dutta
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-02735

Browse the full library · About the Tower of Babel