PCEval: A Benchmark for Evaluating Physical Computing Capabilities of Large Language Models

Inpyo Song, Eunji Jeon, Jangwon Lee · 2026

Read PCEval: A Benchmark for Evaluating Physical Computing Capabilities of Large Language Models by Inpyo Song, Eunji Jeon, Jangwon Lee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-02404.txt.

Description

Inpyo Song, Eunji Jeon, and Jangwon Lee introduce PCEval, a benchmark for what large language models can do in physical computing, where code must drive real hardware such as sensors and circuits. Models are widely used for pure software help, but a program that ignores pin limits or wiring rules fails in the hand. And that side had not been measured in a systematic way.

PCEval grades both the circuit a model designs and the code it writes for that circuit, across projects of rising complexity, with fully automatic checking and no human grading. Thirteen leading models were tested, giving the first reproducible picture of how well current models reason about basic hardware constraints. Code and data were released so later models can be scored the same way.

Catalog record

CREATORInpyo Song, Eunji Jeon, Jangwon Lee
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-02404

Browse the full library · About the Tower of Babel