Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams

Abhishek Kumar, Isha Motiyani, Tilak Kasturi, Ethan Seefried, Prahitha Movva, Tirthankar Ghosal · 2026

Read Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams by Abhishek Kumar, Isha Motiyani, Tilak Kasturi, Ethan Seefried, Prahitha Movva, Tirthankar Ghosal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03410.txt.

Description

Kumar and co-authors release Enginuity, a dataset and benchmark for engineering diagrams. Vision-language models struggle with such diagrams for clear reasons. The drawings pack dense layouts, special symbols, and links to parts tables. No public benchmark had measured model skill in this domain. Past datasets covered flowcharts, science figures, or business files instead.

Enginuity draws its corpus from United States military service and repair manuals. Two tasks are defined on that corpus. One asks models to extract structured parts tables. The other asks free-form questions about the diagrams. Four frontier models were tested with plain and chain-of-thought prompts. On extraction, recall reached 0.61 to 0.87, but description scores stayed very low. All models also showed a clear factual reasoning gap on questions. Token-overlap scores understate skill by two to six times versus semantic scores. The full data and outputs are released for repeat studies.

Catalog record

CREATORAbhishek Kumar, Isha Motiyani, Tilak Kasturi, Ethan Seefried, Prahitha Movva, Tirthankar Ghosal
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.03410

Browse the full library · About the Tower of Babel