Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams
Abhishek Kumar, Isha Motiyani, Tilak Kasturi, Ethan Seefried, Prahitha Movva, Tirthankar Ghosal · 2026
Read Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams by Abhishek Kumar, Isha Motiyani, Tilak Kasturi, Ethan Seefried, Prahitha Movva, Tirthankar Ghosal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03410.txt.
Description
Kumar and co-authors release Enginuity, a dataset and benchmark for engineering diagrams. Vision-language models struggle with such diagrams for clear reasons. The drawings pack dense layouts, special symbols, and links to parts tables. No public benchmark had measured model skill in this domain. Past datasets covered flowcharts, science figures, or business files instead.
Enginuity draws its corpus from United States military service and repair manuals. Two tasks are defined on that corpus. One asks models to extract structured parts tables. The other asks free-form questions about the diagrams. Four frontier models were tested with plain and chain-of-thought prompts. On extraction, recall reached 0.61 to 0.87, but description scores stayed very low. All models also showed a clear factual reasoning gap on questions. Token-overlap scores understate skill by two to six times versus semantic scores. The full data and outputs are released for repeat studies.
Catalog record
| CREATOR | Abhishek Kumar, Isha Motiyani, Tilak Kasturi, Ethan Seefried, Prahitha Movva, Tirthankar Ghosal |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.03410