DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules
Devin Yasith De Silva, Dhaval Patel, Christodoulos Constantinides, Shuxin Lin, Nianjun Zhou, Paul J Adams, Sal Rosato, Nicolas Constantinides, Deborah L. McGuinness, Jayant Kalagnanam, [1pt] IBM Rensselaer Polytechnic Institute Independent Contributor · 2026
Read DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules by Devin Yasith De Silva, Dhaval Patel, Christodoulos Constantinides, Shuxin Lin, Nianjun Zhou, Paul J Adams, Sal Rosato, Nicolas Constantinides, Deborah L. McGuinness, Jayant Kalagnanam, [1pt] IBM Rensselaer Polytechnic Institute Independent Contributor (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08614.txt.
Description
Devin Yasith De Silva and eleven coauthors benchmark language models on turning factory sensor rules into maintenance advice. Engineers write symbolic rules that fire on sensor conditions, but choosing the right fix takes years of asset specific practice. Their DiagnosticIQ test holds thousands of expert checked questions built from real rule and action pairs across many asset types.
Five variants probe failure modes from harder distractors to flipped conditions. Practitioners average well under half correct, confirming deep specialist demand. Top models cluster tightly, yet all crack under harder distractors, and many repeat the original answer even when conditions invert. Industrial AI learns the bottleneck is calibration under change, not raw detection skill.
Catalog record
| CREATOR | Devin Yasith De Silva, Dhaval Patel, Christodoulos Constantinides, Shuxin Lin, Nianjun Zhou, Paul J Adams, Sal Rosato, Nicolas Constantinides, Deborah L. McGuinness, Jayant Kalagnanam, [1pt] IBM Rensselaer Polytechnic Institute Independent Contributor |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 113 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08614