Empirical evaluation of LLMs in predicting fixes of Configuration bugs in Smart Home System

Sheikh Moonwara Anjum Monisha (Author), Atul Bharadwaj (Author) · 2025

Read Empirical evaluation of LLMs in predicting fixes of Configuration bugs in Smart Home System by Sheikh Moonwara Anjum Monisha (Author), Atul Bharadwaj (Author) (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10953.txt.

Description

Sheikh Moonwara Anjum Monisha and Atul Bharadwaj test chat models on smart home faults, in a study from 2025. Smart homes run on config files, and a small slip can break lights, locks, or heat. The team asked if large language models can call the right fix for such bugs.

They took 129 fault threads from the Home Assistant user hub and dug deep into 21 of them. Three models were tried with four prompt forms. Two models hit 80 percent on fix plans when the prompt held both the fault note and the script. One model was the most steady; a third was faster and cheaper with a small score dip. Prompt form swayed scores a lot. The work hints that guided models could join home tech support.

Catalog record

CREATORSheikh Moonwara Anjum Monisha (Author), Atul Bharadwaj (Author)
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10953

Browse the full library · About the Tower of Babel