Towards Detecting Prompt Knowledge Gaps for Improved LLM-guided Issue Resolution

Ramtin Ehsani, Sakshi Pathak, Preetha Chatterjee · 2025

Read Towards Detecting Prompt Knowledge Gaps for Improved LLM-guided Issue Resolution by Ramtin Ehsani, Sakshi Pathak, Preetha Chatterjee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-11709.txt.

Description

Ramtin Ehsani, Sakshi Pathak, and Preetha Chatterjee wrote this 2025 computing paper on why developers' questions to chatbots so often miss. Language models now help resolve software issues, but answers come back wrong or vague when the prompt itself lacks key facts. The team studied where those prompts go thin.

They analyzed 433 developer conversations with ChatGPT inside GitHub issue threads. Four prompt gaps emerged: missing context, missing specifications, several tangled contexts, and unclear instructions. Conversations on issues that stayed open carried gaps in about forty-five percent of prompts, against about thirteen percent on closed, resolved issues. Seven conversation styles appeared, with missing context haunting them all. The authors extract heuristics that could flag a thin prompt before it wastes a round trip.

Catalog record

CREATORRamtin Ehsani, Sakshi Pathak, Preetha Chatterjee
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-11709

Browse the full library · About the Tower of Babel