“How Do I …?”: Procedural Questions Predominate Student-LLM Chatbot Conversations
Alexandra Neagu and Marcus Messer and Peter Johnson and Rhodri Nelson · 2026
Read “How Do I …?”: Procedural Questions Predominate Student-LLM Chatbot Conversations by Alexandra Neagu and Marcus Messer and Peter Johnson and Rhodri Nelson (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-18372.txt.
Description
Alexandra Neagu and three co-authors study what students actually ask educational chatbots. Their 2026 paper examines impasse-driven questions, the help requests students form when stuck, across two contexts: self-study for learning and assessed coursework. The team analyzes over six thousand messages, using eleven language models and three human raters to classify questions under four existing schemas.
Procedural questions, the how-do-I kind, dominate in both contexts, even more so before summative assessment. Language models prove usable as raters, with moderate-to-good agreement, even more consistent than human raters. Yet schemas strain: they struggle with the semantic richness of composite prompts and capture only part of the wider risks and benefits. The authors urge future work to track multi-turn conversation itself, for instance with methods from discursive psychology.
Catalog record
| CREATOR | Alexandra Neagu and Marcus Messer and Peter Johnson and Rhodri Nelson |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-18372