Developing Models of Procedural Skills using an AI-assisted Text-to-Model Approach Conference: Proceedings of the Thirteenth ACM Conference on Learning @ Scale; June 29–July 3, 2026; Seoul, Republic of Korea. Proceedings of the Thirteenth ACM Conference on Learning @ Scale (L@S ’26), June 29–July 3, 2026, Seoul, Republic of Korea ISBN: 979-8-4007-2293-6/2026/06 DOI: 10.1145/3774398.3811603 CCS: Computing methodologies Artificial intelligence CCS: Computing methodologies Knowledge representation and reasoning CCS: Computing methodologies Ontology engineering

Rahul K. Dass, Shubham Puri, Arpit Khandelwal, Xiao Jin, Ashok K. Goel · 2026

Read Developing Models of Procedural Skills using an AI-assisted Text-to-Model Approach Conference: Proceedings of the Thirteenth ACM Conference on Learning @ Scale; June 29–July 3, 2026; Seoul, Republic of Korea. Proceedings of the Thirteenth ACM Conference on Learning @ Scale (L@S ’26), June 29–July 3, 2026, Seoul, Republic of Korea ISBN: 979-8-4007-2293-6/2026/06 DOI: 10.1145/3774398.3811603 CCS: Computing methodologies Artificial intelligence CCS: Computing methodologies Knowledge representation and reasoning CCS: Computing methodologies Ontology engineering by Rahul K. Dass, Shubham Puri, Arpit Khandelwal, Xiao Jin, Ashok K. Goel (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-17624.txt.

Introduction

Dass and four colleagues introduce an AI-assisted text-to-model method for building models of procedural skills for tutoring. Building such representations by hand is a slow bottleneck for AI tutoring. Their method converts instructional materials to schema-complete Task-Method-Knowledge models through ontology-constrained prompting and templates, with experts kept in the loop. Applied to a graduate online AI course, it produced 23 models covering the full course for Ivy, an AI coach. Expert modeling time fell 50 to 70 percent, with models valid and highly reproducible.

Catalog record

CREATORRahul K. Dass, Shubham Puri, Arpit Khandelwal, Xiao Jin, Ashok K. Goel
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47690
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.17624

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Developing Models of Procedural Skills using an AI-assisted Text-to-Model Approach Conference: Proceedings of the Thirteenth ACM Conference on Learning @ Scale; June 29–July 3, 2026; Seoul, Republic of Korea. Proceedings of the Thirteenth ACM Conference on Learning @ Scale (L@S ’26), June 29–July 3, 2026, Seoul, Republic of Korea ISBN: 979-8-4007-2293-6/2026/06 DOI: 10.1145/3774398.3811603 CCS: Computing methodologies Artificial intelligence CCS: Computing methodologies Knowledge representation and reasoning CCS: Computing methodologies Ontology engineering Rahul K. Dass, Shubham Puri, Arpit Khandelwal, Xiao Jin, Ashok K. Goel Abstract . Scalable AI tutoring for procedural skill learning requires structured knowledge representations, yet constructing these representations remains a labor-intensive bottleneck. This paper introduces a new LLM-assisted text-to-model (TTM) methodology that transforms instructional materials into schema-complete Task-Method-Knowledge (TMK) models through ontology-constrained prompting and template-based generation, automating structural scaffolding while preserving expert oversight. Applied to a graduate-level online AI course, the methodology produced 23 TMK models—enabling full-course coverage for Ivy, a deployed AI coach that relies on TMK models to support learners’ procedural understanding, for the first time. AI-assisted authoring reduced expert modeling time by 50-70% while producing structurally valid and highly reproducible models. We evaluate structural validity, semantic alignment, reproducibility, and refinement effort to characterize authoring scalability. Results indicate that the TTM methodology substantially lowers the cost of constructing structured procedural representations, making course-wide deployment of structured AI tutoring systems practically feasible. † † cc-license: by 1. Introduction Online learning environments increasingly integrate generative AI-based intelligent tutoring systems (ITS) to provide interactive support for diverse learner populations ( 26 ; 18 ) . In video-based courses, learners frequently seek clarification about procedural skills. For example, how to implement a method or why a step is valid ( 16 ; 15 ) . While large language models (LLMs) can generate fluent explanations ( 25 ; 8 ) , reliable support for procedural skill learning requires explicit representations of task structure, skill mechanisms (e.g., sequence of actions and logical data conditions), and domain-specific concepts and relations — properties that are difficult to capture through prompt engineering or retrieval alone. Recent hybrid approaches that combine structured knowledge models with generative synthesis offer a promising path toward improving explanation reliability in AI-supported learning systems ( 33 ; 27 ) . One such system is Ivy ( 7 ; 6 ) , a deployed AI coach that combines Task-Method-Knowledge (TMK) models ( 21 ; 30 ; 12 ) —structured representations that encode the goals, mechanisms, and domain concepts of a procedural skill—with a LLM to answer learners’ “how-to” and “why-based” questions. By grounding its explanations in TMK models, Ivy goes beyond retrieval over unstructured text, capturing the causal, teleological, and compositional structure that characterizes procedural skill understanding. However, Ivy’s ability to support a learner’s procedural query depends directly on the availability of a TMK model for the skill in question. At approximately 6–7 hours of expert effort per model ( 7 ) , constructing representations for a complete course remains a significant practical bottleneck. More broadly, the scalability of structured AI tutoring systems depends not only on runtime generation of explanations, but on the feasibility of constructing the underlying knowledge representations. Prior work in ITS has long identified domain modeling and authoring as a primary bottleneck for scalable deployment: building formal representations of skills requires substantial expert effort and limits coverage across courses and curricula ( 23 ; 29 ) . In large-scale online courses that span dozens or hundreds of skills, when each model demands hours of expert design and refinement, full-course coverage becomes impractical. Under these conditions, the scalability challenge lies less in generative capability and more in scalable knowledge authoring infrastructure. To address this bottleneck, we present a new text-to-model (TTM) methodology that uses LLMs to semi-automate the construction of TMK models from instructional materials through ontology-constrained prompting and template-based generation. The methodology automates structural completion tasks such as extracting domain concepts and scaffolding procedural mechanisms while preserving expert oversight for validating causal transitions and failure conditions. Our central research question is: RQ:: To what extent can a novel LLM-assisted TTM methodology reduce the expert effort required to construct TMK models of procedural skills, enabling scalable deployment of structured AI tutoring systems? In a case study using instructional materials from a graduate-level online AI course at Georgia Tech, we applied the TTM methodology to construct 23 TMK models—achieving full-course coverage for Ivy’s structured AI coaching for the first time. AI-assisted authoring reduced expert modeling time by 50–70% while producing structurally valid and reproducible models, indicating that course-wide structured AI tutoring deployment is practically feasible. This work makes three contributions: (1) a new LLM-assisted TTM methodology for semi-automated construction of TMK models from instructional materials, (2) an evaluation framework assessing structural validity, semantic alignment, reproducibility, and refinement effort of TTM-generated models, and (3) empirical evidence that the TTM methodology substantially reduces expert authoring time, enabling realistic course-wide deployment of structured AI tutoring systems such as Ivy. 2. Related Work Text-to-model methods bridge the gap between human-readable text and machine-interpretable representations. Large language models and neural-symbolic integration have recently accelerated progress in this area, enabling end-to-end systems that convert natural language directly into formal knowledge representations. 2.1. Authoring Bottlenecks in Intelligent Tutoring Systems The authoring bottleneck remains the central obstacle to scaling ITS. Murray demonstrated that developing model-tracing tutors required approximately 200–300 hours of expert effort per hour of instruction, with the majority of time spent constructing production rule models ( 22 ) . Aleven reported 1:50 to 1:100 development ratios for cognitive tutors even with improved authoring tools, noting that domain experts struggled particularly with formalizing procedural knowledge into executable rules ( 1 ) . Van Lehn distinguished between “inner loop” authoring (creating individual problem steps and hints) and “outer loop” authoring (curriculum sequencing), identifying inner loop knowledge engineering as the critical bottleneck since outer loop tasks could be automated ( 31 ) . These findings establish that the primary cost in ITS development lies in formalizing domain knowledge, not in runtime generation. This motivates approaches that automate structural drafting while preserving expert oversight for semantic validation. 2.2. LLM-Assisted Structured Knowledge Construction LLMs have demonstrated strong capability for transforming unstructured text into structured representations conforming to predefined schemas. Schema-constrained extraction imposes structural constraints that ensure outputs adhere to specific ontologies or database schemas. KnowCoder ( 19 ) represents schemas as Python classes in a format LLMs can interpret naturally, enabling systematic extraction of structured information while maintaining semantic richness. PARSE ( 28 ) improves structured extraction reliability by automatically refining JSON schemas and applying reflection-based guardrails, directly addressing the hallucination and schema-compliance failures common in LLM-based extraction pipelines. While these approaches demonstrate that LLMs can reliably produce schema-conformant structured outputs, they target general knowledge extraction tasks. Applying schema-constrained generation to the authoring of pedagogically structured procedural models, where correctness of causal transitions and goal decomposition matters as much as structural validity, remains an open problem. 2.3. Evaluation of structured model generation Evaluating LLM-generated structured outputs presents distinct challenges beyond traditional NLP metrics. Zhang demonstrated that even schema-valid knowledge graphs contain substantial ungrounded facts, with hallucination rates varying by relation type and domain complexity ( 32 ) . Automatic metrics frequently misalign with human judgment and struggle to detect structural inconsistencies (

Read the complete text