A Mixed User-Centered Approach to Enable Augmented Intelligence in Intelligent Tutoring Systems: The Case of MathAIde app

Guilherme Guerino ab , Luiz Rodrigues a , Luana Bianchini a , Mariana Alves a , Marcelo Marinho c , Thomaz Veloso a , Valmir Macario c , Diego Dermeval a , Thales Vieira a , Ig Bittencourt ad , Seiji Isotani ae · 2025

Read A Mixed User-Centered Approach to Enable Augmented Intelligence in Intelligent Tutoring Systems: The Case of MathAIde app by Guilherme Guerino ab , Luiz Rodrigues a , Luana Bianchini a , Mariana Alves a , Marcelo Marinho c , Thomaz Veloso a , Valmir Macario c , Diego Dermeval a , Thales Vieira a , Ig Bittencourt ad , Seiji Isotani ae (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00103.txt.

Introduction

Guilherme Guerino and co-authors describe how a math tutoring app was built with teachers in the loop in a 2025 paper. The app, MathAIde, photographs student work, checks the math with computer vision, and suggests feedback. Automated tutors often sideline teachers and strain trust, so the design process put users at the center from the start.

The team brainstormed with teachers, built high-detail prototypes, ran A/B tests, and piloted the app in a real classroom study. The AI proposes fixes for student errors while teachers keep the final call. Results point to good efficiency, ease of use, and adoption promise, above all in schools with few resources. The study offers a model for mixing human judgment with AI in classroom tools.

Catalog record

CREATORGuilherme Guerino ab , Luiz Rodrigues a , Luana Bianchini a , Mariana Alves a , Marcelo Marinho c , Thomaz Veloso a , Valmir Macario c , Diego Dermeval a , Thales Vieira a , Ig Bittencourt ad , Seiji Isotani ae
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-00103

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Mixed User-Centered Approach to Enable Augmented Intelligence in Intelligent Tutoring Systems: The Case of MathAIde app Guilherme Guerino ab , Luiz Rodrigues a , Luana Bianchini a , Mariana Alves a , Marcelo Marinho c , Thomaz Veloso a , Valmir Macario c , Diego Dermeval a , Thales Vieira a , Ig Bittencourt ad , Seiji Isotani ae Abstract Integrating Artificial Intelligence in Education (AIED) aims to enhance learning experiences through technologies like Intelligent Tutoring Systems (ITS), offering personalized learning, increased engagement, and improved retention rates. However, AIED faces three main challenges: the critical role of teachers in the design process, the limitations and reliability of AI tools, and the accessibility of technological resources. Augmented Intelligence (AuI) addresses these challenges by enhancing human capabilities rather than replacing them, allowing systems to suggest solutions. In contrast, humans provide final assessments, thus improving AI over time. In this sense, this study focuses on designing, developing, and evaluating MathAIde, an ITS that corrects mathematics exercises using computer vision and AI and provides feedback based on photos of student work. The methodology included brainstorming sessions with potential users, high-fidelity prototyping, A/B testing, and a case study involving real-world classroom environments for teachers and students. Our research identified several design possibilities for implementing AuI in ITSs, emphasizing a balance between user needs and technological feasibility. Prioritization and validation through prototyping and testing highlighted the importance of efficiency metrics, ultimately leading to a solution that offers pre-defined remediation alternatives for teachers. Real-world deployment demonstrated the usefulness of the proposed solution. Our research contributes to the literature by providing a usable, teacher-centered design approach that involves teachers in all design phases. As a practical implication, we highlight that the user-centered design approach increases the usefulness and adoption potential of AIED systems, especially in resource-limited environments. In theory, our study expands the existing literature by providing empirical evidence of the effectiveness of a user-centered, mixed-methods approach in designing AuI-empowered ITSs. keywords Mixed-method approach, UX research, User-centered design, Augmented Intelligence, Artificial Intelligence in Education, Intelligent Tutoring System † † articletype: ARTICLE TEMPLATE † † affiliation: a Center of Excellence in Social Technologies, Federal University of Alagoas, Maceió, Brazil; b State University of Paraná, Apucarana, Brazil; c Federal Rural University of Pernambuco, Recife, Brazil, d Harvard Graduate School of Education, Cambridge, USA, e Graduate School of Education, University of Pennsylvania, USA 1 Introduction Artificial Intelligence in Education (AIED) aims to improve learning experiences by integrating AIED systems into the educational domain. The benefits of AIED include personalized learning experiences, increased student engagement, tailored feedback, and improved retention rates ( Chen et al. 2022 ; Crompton et al. 2024 ) . Additionally, AIED systems can scale up educational resources to reach a wider audience, promoting a more inclusive and accessible learning environment ( Vinuesa et al. 2020 ; Memarian and Doleck 2023 ) when well designed. AIED systems encompass a range of applications, such as Intelligent Tutoring Systems (ITS) and Adaptive Systems. The literature provides consistent evidence of the contributions of these systems to education ( Major et al. 2021 ; Liu et al. 2020 ; Hillmayr et al. 2020 ; Ma et al. 2014 ) . However, three main concerns must be addressed in AIED systems. First, teachers are vital stakeholders of the AIED system. They decide which tools to use, when, and how to use them ( Xia et al. 2022 ) . Therefore, involving teachers in the design process of AIED systems is essential to ensure that these systems align with educational goals, promote compelling learning experiences, address ethical concerns, and are of high usability ( Topali et al. 2024 ; Dermeval et al. 2018b ; Dermeval et al. 2018a ) . Unfortunately, there has been little AIED research from this perspective ( Alfredo et al. 2024 ; Rodrigues et al. 2023 ; Modén et al. 2021 ) . This research gap may lead to solutions that are not usable or fail to achieve educational goals. Second, even AI tools such as GPT-4 have limitations despite their advanced capabilities. Large Language Models (LLMs) are known to suffer from hallucinations, which generate incorrect or nonsensical answers ( Ji et al. 2023 ) . Even well-established AIED research areas, such as Knowledge Tracing and Recommender Systems, have performance limitations ( Abdelrahman et al. 2023 ; Rahayu et al. 2023 ) . This limitation is crucial because teachers are unlikely to rely on AIED systems unless they trust them ( Xia et al. 2022 ; Chiu et al. 2023 ) . Third, not everyone has access to the resources required to use the AIED systems, which demand technological resources such as computers and the Internet ( Mousavinasab et al. 2021 ) as many regions lack access to such resources ( Gasevic et al. 2018 ; Reimers 2022 ) . This issue led to research presenting an alternative interface for students to benefit from AIED systems, where teachers act as a proxy between the students and the system ( Isotani et al. 2023 ) . Instead of interacting with an ITS directly, through a keyboard or a touchscreen, students continue solving tasks on paper and sheets. Then, the teachers photograph the paper sheets using a mobile ITS, which assesses the student’s solution and presents personalized feedback/recommendations for the teachers that, in turn, deliver those insights to the student ( Silva et al. 2023 ; Patel et al. 2022 ; Davis et al. 2020 ) . Notably, this alternative interface heavily depends on the ITS supporting handwritten instead of keyboard/touchscreen input. However, a recent literature review revealed a substantial lack of ITSs for math education that supports handwritten input ( Rodrigues et al. 2024b ) . Implementing such an AIED system requires state-of-the-art AI features, which pose substantial technical challenges ( Xia et al. 2022 ; Chiu et al. 2023 ) . In response to this challenge, Augmented Intelligence (AuI) emerges as a viable solution ( Broschert et al. 2019 ) . AuI, also known as Intelligence Amplification, refers to the use of AI systems to enhance human intelligence rather than aiming for these systems to operate autonomously ( Broschert et al. 2019 ) . Recognizing the technical challenges of implementing advanced AI techniques within mobile ITS, AuI offers an alternative in which ITS suggests a solution to a given task (e.g., assessing a handwritten equation) but relies on human intelligence to provide a definitive answer and learn over time ( Yau et al. 2021 ) . For instance, a teacher could indicate that the ITS assessment was inaccurate and give the proper answer to it. The ITS can then adjust its suggestions and use this feedback to improve future analyses. However, to our knowledge, previous research has not explored AuI to enhance the design of mobile ITS supporting handwritten input ( Alfredo et al. 2024 ; Topali et al. 2024 ) . Based on that context, the following research question (RQ) remains to be addressed: • How to design AuI so that it empowers mobile ITS supporting handwritten input? We address this RQ through a user-centered mixed methods approach involving designing, developing, and evaluating an AuI feature for MathAIde. Designed to support mathematics education, MathAIde is one of the few

Read the complete text