Learner Attentiveness and Engagement Analysis in Online Education Using Computer Vision
Sharva Gogawale,, Madhura Deshpande, Parteek Kumar, Irad Ben-Gal · 2024
Read Learner Attentiveness and Engagement Analysis in Online Education Using Computer Vision by Sharva Gogawale,, Madhura Deshpande, Parteek Kumar, Irad Ben-Gal (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-00429.txt.
Introduction
Sharva Gogawale, Madhura Deshpande, Parteek Kumar, and Irad Ben-Gal tackle a weak point of online classes in work published in 2024. A teacher on a video call cannot easily tell who is paying attention. The team builds a system that reads a learner's live video feed and scores how alert and engaged that learner seems.
Their method trains a neural network on a public data set of student videos called DAiSEE. A second step turns the model's labels into a single attentiveness index for the teacher. In tests, the system beat earlier methods at spotting attention. The authors present it as a practical tool that can run in real time during live lessons.
Catalog record
| CREATOR | Sharva Gogawale,, Madhura Deshpande, Parteek Kumar, Irad Ben-Gal |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-00429
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learner Attentiveness and Engagement Analysis in Online Education Using Computer Vision Sharva Gogawale,, Madhura Deshpande, Parteek Kumar, Irad Ben-Gal Abstract In recent times, online education and the usage of video-conferencing platforms have experienced massive growth. Due to the limited scope of a virtual classroom, it may become difficult for instructors to analyze learners’ attention and comprehension in real time while teaching. In the digital mode of education, it would be beneficial for instructors to have an automated feedback mechanism to be informed regarding learners’ attentiveness at any given time. This research presents a novel computer vision-based approach to analyze and quantify learners’ attentiveness, engagement, and other affective states within online learning scenarios. This work presents the development of a multiclass multioutput classification method using convolutional neural networks on a publicly available dataset - DAiSEE. A machine learning-based algorithm is developed on top of the classification model that outputs a comprehensive attentiveness index of the learners. Furthermore, an end-to-end pipeline is proposed through which learners’ live video feed is processed, providing detailed attentiveness analytics of the learners to the instructors. By comparing the experimental outcomes of the proposed method against those of previous methods, it is demonstrated that the proposed method exhibits better attentiveness detection than state-of-the-art methods. The proposed system is a comprehensive, practical, and real-time solution that is deployable and easy to use. The experimental results also demonstrate the system’s efficiency in gauging learners’ attentiveness. keywords Affective states, engagement detection, online education, real-time attentiveness detection, machine learning † † equal-contributors: These authors contributed equally to this work. † † equal-contributors: These authors contributed equally to this work. 1 Introduction The Covid-19 pandemic has significantly transformed the learning approaches of both instructors and learners, as their educational settings have shifted from traditional physical classrooms to digital ones. Millions of people globally use platforms such as Google Meet, Zoom, Teams, and WebEx to deliver lessons, conduct one-on-one sessions, and arrange seminars or conferences. E-learning has become substantially popular due to its feasibility, high demand, and the multitude of benefits that it offers. Although e-learning provides learners with flexible and self-paced learning schedules, it also results in certain inevitable challenges 1 2021 . In the traditional offline model of education, instructors can analyze learners’ engagement and responses to the material being taught by observing their body language, head movement, and various facial cues. Past studies have proposed that learning comprises a diverse range of cognitive and affective states. These studies have recognized facial expressions that are linked to learning-centered states, which typically include boredom, confusion, engagement, and frustration 2 2009 . In the online mode of education, there is an absence of a real-time feedback mechanism between the learners and instructors. This model lacks a real-time feedback mechanism between the learners and instructors, making it extremely difficult for instructors to analyze learners’ engagement levels and attentiveness. As a result, instructors are often unable to adjust their teaching based on learners’ visual responses.Consequently, the learning experiences and outcomes could be negatively impacted. Understanding the affective states of learners can facilitate more effective guidance and allow instructional design to be tailored to their needs. This can also help learners who require assistance and could reduce dropout rates in online learning 3 2022 . Enhancing learner performance and engagement is crucial for online learning. Therefore, there is a pressing need to develop more affordable and accessible methods for capturing, monitoring and analyzing learners’ attention. By doing so, the effectiveness of these environments will be maximized and they will be conducive for learners to achieve their educational goals 5 2023 . Engagement detection prevails in multiple domains and it is done using various surveys, eye trackers, head movements, gaze detection, and gauging the seven universal emotions viz. happiness, surprise, contempt, sadness, fear, disgust, and anger. In the education domain, it is crucial to understand how learners acquire, comprehend, and analyze information. The brain’s function is contingent upon the integration of both cognitive and affective processing. The affective state of a learner has proved to play a very significant role. Holistic improvement in creative thinking is observed in individuals experiencing positive emotional states, whereas those in negative emotional states prioritize setting goals for performance (well-being) over mastery (learning) 6 2020 . Facial image analysis is the most highly recognized external factor for detecting facial expressions and affective states. Facial features can be used to identify the key emotional states that are represented by learners’ expressions during learning 7 2021 . Although this area has exhibited some advancements, multiple challenges exist, particularly in accurately associating facial expressions with the underlying emotions of learners. Advances in neural networks and deep learning techniques have enabled the visual perception of machines to significantly progress, but only a few efforts have been made to detect learners’ attentiveness in online learning. In this research, a hybrid architecture using a convolutional neural network (CNN) is presented. This is a popular deep-learning approach that has been used to analyze learners’ aggregated affective state patterns, including understanding learning-centered emotions such as boredom, engagement, confusion, and frustration, and estimating their attentiveness states throughout the online session. Furthermore, a novel comprehensive framework is proposed for implementing a low-cost system for emotion recognition and attentiveness estimation, which can be integrated into video conferencing platforms or e-learning platforms. Since instructors require external perceptible factors to gauge perceived attentiveness, this automation can provide critical insight into their learners’ learning experiences. The remainder of this paper is organized as follows. A brief overview of engagement estimation is provided in Section 2 . The methods implemented in this research, as well as descriptions of the used dataset and models, are introduced and discussed in Section 3 . The development of the Attentiveness Index is outlined in Section 4 . The end-to-end pipeline is illustrated in Section 5 . The performance evaluation results of our proposed method compared with other recent works are discussed in Section 6 . Finally, the conclusions and potential future extensions of the proposed framework are presented in Section . 2 Related Work Over the last few years, extensive efforts have been dedicated to detecting and analyzing learner engagement 4 2018 , 53 2024 . This ongoing research on automated cognitive engagement detection in online classrooms is crucial for adapting to the unique challenges and opportunities presented by the digital mode of education 8 2024 . At a general level, automated methods for detecting learners’ engagement can generally be categorized into computer vision-based methods and multimodal methods. Multimodal techniques are typically employed for identification tasks and utilize expensive equipment for engagement detection. Yang et al. 9 2013 used an eye-tracking system (FaceLAB 4.5) to observe and evaluate the variations and patterns of visual attention in university learners during PowerPoint presentations. Similarly, Kao et al. 10 2019 used the ASL Mobile Eye-XG eye tracker to investigate learners’ engagement while reading. The eye-tracking data was wirelessly transmitted to a system for analysis. Leite et al. 43 2015 implemented both manual and automation techniques to gauge disengagement by analyzing different types of data, namely audio, visual, contextual, etc. Kapoor et al. 44 2005 extracted multimodal sensory information through postural shifts and facial expressions measured using a camera and pressure-sensing chair to classify children’s