From Formulas to Figures: How Visual Elements Impact User Interactions in Educational Videos

Wolfgang Gritz, Hewi Salih, Anett Hoppe, Ralph Ewerth · 2025

Read From Formulas to Figures: How Visual Elements Impact User Interactions in Educational Videos by Wolfgang Gritz, Hewi Salih, Anett Hoppe, Ralph Ewerth (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01753.txt.

Introduction

Wolfgang Gritz, Hewi Salih, Anett Hoppe, and Ralph Ewerth study how visuals shape behavior in learning videos in 2025. Lab studies abound, but real viewing data show what students actually do. The team focused on visual complexity: how many clear, meaningful elements fill a frame, from formulas to diagrams.

They built a finer taxonomy of visual objects and tagged twenty-five physics and chemistry videos. Busier frames, above all dense text, came with more pauses, rewinds, and dropouts. The authors share their code and draw lessons for STEM video design: rich visuals help learning only when students can keep up.

Catalog record

CREATORWolfgang Gritz, Hewi Salih, Anett Hoppe, Ralph Ewerth
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-01753

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Formulas to Figures: How Visual Elements Impact User Interactions in Educational Videos Wolfgang Gritz, Hewi Salih, Anett Hoppe, Ralph Ewerth Abstract Educational videos have become increasingly relevant in today’s learning environments. While prior research in laboratory studies has provided valuable insights, analyzing real-world interaction data can enhance our understanding of authentic user behavior. Previous studies have investigated technical aspects, such as the influence of cuts on pausing behavior, but the impact of visual complexity remains understudied. In this paper, we address this gap and propose a novel approach centered on visual complexity, defined as the number of visually distinguishable and meaningful elements in a video frame, such as mathematical equations, chemical formulas, or graphical representations. Our study introduces a fine-grained taxonomy of visual objects in educational videos, expanding on previous classifications. Applying this taxonomy to 25 25 videos from physics and chemistry, we examine the relationship between visual complexity and user behavior, including pauses, in-video navigation, and session dropouts. The results indicate that increased visual complexity, especially of textual elements, correlates with more frequent pauses, rewinds, and dropouts. The results offer a deeper understanding of how video design affects user behavior in real-world scenarios. Our work has implications for optimizing educational videos, particularly in STEM fields. We make our code publicly available 1 1 1 https://github.com/TIBHannover/from_formulas_to_figures . 1 Introduction Educational videos have become a central component of modern education, allowing learners to control their learning pace and interact with the material. The rise of online education platforms and the digital transformation of traditional teaching formats has increased their importance [ 22 , 23 , 27 ] . Videos are particularly effective in enhancing comprehension and retention since they combine visual and auditory elements, as highlighted in the Cognitive Theory of Multimedia Learning (CTML) [ 17 ] . However, while the CTML provides a robust framework for understanding the benefits of multimedia learning, its key principles still require further testing and validation in real-world settings [ 18 ] . While there is extensive research on the effects of pausing, less attention has been paid to the motivation behind user interactions [ 3 , 10 , 15 , 28 , 30 ] . Furthermore, most of the research refers to controlled laboratory conditions and only a few to realistic interaction data [ 2 , 21 ] . In this regard, the role of visual complexity (i.e., the number of meaningful elements) remains understudied. Understanding the relationship between visual complexity and user behavior is essential, as excessive visual complexity could increase extraneous cognitive load, making it difficult for learners to process information. This paper aims to shed light on the relationship between visual complexity and its influence on user interactions (i.e., pausing, rewinding, forwarding, and dropping out). To explore this relationship, we first developed a taxonomy for visual objects in the context of science and technology education (STEM). Visually complex content, such as formulas, diagrams, and graphs, is ubiquitous in these subjects and might pose particular challenges. We annotated 25 videos from a channel with physics, chemistry, and math videos from an open-source video platform 2 2 2 https://av.tib.eu/ and gathered corresponding interaction data. Our statistical evaluation relies on a permutation-based approach. We adopted the Dynamic Time Warping (DTW) algorithm for the test statistic to capture the interplay of visual complexity and possibly delayed user interactions. We aim to answer the following research question: RQ: How does visual complexity in STEM learning videos influence user interactions such as pausing, seeking, and dropping out? The remainder of the paper is structured as follows: Section 2 comprises research related to our work. In Section 3 , we present the creation of our dataset, including the selection of videos and the development of the taxonomy for visual objects, as well as the annotation process. Section 4 details the methodology, explaining our data processing approach and the modified DTW algorithm. In Section 5 , we discuss the results of our analysis, highlighting key findings on the relationship between visual complexity and user interactions. Finally, Section 6 concludes the paper summarizing our findings. 2 Related Work In this section, we first summarize recent advances in video learning research regarding user interactions (Section 2.1 ). Subsequently, we highlight research on visual complexity in educational videos (Section 2.2 ). 2.1 User Interactions One key principle of CTML is the segmenting effect, which suggests that learning can be improved when multimedia content is broken down into smaller, manageable segments rather than presented as a continuous stream. This might prevent cognitive overload by allowing learners to process information in chunks, making integrating new knowledge into their existing mental models easier. In their meta-analysis, Rey et al. [ 26 ] concluded that learners could benefit from dividing videos into segments. Specifically, they found that learners with higher prior knowledge derive greater advantage from segmentation as they can use the additional processing time to integrate new information with their existing knowledge base more effectively. This leads to better retention and transfer performance. Spanjers et al. [ 30 ] found two different explanations for this phenomenon: (1) the slight pauses between the segments help to reduce cognitive load [ 16 , 31 ] , and (2) the segmentation itself cues the learning of the underlying structure [ 32 ] . By segmenting the content, learners might be less likely to become overwhelmed, and they can focus more effectively on understanding each part. Pauses can be distinguished into learner-determined pauses and system-determined pauses. Learner-determined pauses allow individuals to control the pace and choose when to stop the video according to their needs. However, recent studies found that learners barely use the pause function [ 3 , 10 ] . On the other hand, system-determined pauses are typically predefined according to the structure of the learning material [ 20 ] ; however, they can also be implemented at other points, such as around particularly complex sections [ 15 ] , to guide learners through challenging content. Liu et al. [ 15 ] placed system-determined pauses before and after sections with high element interactivity within educational videos. They found that these pauses increased learning outcomes and efficiency compared to a control group viewing continuous videos (without pauses). However, they observed no significant difference in learning outcomes between the group using system-determined pauses and the group using learner-determined pauses. While most previous studies [ 3 , 10 , 15 , 28 , 30 ] have been conducted in controlled lab settings, research using authentic usage data has been limited. Some studies analyzed authentic viewing patterns to identify user types [ 1 , 9 ] , but few explore what makes learners interact in real-world settings [ 2 , 21 ] . Merkt et al. [ 21 ] analyzed the influence of human-annotated meaningful structure, difficulties in comprehension, and structural breakpoints derived from an automatic shot boundary detection algorithm. They found a connection between pausing behavior and meaningful structure but not with the structural breakpoints. Anders et al. [ 2 ] also found a relationship between pausing behavior and meaningful structure, and additionally with mind-wandering. 2.2 Visual Complexity in Educational Videos Visual complexity, in general, can be described as the level of detail within an image [ 5 ,

Read the complete text