Enhancing the Learning Experience: Using Vision-Language Models to Generate Questions for Educational Videos

Markos Stamatakis, Joshua Berger, Christian Wartena, Ralph Ewerth, Anett Hoppe · 2025

Read Enhancing the Learning Experience: Using Vision-Language Models to Generate Questions for Educational Videos by Markos Stamatakis, Joshua Berger, Christian Wartena, Ralph Ewerth, Anett Hoppe (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01790.txt.

Introduction

Markos Stamatakis and co-authors test whether vision and language models can write good study questions for teaching videos, in a 2025 paper. Online videos teach flexibly, yet keeping viewers active is hard. Auto written questions could prompt thought and check grasp.

The team probed models as shipped, then after fine tuning on course content. They varied which video channels the model saw, from frames to speech, and judged the effect on question quality. Teachers rated sample questions for relevance, answerability, and difficulty. Raw models fell short on variety and fit, while tuning helped. The paper lists what future data sets need and where the work should head.

Catalog record

CREATORMarkos Stamatakis, Joshua Berger, Christian Wartena, Ralph Ewerth, Anett Hoppe
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-01790

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Enhancing the Learning Experience: Using Vision-Language Models to Generate Questions for Educational Videos Markos Stamatakis, Joshua Berger, Christian Wartena, Ralph Ewerth, Anett Hoppe Abstract Web-based educational videos offer flexible learning opportunities and are becoming increasingly popular. However, improving user engagement and knowledge retention remains a challenge. Automatically generated questions can activate learners and support their knowledge acquisition. Further, they can help teachers and learners assess their understanding. While large language and vision-language models have been employed in various tasks, their application to question generation for educational videos remains underexplored. In this paper, we investigate the capabilities of current vision-language models for generating learning-oriented questions for educational video content. We assess (1) out-of-the-box models’ performance; (2) fine-tuning effects on content-specific question generation; (3) the impact of different video modalities on question quality; and (4) in a qualitative study, question relevance, answerability, and difficulty levels of generated questions. Our findings delineate the capabilities of current vision-language models, highlighting the need for fine-tuning and addressing challenges in question diversity and relevance. We identify requirements for future multimodal datasets and outline promising research directions. 1 Introduction The use of video content has grown across web platforms, transforming how information is shared and consumed in social media and for entertainment. In education, videos offer unique advantages: they combine visual and auditory elements to enhance the learning experience [ 25 , 40 ] while providing students with flexibility in when and where they study [ 36 ] . However, this flexibility and passive engagement of video watching can reduce active engagement. Question generation addresses this challenge by prompting learners to process the content actively [ 16 ] . Automatically generating questions from the most relevant information in a video can enhance learner interaction and knowledge retention. Before large language models (LLMs), Natural Language Processing (NLP) relied on rule-based systems, machine learning, and early neural networks [ 46 ] . However, recent research has shown that LLMs achieve groundbreaking results, significantly improving tasks like text summarization and translation. Models like GPT-4 [ 39 ] (Generative Pre-trained Transformer) and vision-language models (VLMs), such as OpenFlamingo [ 2 ] , BLIP-2 [ 26 ] and LLaVA [ 31 , 32 ] can process images and text. While video content analysis with these models is still in early stages, focusing mainly on monomodal, text-oriented approaches using speech transcripts [ 29 , 33 , 34 ] , some newer models can process videos multimodally [ 27 , 38 , 54 ] . We explore the potential of multimodal VLMs for educational video question generation using answer-unaware question generation. We investigate how current models perform without training and whether fine-tuning with appropriate datasets can improve question quality. Beyond standard automatic metrics, we introduce a custom metric that compares questions with speech transcripts to assess content alignment. This paper examines these capabilities through the following research questions (RQs): RQ1: How well do VLMs generate learning-oriented questions from educational videos in zero-shot settings? RQ2: Does fine-tuning VLMs improve question quality? RQ3: Do existing datasets suffice for high-quality question generation? RQ4: What dataset characteristics are essential for training and evaluating question generation models? We first present related work regarding question generation and useful datasets (Section 2 ). Then, we outline the used data and chosen models (Section 3 ). Later, we discuss our experiments and their results, including zero-shot, fine-tuning, an ablation study, and a statistical and manual qualitative evaluation (Section 4 ). Finally, a conclusion is drawn (Section 5 ). 2 Related Work This section presents related work on automatic question generation for educational tasks (Section 2.1 ), and relevant datasets (Section 2.2 ). 2.1 Question Generation for Educational Material Recent research on educational question generation can be classified into two main approaches in our context: one focuses on monomodal data sources like text documents, and another focuses on multimodal data like videos. Much research has been done on text-based approaches regarding whether LLMs are capable of a more learning-focused question generation. Specifically, the research explored to what extent LLMs can generate questions of various complexity levels and whether they can accurately classify a given question into the appropriate category [ 1 , 19 , 35 , 45 ] . Their focus is primarily on Bloom’s taxonomy [ 11 ] , while Graesser’s taxonomy [ 7 ] also appears in the categorization of questions. Furthermore, studies have investigated if generated questions are meaningful in terms of learning goals [ 5 , 12 , 28 , 37 , 47 , 52 ] . This is facilitated by defined metrics such as topic relevance, answerability, and learning utility. It can be observed that, in addition to the standard generation of questions, multiple-choice questions are also a focus, requiring an additional evaluation of their answer options. Looking at multimodal data, it becomes evident that research focuses more on using only one modality to create questions for this kind of data. Especially for videos, the question generation mostly relies on speech transcripts [ 4 , 9 , 14 , 49 ] . This comes with the drawback that the visual content of videos is ignored, preventing generating questions on this information. 2.2 Relevant Datasets While generic question-answering datasets, such as SQuAD [ 42 ] and WikiQA [ 51 ] , are useful for training and evaluating question-generation models, they often lack the educational focus required for our study. In contrast, datasets like EduQG [ 18 ] are designed for educational purposes, providing multiple-choice questions and answers tailored to this domain. These datasets focus especially on text-based data, making them suitable only for monomodal approaches. When considering multimodal approaches, particularly those involving videos for our study, existing video datasets cover a wide range of domains, including TV series [ 23 , 24 ] , tutorial videos [ 10 ] , activities in web videos [ 53 ] , medicine [ 17 ] , and more general videos [ 50 ] . In this context, LearningQ [ 9 ] stands out as a dataset specifically designed for question generation in education, containing instructor-designed questions from TED-Ed and learner-generated questions from Khan Academy for corresponding videos and articles. TQA [ 21 ] is an educational dataset that collects online textbooks to facilitate multimodal question generation. It incorporates texts, diagrams, and images, with the current version also including videos and questions about the content. To the best of our knowledge, no suitable dataset currently includes annotations for the temporal position of questions, necessitating the use of entire videos for question generation rather than relevant segments. Additionally, a higher diversity of video presentation styles in a dataset would be beneficial for a more detailed analysis of model behavior. 3 Datasets, Models, Prompt Design and Metrics Our review of educational question generation highlights a research gap concerning multimodal approaches and the investigation of the capabilities of state-of-the-art VLMs for generating questions from educational videos. We evaluate their performance with and without fine-tuning (RQ1, RQ2), assess the adequacy of current datasets (RQ3), and identify key dataset characteristics for effective training and evaluation (RQ4). Furthermore, we explore different prompts and evaluation metrics to gain deeper insights into model behavior (RQ1, RQ2). 3.1 Datasets We conducted our experiments using the LearningQ dataset [ 9 ] , which

Read the complete text