ClickSight: Interpreting Student Clickstreams to Reveal Insights on Learning Strategies via LLMs

Bahar Radmehr, Ekaterina Shved, Fatma Betül Güreş, Adish Singla, Tanja Käser · 2025

Read ClickSight: Interpreting Student Clickstreams to Reveal Insights on Learning Strategies via LLMs by Bahar Radmehr, Ekaterina Shved, Fatma Betül Güreş, Adish Singla, Tanja Käser (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15410.txt.

Introduction

Bahar Radmehr, Ekaterina Shved, Fatma Betül Güreş, Adish Singla, and Tanja Käser wrote this paper in two thousand twenty-five. It presents ClickSight, a pipeline that uses a large language model to read student clickstreams and name the learning strategies in play.

Click logs are rich but hard to read, and past tools lean on hand traits or expert marks. ClickSight takes the raw clicks and a list of candidate strategies, and writes a text account of what the student did. Four prompt plans are tried, and self-refinement is tested for lift. Expert grades in two open learning settings show fair reads, with quality shifting by prompt plan and little gain from refinement.

Catalog record

CREATORBahar Radmehr, Ekaterina Shved, Fatma Betül Güreş, Adish Singla, Tanja Käser
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-15410

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ClickSight: Interpreting Student Clickstreams to Reveal Insights on Learning Strategies via LLMs Bahar Radmehr, Ekaterina Shved, Fatma Betül Güreş, Adish Singla, Tanja Käser Abstract Clickstream data from digital learning environments offer valuable insights into students’ learning behaviors, but are challenging to interpret due to their high dimensionality and granularity. Prior approaches have relied mainly on handcrafted features, expert labeling, clustering, or supervised models, therefore often lacking generalizability and scalability. In this work, we introduce ClickSight , an in-context Large Language Model (LLM)-based pipeline that interprets student clickstreams to reveal their learning strategies. ClickSight takes raw clickstreams and a list of learning strategies as input and generates textual interpretations of students’ behaviors during interaction. We evaluate four different prompting strategies and investigate the impact of self-refinement on interpretation quality. Our evaluation spans two open-ended learning environments and uses a rubric-based domain-expert evaluation. Results show that while LLMs can reasonably interpret learning strategies from clickstreams, interpretation quality varies by prompting strategy, and self-refinement offers limited improvement. ClickSight demonstrates the potential of LLMs to generate theory-driven insights from educational interaction data. 1 Introduction The rise of digital learning environments has enabled more personalized, scalable, and data-driven educational experiences [ 12 ] . These environments facilitate gathering rich clickstream data—detailed, time-stamped records of student interactions—which can offer valuable insights into learning behaviors. Such insights can assist researchers in understanding students’ behaviors better, help instructors provide timely interventions to struggling students, and assist students in improving their learning strategies by providing feedback [ 3 ] . However, extracting meaning from clickstreams remains challenging due to their high dimensionality and granularity that makes expert annotation labor-intensive and cognitively demanding [ 7 ] . Prior work in AI for education has employed a range of techniques to detect learning strategies and analyze student behavior from clickstreams. These include designing domain-specific features for various learning strategies [ 21 , 8 ] , clustering students based on behavioral patterns [ 21 ] , sequential pattern mining to uncover frequent interaction traces [ 1 , 25 , 27 ] , supervised classifiers trained on annotated data [ 1 ] , and knowledge tracing to model evolving knowledge states [ 8 ] . While these methods yield valuable insights, they often require significant manual effort and domain expertise and are difficult to generalize across contexts. Large Language Models (LLMs) like GPT-4o [ 13 ] offer new opportunities to support educational technology [ 4 ] . LLMs have been applied to a range of educational tasks, including content generation [ 19 ] , feedback provision [ 14 ] , and student simulation [ 15 ] . Given their strong ability to recognize patterns [ 11 ] , LLMs hold promise for interpreting clickstream data through the lens of learning strategies. However, this potential remains largely unexplored. In this paper, we introduce ClickSight , an in-context LLM-based pipeline for interpreting student clickstreams via predefined learning strategies. Given a student’s clickstream and a list of learning strategies, ClickSight interprets the student’s behavior. We consider four prompting strategies and examine the impact of self-refinement, where the LLM critiques and revises its own outputs. We extensively evaluate ClickSight across two environments from different domains: PharmaSim [ 16 ] , a simulation for pharmacy assistant diagnostic training, and Beer’s Law Lab [ 21 ] , a virtual chemistry lab for secondary level students. To assess interpretation quality, we develop a rubric with four criteria grounded in human-centered explanation theory, and conduct domain expert evaluation based on it. We address two research questions: (1) which prompting strategy leads to the best strategy-based interpretation of students’ clickstreams (RQ1) , and (2) how does self-refinement affect interpretation quality (RQ2) ? Our results show that ClickSight enables high-quality interpretations, with one prompting strategy outperforming others. Self-refinement leads to modest improvements for some strategies, while introducing errors in others. Our implementation of ClickSight is available on our GitHub repository: https://github.com/epfl-ml4ed/ClickSightLBR25 . 2 Methodology ClickSight consists of three stages (see Fig. 1 ). In the first stage, we describe the simulation setting, identify relevant learning strategies, and organize student clickstreams into a structured format. The next stage prompts an LLM to generate textual interpretations grounded in the contextual information constructed in the first stage. Finally, we apply a rubric-based domain-expert evaluation to assess the quality, clarity, and relevance of the generated interpretations. Figure 1: The ClickSight pipeline interprets student clickstreams in three stages. (1) clickstream data from PharmaSim and Beer’s Law Lab are structured, and context and learning strategies are gathered; (2) an LLM is prompted using one of four prompting strategies ( Zero-shot , Chain-of-Thought , Meta-Prompting , Chain-of-Prompts ) with optional self-refinement to generate interpretations; (3) human experts assess the outputs using a quality rubric. 2.1 Gathering Environment Context We use data from two different educational environments PharmaSim [ 16 ] and Beer’s Law Lab [ 21 ] . PharmaSim is a scenario-based learning environment for pharmacy assistant training, where students develop diagnostic skills through simulated interactions with a virtual patient. In the selected scenario (see Fig. 1 ), a mother visits a pharmacy for a breastfeeding issue, which the student investigates by performing actions such as asking the patient questions, seeking help from the pharmacist, researching medicines, examining shelf products, and consulting medical documentation. Student Clickstreams for PharmaSim. Following the representations used in existing literature [ 17 ] , we structured each clickstream as a sequence of function calls corresponding to specific action types in the simulation. For example, the action of asking the virtual mother about her symptoms at time t is represented as: discuss(mother, symptoms, t ) [output: My breast hurts...] . Learning Strategies for PharmaSim. We gathered nine diagnostic strategies observed in student behavior in environments like PharmaSim, based on prior work. These vary in alignment with learning goals and span different aspects of the environment. Among diagnostic conversation strategies highlighted by prior work [ 5 , 6 ] , effective strategies include LINDAAFF , where students ask targeted questions about symptom details, and Inquiry about Relevant External Factors , which addresses influences like the baby’s health. Ineffective approaches include Premature Closure (settling on a diagnosis too early), Random Inquiry (asking about irrelevant topics), and Insufficient Inquiry (failing to collect enough information). Research strategy describes how students use the documentation, compendium, and shelf. Inspired by prior work on self-regulation [ 29 ] this strategy includes three types: Targeted Research, where students focus on relevant materials; Unfocused Research, characterized by scattered, aimless browsing; and Minimal Research, involving little or no use of these tools. Hint-seeking strategy can take the form of Thoughtful Hint Seeking, where students request hints at appropriate moments after meaningful engagement; Premature Hint Seeking, in which students rely on hints too early without sufficient exploration; and No Hint Seeking, where students do not request any hints during the interaction [ 20 ] . Finally, students may engage in Iterative Reflection , pausing between key actions to process and consolidate information [ 18 ] , or exhibit Gaming the System , where they rush through tasks without meaningful engagement [ 2 ] . Beer’s Law Lab allows students to investigate how cuvette width, solution concentration, and laser color-solution combination affect light absorbance (see Fig. 1 ).

Read the complete text