Breaking the Plane: Exploring Real-Time Visualization of 3D Surfaces in Augmented Reality with Handwritten Input Conference: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems; May 11–16, 2024; Honolulu, HI, USA Extended Abstracts of the CHI Conference on Human Factors in
Liam Esparraguera, Brian Lou, Kris Selberg, Jenny Sun, Beza Desta, Andrés Monroy-Hernández, Parastoo Abtahi · 2025
Read Breaking the Plane: Exploring Real-Time Visualization of 3D Surfaces in Augmented Reality with Handwritten Input Conference: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems; May 11–16, 2024; Honolulu, HI, USA Extended Abstracts of the CHI Conference on Human Factors in by Liam Esparraguera, Brian Lou, Kris Selberg, Jenny Sun, Beza Desta, Andrés Monroy-Hernández, Parastoo Abtahi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05616.txt.
Introduction
Liam Esparraguera and six co-authors present Breaking the Plane, an augmented reality app for headsets, in work from 2025. Students write a math equation by hand, and the app parses it and plots the matching three-dimensional surface in real time, which they can then move and explore. Earlier systems did parsing or 3D plots, but not both together with live interaction.
In user tests the system beat other tools on engagement and matched a popular visualizer on ease of use. Users rated it the most effective aid for problem solving and said they would pick it again. The work targets teaching, where fast authoring of visual material matters.
Catalog record
| CREATOR | Liam Esparraguera, Brian Lou, Kris Selberg, Jenny Sun, Beza Desta, Andrés Monroy-Hernández, Parastoo Abtahi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-05616
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Breaking the Plane: Exploring Real-Time Visualization of 3D Surfaces in Augmented Reality with Handwritten Input Conference: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems; May 11–16, 2024; Honolulu, HI, USA Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’24), May 11–16, 2024, Honolulu, HI, USA DOI: 10.1145/3613905.3651032 ISBN: 979-8-4007-0331-7/24/05 CCS: Human-centered computing Interactive systems and tools Liam Esparraguera, Brian Lou, Kris Selberg, Jenny Sun, Beza Desta, Andrés Monroy-Hernández, Parastoo Abtahi Abstract . We introduce Breaking the Plane, an augmented reality (AR) application built for AR headsets that enables users to visualize 3D mathematical functions using handwritten input. Researchers have demonstrated that overlaying 3D visualizations of mathematical concepts through AR increases motivation to learn mathematics and enhances mathematical learning, while also revealing that equation parsing via optical character recognition (OCR) makes the authoring of teaching materials more accessible and time-efficient for instructors. Previous works have developed AR systems that separately employ equation parsing and 3D mathematical visualizations, but work has yet to be done to combine those features by enabling real-time interactions and dynamic visualizations that help users learn in situ. We address this issue by developing an interactive AR system featuring handwritten equation parsing, object manipulation, and a custom 3D function plotter. We evaluated our system using a within-subjects study wherein 10 participants compared our system to two other commonly used 3D visualization systems: GeoGebra Desktop and GeoGebra AR. We found Breaking the Plane significantly surpassed other tools in engagement, achieved comparable ease of use to GeoGebra Desktop, and was rated as the most effective in aiding problem-solving, with a strong preference among participants for future use. Figure 1. Images during use of the Breaking the Plane system. (A) The system scans the existing equation on the whiteboard z=\sin(x)+\cos(y) and renders a 3D visualization of the graph in augmented reality (B) The user modifies the equation on the whiteboard to z=3\sin(x)+\cos(y) (C) The headset scans the new equation in real-time and renders a 3D visualization of the graph that the user can manipulate by grabbing, rotating, scaling, moving axes, etc. \label fig:teaser 1. Introduction For many students, the development of an intuitive understanding of multidimensional mathematical objects, such as multivariate functions and quadric surfaces, has proven a challenge ( Gutierrez et al. 2004 ) . This is in part due to the inherent difficulty in mentally visualizing abstract mathematical equations (e.g., z=x^{2}+y^{2} ) as geometric objects and representing these objects on a 2D surface (e.g., paper, blackboard) ( Ismail et al. 2020 ) . Researchers have proposed AR technology as a solution to help students better understand 3D mathematical objects, and AR systems have demonstrated potential to increase motivation and improve learning outcomes in mathematics education ( Bulut and Borromeo Ferri 2023 ) . Recent work has developed mobile AR interfaces that utilize an equation parsing component to enable existing teaching materials to be adapted for AR use through real-time visualization of 2D functions, creating a more interactive learning experience ( Chulpongsatorn et al. 2023 ) . We extend this work to a head-mounted display (HMD) system featuring real-time handwritten equation parsing. This approach leverages the unique strengths of AR in aiding students’ comprehension and engagement in 3D mathematics and problem-solving, while also offering the dynamic responsiveness and hands-free control of visualizations made possible by this novel input method. Thus, the user can benefit from computer-aided math visualizations without needing to context-switch between apps or devices while problem-solving. In this paper, we design and discuss Breaking the Plane, an AR headset application that uses real-time equation recognition, enabled by a Wizard-of-Oz implementation, and a 3D function plotter to improve ease of use and engagement when assisting students in the visualization of 3D mathematical objects. We evaluate our system on effectiveness of engagement, ease of use, and enhancement of problem-solving abilities relative to several other visualization techniques commonly used in math pedagogy. We found that participants using our system felt much more engaged with conceptual mathematical problems, experienced better ease of use than with an existing mobile AR system (GeoGebra AR), and were likely to use the system again in the future. 2. Related Work 2.1. Augmented Reality in Mathematics Education Researchers have demonstrated that AR systems in mathematics education which assist students by visualizing functions, vectors, and equations in 3D space enhance mathematical learning, contribute to memory retention, and increase motivation to learn mathematics, among other benefits ( Chang et al. 2022 ; Gargrish et al. 2021 ; Liono et al. 2021 ; Salinas et al. 2013 ) . Consequently, AR technology has been proposed as a solution to help students better understand 3D mathematical objects but has yet to be widely adopted in classrooms, despite having been shown to improve learning outcomes in mathematical education ( da Silva et al. 2019 ) . A survey of 106 teachers found that the second most frequently stated blocker for adopting AR in the classroom was "I prefer that my students create AR rather than use pre-created programs" ( da Silva et al. 2019 ) . A systematic review also found that one of the most frequent disadvantages of AR in mathematics education is the difficulty of developing materials through AR ( Palancı and Turan 2021 ) . An AR application with real-time equation parsing would enable students to author personalized visualizations to support their learning. 2.2. Current Approaches to Mathematics Visualization 2.2.1. Plotting Graphs in 3D Space Existing AR systems have been developed to address the challenges in visualizing equations. GeoGebra provides a system to plot equations without AR on desktops (GeoGebra Desktop) and with AR on mobile (GeoGebra AR) ( Celen 2020 ) . GeoGebra AR is a mobile app that allows users to input equations manually and visualize their corresponding 3D graphs in a mixed-reality space with AR. Researchers evaluated the effectiveness of this app in aiding students and found the system to be practical and effective, resulting in students having reported improved visual-spatial understanding after using the tool ( Mailizar et al. 2021 ) . However, in this study, the small size of smartphone screens was cited as a limitation, as users could not see much detail in the augmented graphs, suggesting a lack of ease of use ( Mailizar et al. 2021 ) . Alternatively, GeoGebra Desktop is a widely adopted desktop app that allows users to manually input equations and visualize 3D equations on a traditional computer screen ( Celen 2020 ) . However, in a previous study, students reported low confidence in using GeoGebra Desktop, which was hypothesized to be due to insufficient time dedicated to learning keystrokes ( Shadaan and Leong 2013 ) . Equation parsing would eliminate the need for learning keystrokes. 2.2.2. Equation Parsing in AR To address this problem, researchers recently developed an AR interface for mathematical equations adding OCR input ( Chulpongsatorn et al. 2023 ) . The interface allows users to point a mobile device camera at an equation in a textbook, which the system parses and generates an interactive plot which is then overlaid on the device’s display. This study found that such a system created a more engaging learning experience for students compared to traditional approaches and