Intent Lenses: Inferring Capture-Time Intent to Transform Opportunistic Photo Captures into Structured Visual Notes
Ashwin Ram, Aeneas Leon Sommer, Martin Schmitz, Jürgen Steimle · 2026
Read Intent Lenses: Inferring Capture-Time Intent to Transform Opportunistic Photo Captures into Structured Visual Notes by Ashwin Ram, Aeneas Leon Sommer, Martin Schmitz, Jürgen Steimle (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-09438.txt.
Introduction
Ram, Sommer, Schmitz, and Steimle introduce Intent Lenses for turning opportunistic photos into structured visual notes. Quick photos of slides, exhibits, or artifacts preserve information but rarely become meaningful notes. Existing automatic note generation often yields generic summaries missing what users meant to capture. Intent Lenses reify inferred capture-time intent into reusable objects encoding function, sources, and detail level. Built with large language models for academic conference photos on a spatial canvas, the system was tested with nine academics, whose notes matched expectations and aided deeper sensemaking.
Catalog record
| CREATOR | Ashwin Ram, Aeneas Leon Sommer, Martin Schmitz, Jürgen Steimle |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77598 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.09438
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Intent Lenses: Inferring Capture-Time Intent to Transform Opportunistic Photo Captures into Structured Visual Notes Ashwin Ram, Aeneas Leon Sommer, Martin Schmitz, Jürgen Steimle Abstract . Opportunistic photo capture (e.g., slides, exhibits, or artifacts) is a common strategy for preserving information encountered in information-rich environments for later revisitation. While fast and minimally disruptive, such photo collections rarely become meaningful notes. Existing automatic note-generation approaches provide some support but often produce generic summaries that fail to reflect what users intended to capture. We introduce Intent Lenses , a conceptual primitive for intent-mediated note generation and sensemaking. Intent Lenses reify users’ capture-time intent inferred from captured information into reusable interactive objects that encode the function to perform, the information sources to focus on, and how results are represented at an appropriate level of detail. These lenses are dynamically generated using the reasoning capabilities of large language models. To investigate this concept, we instantiate Intent Lenses in the context of academic conference photos and present an interactive system that infers lenses from presentation captures to generate structured visual notes on a spatial canvas. Users can further add, link, and arrange lenses across captures to support exploration and sensemaking. A study with nine academics showed that intent-mediated notes aligned with users’ expectations, providing effective overviews of their captures while facilitating deeper sensemaking. Figure 1 . Intent Lenses are reusable interactive representations of users’ capture-time intent, inferred from opportunistic photo captures in information-rich settings such as academic conferences. These lenses automatically transform captured photos into structured visual notes that align with what users intended to capture. Users can further treat inferred lenses as reusable objects that can be reapplied to presentations (e.g.. generating a related work lens), extended through custom lenses (applications lens), and spatially arranged to support note generation, exploration, and sensemaking across captures. 1. Introduction Capturing photos has become a ubiquitous strategy for taking notes of information encountered in physical, information-rich and fast-paced environments, such as conferences, lectures, exhibitions, or trade-shows. Attendees of such environments must simultaneously perform demanding cognitive activities, such as listen to speakers, interpret visual content, engage in conversations, and move between sessions. Manual note-taking can divert attention from these activities, forcing a trade-off between engaging in the moment and documenting information for later use. In contrast, capturing photos of slides, posters or exhibits is fast and minimally disruptive, supporting a capture now, make sense later workflow ( 47 ; 39 ) . Despite the prevalence of this behavior, our formative study with four academics found that such photo collections rarely become usable notes that users can revisit, interpret, or synthesize into meaningful knowledge. This reflects two intrinsic challenges of opportunistic captures: First, captured content is inherently sparse. A photo of a slide or exhibit lacks the narration, sequential context, and surrounding discourse that give it meaning. Second, the intent behind each capture (what the user found significant and worth returning to) remains implicit in the act of taking the photo. Recent advances in AI have contributed ways to automatically generate notes from captured content, particularly for lecture videos and online meetings ( 43 ) , and such approaches could in principle be applied to photo collections. However, the resulting notes are often generic, focusing on describing visible content rather than capturing what users found meaningful. To address this limitation, more recent work incorporates user intent into the note generation process, either through explicit input during capture (e.g., micro-notes) ( 15 ) or through retrospective prompting and querying ( 27 ) . However, in practice, users rarely articulate intent during mobile capture, as the act is fast and opportunistic, and retrospectively eliciting intents across many photos is cognitively demanding and often inaccurate. We propose treating capture-time intent not as something that users must declare, but as something that can be inferred from contextual captures. For example, when a person photographs a slide during a talk, they are selectively capturing something they anticipate revisiting, such as a key method, an intriguing result, or an idea for future work. Inferring these implicit intents can help generate notes that better align with what users intended to capture. Furthermore, because many such intents recur across multiple captures, we argue that intent should persist beyond one-time note generation as a reusable level of interaction through which users can explore, compare, and synthesize captured information over time. In this work, we explore how such intent (and other relevant contextual data) can be inferred from sparse captures and operationalized to support more meaningful note generation and sensemaking. Building on this perspective, we introduce Intent Lenses , a conceptual primitive for reifying implicit intents inferred from contextual captures. Each intent lens encodes a particular way of engaging with the captured information by specifying the function to perform, the information sources to focus on, and how the output is visually represented at an appropriate level of detail. Applying different lenses to the same underlying data surfaces different patterns and relationships, much like optical lenses that bring certain features into focus while leaving others in the background. In this way, Intent Lenses act as persistent, reusable operators over data that generate content aligned with users’ intent. Intent Lenses build on prior work that represents LLM interactions as user-specified objects ( 19 ; 30 ) , but crucially differ in that they are dynamically generated from contextual captures, allowing interpretations to emerge from data rather than explicit user input. To investigate and evaluate this concept concretely, we focus on a domain where opportunistic photo capture is well-established and its limitations are acutely felt: academic conference attendance. Based on findings from the formative user study, we developed an interactive system that infers Intent Lenses from conference photo captures and uses them to automatically generate intent-mediated visual notes ( Figure 1 ). The system further augments captured photos with additional context such as the corresponding digital article when available, providing richer signals for lens-based interpretation. Users can further create, link, and arrange lenses across multiple presentations, constructing evolving visual notes that support exploration, comparison, and synthesis. By reusing lenses across different presentations, users can generate consistent views of information, surface relationships across talks, and externalize their sensemaking process. We evaluated the prototype through a user study with nine academics from different scientific disciplines. Our findings show that Intent Lenses facilitated the generation of notes that aligned with what users expected to capture, providing meaningful overviews of their conference experiences while supporting deeper post-hoc sensemaking. In summary, we contribute: • A formative study of photo-based note-taking at academic conferences, identifying key challenges in revisiting opportunistic captures and common user intents behind them. • The concept of Intent Lenses as a primitive for reifying implicit intent from contextual photo captures. We present a technical realization that infers capture-time intent from sparse captures using multimodal structured reasoning and operationalize it as reusable prompt-based representations. • An interactive system that instantiates Intent Lenses on a spatial canvas, enabling users to generate, apply, and compose lenses over captured artifacts to create structured, revisitable visual notes. • Evaluation results providing insights into how Intent Lenses support note generation, exploration, and sensemaking, along with design implications for future intent-mediated systems. 2. Related Work 2.1. Approaches and Tools for Note-Taking HCI research