Navigating Speech Recording Collections with AI-Generated Illustrations Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research a
Sirina Håland, Trond Karlsen Strøm, Petra Galuščáková · 2025
Read Navigating Speech Recording Collections with AI-Generated Illustrations Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research a by Sirina Håland, Trond Karlsen Strøm, Petra Galuščáková (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-04182.txt.
Introduction
Sirina Håland, Trond Karlsen Strøm, and Petra Galuščáková published this paper in 2025. It proposes a new way to browse speech archives: with pictures and mind maps instead of keyword lists. Recorded talks pile up faster than anyone can search them by transcript alone.
Their web application takes a speech collection, here over two thousand TED talks, and organizes it into interactive mind maps. Language models summarize and connect topics; image generators illustrate the nodes, giving browsers a visual scent to follow. Users wander the map, drilling from theme to talk, instead of guessing search terms. Early user tests with a standard usability questionnaire suggested the approach makes exploring large collections easier and more inviting. The authors present the system as a complement to search, not a replacement. Speech archives in libraries and media houses, they argue, need navigation designed for curiosity, not only for known-item retrieval.
Catalog record
| CREATOR | Sirina Håland, Trond Karlsen Strøm, Petra Galuščáková |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-04182
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Navigating Speech Recording Collections with AI-Generated Illustrations Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’25), July 13–18, 2025, Padua, Italy DOI: 10.1145/3726302.3730136 ISBN: 979-8-4007-1592-1/2025/07 CCS: Information systems Web interfaces CCS: Information systems Search interfaces CCS: Information systems Speech / audio search Sirina Håland, Trond Karlsen Strøm, Petra Galuščáková Abstract . Although the amount of available spoken content is steadily increasing, extracting information and knowledge from speech recordings remains challenging. Beyond enhancing traditional information retrieval methods such as speech search and keyword spotting, novel approaches for navigating and searching spoken content need to be explored and developed. In this paper, we propose a novel navigational method for speech archives that leverages recent advances in language and multimodal generative models. We demonstrate our approach with a Web application that organizes data into a structured format using interactive mind maps and image generation tools. The system is implemented using the TED-LIUM 3 dataset, which comprises over 2,000 speech transcripts and audio files of TED Talks. Initial user tests using a System Usability Scale (SUS) questionnaire indicate the application’s potential to simplify the exploration of large speech collections. 1. Introduction A large volume of digital spoken content has accumulated online, and diverse speech media collections such as podcasts, talks, and radio programs continue to emerge. Such speech media collections contain a wealth of knowledge and insights, but their sheer volume makes the information useless unless spoken audio can be effectively browsed and searched. Although speech recording remains one of the easiest ways to store information, acquiring information from speech collections remains problematic due to the unique properties of audio content, which is sequential and time-based. Users are also becoming increasingly familiar with the concept of speech recordings as an information source, leading to a growing expectation that access to speech media should be as intuitive, reliable, and convenient as access to conventional text media ( Larson and Jones 2012 ) . In this work, we aim to create a novel navigational approach for spoken collections using recent advances in generative multimodal models. Our proposed method focuses on exploratory search and zero-shot recommendation. We assume a scenario in which the user seeks to listen to recordings discussing topics of interest. We focus on a domain of informative speech recordings, such as popular science talks and podcasts, or courses and conference recordings. A user would like to quickly review the most important topics across all available recordings, select the topics according to their interests, and listen to the recordings discussing such topics. This scenario might serve as an alternative to the more common approaches where users either select recordings purely based on metadata provided by the author of the recording or rely on recommendations based on history and popularity. Our method can lead to more serendipitous discoveries and new insights than these methods and has the potential to be more intuitive, interactive, and user-friendly than these methods. Our method is also complementary to the methods based on keywords search. Our focus in this paper is the proposal and evaluation of this novel navigational method, which helps users explore spoken collections and discover new information. Visual representations of the content and graph-based organization of the recordings can help users gain a broader and faster understanding of the topics covered in the speech collection and assist them in finding potentially interesting recordings in a fun and interactive way. We further describe our demo implementation 1 1 1 https://mindmap.ux.uis.no which uses a collection of TED talks. The source code is freely available 2 2 2 https://github.com/SirinaHaaland/mind-map , and the methods can be applied to other similar collections. 2. Related Work Research in Human-Computer Interaction and Information Retrieval has addressed some of the challenges in navigating speech media collections. Core approaches use Automatic Speech Recognition (ASR) to convert speech into text, enabling further analysis and retrieval. Larson and Jones 2012 emphasize the need for advanced search and retrieval techniques in multimedia content, including speech media, to enhance discoverability. Their work underscores the importance of ASR as a foundational element in this domain. The TED-LIUM dataset, often used in ASR research, illustrates such efforts to enhance speech recognition accuracy ( Rousseau et al. 2014 ; Hernandez et al. 2018 ) . Research on interactive visualization techniques for multimedia content, such as the work by Shamma et al. 2007 , then explores how visual interfaces can support the discovery and exploration of large media collections, addressing both information overload and engagement. Specific for podcasts, Park et al. 2024 propose a novel browsing method that uses Large Language Models (LLMs) for topic segmentation and integrates generative AI-based image cues to enhance navigation. Their study shows that segmentation and visual cues improve user comprehension and ease of navigation in podcast content compared to traditional keyword searches. Expanding on the integration of AI and multimedia, Xia 2020 introduce Crosscast, an automated system that leverages NLP and text mining techniques to identify geographic locations and descriptive keywords in podcast transcripts to synchronize relevant visual content with the audio narration. Their user study demonstrates that the majority of participants preferred Crosscast-generated audio-visual podcasts over traditional audio-only formats, highlighting the potential of combining audio with visuals to enhance user engagement and understanding. Building on this direction, we apply generative models to a broader range of topics. Additionally, Li et al. 2021 present a system that generates hierarchical summaries of long-form spoken dialogues, such as interviews and podcasts, using advanced ASR and Natural Language Processing techniques. Their interface allows users to choose from varying levels of detail—short, medium, or long summaries—alongside the original ASR transcript and audio, making it easier to quickly comprehend and navigate dense spoken content according to their needs. In addition, Ghazimatin et al. 2024 focus on simplifying the structure of long recordings by automatized segmentation into chapters. In contrast to these works, we focus on visualizing the entire recording to provide an easily comprehensible overview. Visualization of the retrieved text documents was studied by Ganguly et al. 2013 , who use colored distributions of the document topics. Recent advancements in video retrieval technologies are then being demonstrated in the annual Video Browser Showdown (VBS) 3 3 3 https://videobrowsershowdown.org/ . Tools such as VISIONE 5.0 ( Amato et al. 2024 ) , Vibro ( Schall et al. 2024 ) , PraK ( Lokoč et al. 2024 ) , and DiveXplore ( Schoeffmann and Nasirihaghighi 2024 ) offer insights into how AI models and interactive systems can enhance multimedia content exploration. 3. Interface Overview and Architecture We aim to address several challenges important for quality exploratory navigation in a collection of speech recordings, such as accessibility, reduced information overload, interactivity, and user engagement. To do this, we develop interactive visual graphs, referred to as Mind Maps, that guide users through the dataset. Visual representations of the content, created automatically based on the transcripts, provide a quick overview of the topic without requiring users to read or listen to the recording. This is especially important for improving accessibility and enhancing user engagement, which is particularly challenging for speech