Rambler in the Wild: A Diary Study of LLM-Assisted Writing With Speech

Xuyu Yang, Wengxi Li, Matthew G. Lee, Zhuoyang Li, J.D. Zamfirescu-Pereira, Can Liu · 2025

Read Rambler in the Wild: A Diary Study of LLM-Assisted Writing With Speech by Xuyu Yang, Wengxi Li, Matthew G. Lee, Zhuoyang Li, J.D. Zamfirescu-Pereira, Can Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05612.txt.

Introduction

Xuyu Yang and five co-authors follow writers who dictate to AI in a 2025 study. Speaking can be faster than typing and kinder to new writers. Tools like Rambler pair dictation with language model cleanup. The team asked how real writers use such a tool day to day.

Twelve academic and creative writers kept diaries for ten days. They drafted blog posts, stories, scripts, and notes by voice. Many started from outlines, then expanded section by section. Others poured out loose thoughts and let the tool sort them. Interviews probed the mental feel and output gains of voice writing. Writers reported new freedom alongside fresh frustrations. The authors draw design lessons for speech based writing tools.

Catalog record

CREATORXuyu Yang, Wengxi Li, Matthew G. Lee, Zhuoyang Li, J.D. Zamfirescu-Pereira, Can Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-05612

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Rambler in the Wild: A Diary Study of LLM-Assisted Writing With Speech Xuyu Yang, Wengxi Li, Matthew G. Lee, Zhuoyang Li, J.D. Zamfirescu-Pereira, Can Liu Abstract . Speech-to-text technologies have been shown to improve text input efficiency and potentially lower the barriers to writing. Recent LLM-assisted dictation tools aim to support writing with speech by bridging the gaps between speaking and traditional writing. This case study reports on the real-world writing experiences of twelve academic or creative writers using one such tool, Rambler, to write various pieces such as blog posts, diaries, screenplays, notes, or fictional stories, etc. Through a ten-day diary study, we identified the participants’ in-context writing strategies using Rambler, such as how they expanded from an outline or organized their loose thoughts for different writing goals. The interviews uncovered the psychological and productivity affordances of writing with speech, pointing to future directions of designing for this writing modality and the utilization of AI support. DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 Conference: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’25), April 26-May 1, 2025, Yokohama, Japan DOI: 10.1145/3706599.3706676 ISBN: 979-8-4007-1395-8/25/04 CCS: Human-centered computing Empirical studies in HCI 1. Introduction In the realm of writing tools, the integration of speech input technology has emerged as a transformative force, offering users an alternative mode of interaction that transcends traditional paper & pen-based or keyboard-based methods ( Lin et al., 2024 ; Ruan et al., 2018 ) . Leveraging speech input to empower writing stems from speech’s inherent ability to bridge the gap between thought and text, allowing users to articulate their ideas naturally and fluidly ( Lin et al., 2024 ; Mehra et al., 2023 ; Crystal, 1995 ; Ruan et al., 2018 ) . Through speech, individuals can bypass the constraints imposed by manual typing, enabling a more direct and expressive form of communication ( Lin et al., 2024 ) . However, challenges such as ambient noise interference, dialectal variations, and inaccuracies in speech recognition systems can hinder the accuracy and dependability of transcriptions, necessitating additional efforts to refine the converted speech text ( Karat et al., 1999 ) . Previous research has explored the use of natural language processing (NLP) and large language models (LLMs) to assist in writing ( Dang et al., 2022 ) and reviewing spoken dialogue ( Li et al., 2021 ; Li et al., 2023 ) , especially in cleaning disfluency and speech recognition errors ( Liao et al., 2023 ; Bassi et al., 2023 ; Tanaka et al., 2018 ) . Users can also shorten, summarize or replace the selected text with LLM suggestions or prompt the model for text generation ( Yang et al., 2022 ) . In addition, LLMs can support macro-level structural revision, which moves beyond automatic text summarization to semantic manipulation with writers in semantic control ( Arnold et al., 2021 ) . For example, Dang et al ( Dang et al., 2022 ) introduces a writing tool that provides on-the-fly paragraph summarization along with the original text. Users could interact with the summaries of paragraphs, such as reorganization via drag and drop, which manipulates the original text in parallel. Our recent research ( Lin et al., 2024 ) presents an LLM-powered graphical user interface, called Rambler, that supports gist-level manipulation of dictated text with two main sets of functions: gist extraction and macro revision. To evaluate the effectiveness of this approach, we conducted a lab study, where 12 participants were asked to compose two articles each, one using Rambler and the other using a baseline (a standard STT editor + ChatGPT). Participants chose from broad writing topics provided by the experimenter for completing the tasks. The findings showed an overwhelming preference for Rambler over the baseline (10 out of 12) and demonstrated diverse writing strategies adopted by participants. However, while a lab study in a controlled environment ensures comparability between conditions, it lacks real-world context, and could not test the middle- or long-term usage of the approach. In our case, since participants were writing content for a lab study, there was low incentive for them to edit it as thoroughly as they might for their own writing in real life. Additionally, there might not be enough time in a one-hour lab study for participants to adopt a new writing paradigm using speech as text input modality. As a follow-up work, we conducted an in-the-wild study to understand the real-world longer-term usage of Rambler for creative writing tasks. The goal was to understand the affordance of writing with speech and its user acceptance as well as how the functions of Rambler could be appropriated into users’ diverse writing habits. Through a few focus groups and a ten-day diary study, participants wrote three articles on their own topics with personal device(s) and in their real-life environment. We report on participants’ writing strategies and workflows led by distinctive writing goals. The surveys and interviews about their experiences provide insights into their incentives and obstacles in adopting the speech modality for writing and the role of AI assistance in the process. Based on the findings, we discuss the implications of designing STT-based writing tools and opportunities for AI support. 2. Background of a Speech-Based Writing Tool – Rambler The speech-to-text tool we used is Rambler ( Lin et al., 2024 ) , which has users dictate their impromptu thoughts, placing each segment of recording into a Ramble . Rambler provides three main functions. First, it supports users in dictation by cleaning up transcripts. Each recording is transcribed by a real-time STT API and then preprocessed with an LLM that automatically cleans up any disfluencies and punctuation errors, completes broken sentences, and smooths transitions in the raw transcript. Users can also use the Respeaking function to dictate new content to replace or add to the existing Ramble (the microphone button in Figure 1 ). Second, it can extract the gists in the text to support visualization and interaction with that text. The keywords of the content are shown automatically after recording, and users can adjust the selection on demand. Based on the gists selection, Rambler provides Semantic Zoom to show multiple summarization levels of each Ramble (the bottom slider). Besides, Rambler provides several LLM-based functions for users to perform macro-revisions on the content. Semantic Split operates on an individual Ramble to divide one Ramble into N Rambles based on its content (the scissors button in Figure 1 ). Semantic Merge merges the contents of multiple selected Rambles into one Ramble (the merge button in Figure 1 ). Custom Magic Prompt opens up the possibility for users to define any custom transformation on Ramble content by directly inputting an LLM prompt (the magic wand button in Figure 1 ). Rambler also provides manual editing functions. Manual Merge concatenates the text of two Rambles into one Ramble (through drag and drop). Manual Split splits a Ramble into two at the cursor position. Users can always drag and drop the Ramble to reorder or

Read the complete text