Conversational Customization of Productivity Systems: A Design Probe of Malleable AI Interfaces Conference: UIST 2026; ISBN: ;
Karthik Sreedhar, Aryan Kaul, Lydia B. Chilton · 2026
Read Conversational Customization of Productivity Systems: A Design Probe of Malleable AI Interfaces Conference: UIST 2026; ISBN: ; by Karthik Sreedhar, Aryan Kaul, Lydia B. Chilton (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11149.txt.
Introduction
Karthik Sreedhar, Aryan Kaul, and Lydia B. Chilton probe chat-based reshaping of work systems through an email system that users reshape in natural language. End-user tweaks have historically stayed rare and confined to setup or breakdown moments, but generative AI now makes such reshaping technically feasible.
Participants reshaped inbox categories, added interface elements, and authored new workflow behaviors by talking to the system over several days in their own workflows. Changes mostly adapted and narrowed existing patterns rather than inventing entirely novel features, and reshaping changed how users engaged with their inboxes. Design teams learn what people actually reshape, when they choose to, and how reshaping changes everyday tool use.
Catalog record
| CREATOR | Karthik Sreedhar, Aryan Kaul, Lydia B. Chilton |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74336 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.11149
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Conversational Customization of Productivity Systems: A Design Probe of Malleable AI Interfaces Conference: UIST 2026; ISBN: ; Karthik Sreedhar, Aryan Kaul, Lydia B. Chilton Abstract . Customization has long been a central goal in interactive systems, yet prior work shows that end-user tailoring occurs infrequently and is often confined to initial setup or moments of breakdown. Recent advances in generative AI suggest that highly malleable systems—where users can modify system behavior through natural language—are now technically feasible. However, it remains unclear how such malleability is used in practice: What kinds of customizations do users create, when do they choose to customize, and how do these modifications shape their experience of everyday tools? We present a design probe that uses a conversationally customizable email system as an instrument to study how users create and refine functionality within everyday tools. The system allows users to iteratively modify their inbox by restructuring categories, introducing interface elements, and authoring new workflow behaviors directly through natural language interaction. We study how participants create, refine, and use these features over several days within their own email workflows. We find that users’ customizations are often grounded in existing patterns, which they adapt and specialize to fit their needs, rather than generating entirely novel functionality. Malleability changes how users engage with their inbox, shifting it from a fixed interface to a flexible data layer shaped through user-authored features. At the same time, customization introduces new forms of risk, including mis-specified behavior, unintended filtering, and uncertainty around outcomes, which users manage through ongoing oversight and refinement. These findings highlight how conversational customization becomes embedded within everyday interaction, and point toward the need for systems that support iterative refinement, visibility into behavior, and safe experimentation as users shape their own tools. 1. Introduction Modern productivity tools increasingly embed AI for tasks like prioritization, categorization, recommendation, and content generation. Foundation models make it possible to interpret and transform messy work artifacts such as emails, documents, and calendar events. New integrations and agentic workflows allow users to express intent in natural language and have systems carry out multi-step actions across applications. This shifts expectations from configuring fixed features to shaping tool behavior in an ongoing, conversational way. Because productivity work is high-frequency and idiosyncratic, even small improvements in workflow fit can compound into meaningful gains in time and attention. As a result, the limiting factor is often not model capability, but whether systems can adapt to a user’s evolving way of working. This creates an opportunity to treat software not as a fixed product, but as a malleable system that users can iteratively make their own. In practice, productivity work is open-ended, evolves over time, and varies substantially across users and contexts. Users frequently encounter “almost-right” system behavior, where a small change or missing capability would make the system significantly more useful. However, existing tools provide little support for lightweight, in-the-moment modification of system behavior. Most systems expose intelligence as fixed features or pre-defined settings that are configured once and rarely revisited. Generative AI introduces a different possibility: systems that are continuously modifiable through natural language. This raises a central question: What happens when users can directly modify the behavior of the systems they use every day? Customization has long been a central goal in HCI, with prior work arguing that systems should adapt to users’ evolving practices rather than require users to conform to fixed assumptions. Research on end-user development, programming-by-demonstration, and tailorable systems emphasizes that meaningful customization must occur during use, not only at design time. Despite this, widespread end-user tailoring remains rare in everyday productivity tools due to barriers such as scripting requirements, abstract configuration interfaces, and detachment from real workflows. Recent generative AI systems lower the technical barrier to customization by enabling users to express changes through natural language. However, most existing systems focus on modifying outputs or presentation, or operate in exploratory environments rather than embedding customization into tools users rely on daily. As a result, while highly malleable systems are now technically feasible, it remains unclear how users engage with them in practice when customization is embedded directly into use. To investigate this design space, we conducted a design probe using a conversationally malleable email system. We chose email because it is a high-frequency, everyday tool that users already rely on extensively, making it a natural setting for observing real customization behavior. The system presents a representation of a user’s inbox augmented with a conversational agent that allows users to modify system behavior itself, not just its outputs. Users can create and refine features such as new categories, interface elements, and workflow behaviors through natural language interaction, with changes occurring incrementally and in context. We worked with eight users over a three-day period, combining guided onboarding with continued use and iterative refinement of user-authored features. On the first day, users proposed and implemented a feature based on their own workflow needs. In subsequent sessions, they reflected on use, refined features, explored others’ features, or proposed new ones. Rather than evaluating a fixed system, we use this probe to study how users shape system behavior through feature creation and use. We find that when customization is directly accessible through conversational interaction, users actively reshape their tools in response to everyday friction. Rather than creating entirely novel functionality, participants grounded their features in existing patterns and iteratively adapted them to fit their workflows. These customizations transformed the inbox from a fixed interface into a flexible data layer, with user-authored features shaping how information is accessed and acted upon. We identify three key patterns. First, users’ ideas for customization are largely shaped by existing examples, which they adapt and specialize to their own workflows. Second, malleability changes how users experience their inbox, shifting it from a fixed interface to a personalized information environment. Third, user-authored functionality introduces new risks, which users manage through oversight, iteration, and selective trust. This work provides empirical and design insights into how users create, refine, and evaluate custom functionality in malleable AI systems. Rather than evaluating system performance, this work uses a system as a design probe to surface how users engage with malleable interfaces in practice. System modification should be treated as a first-class interaction, not an advanced or hidden capability. Conversational interfaces can function as meta-interfaces that allow users to inspect and control system behavior itself. As intelligent systems become more powerful, usability hinges not only on what systems can do, but on how easily users can shape them to fit their own workflows. 2. Related Work 2.1. Customization as Ongoing Adaptation The idea that software should adapt to users has motivated decades of HCI research. Early work on user-tailorable systems argued that systems must remain modifiable during use and developed button-based tailoring mechanisms that allowed users to encode preferred interaction patterns without programming expertise ( 13 ) . Subsequent research found that tailoring is uncommon in everyday practice, tending to cluster around moments of initial setup, breakdown, or environmental change ( 12 ) . A follow-up study two decades later observed that the same barriers persisted, suggesting the problem is not simply technical; interface design itself can either trigger or suppress customization behavior ( 1 ) . End-user programming research has