Interaction Design with Generative AI: An Empirical Study of Emerging Strategies Across the Four Phases of Design CCS: Human-centered computing Empirical studies in interaction design CCS: Human-centered computing User centered design CCS: Human-centered computing Empirical studies in HCI CCS: Human
Marie Muehlhaus, Jürgen Steimle · 2024
Read Interaction Design with Generative AI: An Empirical Study of Emerging Strategies Across the Four Phases of Design CCS: Human-centered computing Empirical studies in interaction design CCS: Human-centered computing User centered design CCS: Human-centered computing Empirical studies in HCI CCS: Human by Marie Muehlhaus, Jürgen Steimle (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-02662.txt.
Description
Abstract . Generative Artificial Intelligence (Generative AI) holds significant promise in reshaping interactive systems design, yet its potential across the four key phases of human-centered design remains underexplored. This article addresses this gap by investigating how Generative AI contributes to requirements elicitation, conceptual design, physical design, and evaluation.
Catalog record
| CREATOR | Marie Muehlhaus, Jürgen Steimle |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 84 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-02662
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Interaction Design with Generative AI: An Empirical Study of Emerging Strategies Across the Four Phases of Design CCS: Human-centered computing Empirical studies in interaction design CCS: Human-centered computing User centered design CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing Natural language interfaces Marie Muehlhaus, Jürgen Steimle Abstract . Generative Artificial Intelligence (Generative AI) holds significant promise in reshaping interactive systems design, yet its potential across the four key phases of human-centered design remains underexplored. This article addresses this gap by investigating how Generative AI contributes to requirements elicitation, conceptual design, physical design, and evaluation. Based on empirical findings from a comprehensive eight-week study, we provide detailed empirical accounts and comparisons of successful strategies for diverse design activities across all key phases, along with recurring prompting patterns and challenges faced. Our results demonstrate that Generative AI can successfully support the designer in all key phases, but the generated outcomes require manual quality assessments. Further, our analysis revealed that the successful prompting patterns used to create or evaluate outcomes of design activities require different structures depending on the phase of the design and the specific design activity. We derive implications for designers and future tools that support interaction design with Generative AI. 1. Introduction The user-centered design cycle is commonly modeled in four key phases, comprising requirement elicitation, conceptual and physical design, and evaluation (ISO 9241-210:2019). Thus far, human-centered design in large parts has been an activity performed by humans with humans, requiring considerable manual effort, time, resources, and methodological expertise. Recent advances in Generative Artificial Intelligence (Generative AI) hold promise to support the design process in ways that were solely attributed to humans so far. Of note, now widely available and easily accessible Large Language Models (LLMs) like ChatGPT ( Brown et al. 2020 ) , Bard 1 1 1 Google Bard: https://bard.google.com/chat (accessed 02/05/2024) , and Copilot 2 2 2 Microsoft Copilot: https://copilot.microsoft.com/ (accessed 02/05/2024) , encode both general and domain-specific knowledge that can be relevant for design processes, while powerful image-generation models like DALL-E ( Ramesh et al. 2021 ) and Midjourney 3 3 3 Midjourney: https://www.midjourney.com/ (accessed 02/05/2024) hold promise for visual design tasks. Recent work highlights the potential of Generative AI to reshape the traditional human-centered design process ( Schmidt 2023 ; Schmidt et al. 2024 ) . It was suggested that Generative AI might complement the design process, extend the range of existing methods for human-centered design, and in parts might ultimately even replace costly human work ( Schmidt et al. 2024 ) . Initial work started exploring Generative AI’s potential for specific design activities, such as for persona creation ( Goel et al. 2023 ) , or generating synthetic user data ( Hämäläinen et al. 2023 ) . However, only very little work has investigated how Generative AI tools can be used for interaction design from a holistic and more practical perspective. This aligns with prior work that highlights the difficulties of prompt design to steer the generated outputs and points to a lack of systematic research from an HCI perspective ( Dang et al. 2022 ) . Specifically, we currently lack a comprehensive understanding of Generative AI’s potential across the key phases of the design process, alongside a systematic investigation of successful usage strategies and comparison of how these strategies might be different or similar in different design phases. This is crucial to guide more targeted use of Generative AI in interaction design. This article presents empirical insights from a systematic exploration of Generative AI’s potential in the user-centered design cycle. These allow us to systematically compare different strategies and prompting patterns of Generative AI across the different phases of design. We conducted an eight-week study (n=10) within a graduate-level human-computer interaction course, in which participants used GPT-4 and DALL-E3 to explore how Generative AI can support interaction design for a given design task. The participants engaged in diverse design activities in a systematic procedure, covering the four design phases, and documented successful and failed approaches of how they used Generative AI. We evaluate selected AI-generated outcomes or artifacts through another user study (n=7), in which a new pool of participants was tasked to rate the quality, adequacy of provided detail, and relevance to the given design task. A qualitative analysis revealed that Generative AI offered helpful support to designers across all phases of user-centered design, spanning such versatile activities as analyzing usage contexts, creating design solutions and performing critical assessments not only of design solutions but also of evaluation methods. For each of them, we report on successful strategies identified by our participants. These strategies cover versatile design activities across different design phases, for instance, demonstrating how Generative AI was used to create a set of elaborated personas or to evaluate an experimental design. We further extend beyond prior work by providing in-depth empirical accounts of successful prompting patterns, including their key elements, example inputs and outputs, and by discussing the quality of the provided results. We conclude by discussing remaining challenges and implications for designers, for future tools and for the role of Generative AI in interaction design. The key findings of this article are: • Generative AI can effectively support the designer across various design activities and phases to create artifacts for design activities from scratch, to iteratively refine them, or to gather feedback on existing artifacts. Yet, the generated artifacts require manual refinement. Based on these insights, we propose a novel meta-design cycle that interleaves rapid AI-driven iterations and human guidance (see Figure 7 ). • We provide detailed empirical accounts and practical insights of successful examples for the requirements analysis, design and evaluation phases of user-centered design. We further discuss the AI-generated artifacts by evaluating their quality, the adequacy of the level of detail provided, and their relevance to the given design task. • We identify recurring prompting patterns of successful strategies. Our results reveal that design activities in earlier design phases benefited from single and oftentimes highly structured prompts, whereas iterative prompting proved helpful for advanced design phases. To gather feedback on existing artifacts, participants deployed single prompts that provided minimal additional information. Persona patterns ( White et al. 2023 ) were deployed throughout various design phases, but with different purposes depending on the respective phase. • We identify challenges that our participants faced across the key phases of design, comprising stereotypical representation of users and scenarios in the early design phases, difficulties in generating functional designs and conveying technical or subjective information relevant to later design phases, and ethical concerns in using commercial Generative AI tools in interaction design. This research offers practical insights into the use of Generative AI in user-centered design. We envision our work to enrich our understanding of the potential of Generative AI and promote more effective and innovative design practices in interactive systems–a relevant consideration, given the rapid advancement of Generative AI capabilities and the need for a timely understanding of its potential impacts and risks. Thereby, this paper is targeted at designers, developers of AI tools, and researchers interested in practical applications of Generative AI. 2. Related Work This work is motivated by prior work on AI to support creativity and co-creation, the rise of Generative AI and LLMs,