Partnering with Generative AI: Experimental Evaluation of Human-Led and Model-Led Interaction in Human-AI Co-Creation CCS: Human-centered computing Empirical studies in HCI CCS: Applied computing Arts and humanities CCS: Human-centered computing Empirical studies in HCI
Sebastian Maier, Manuel Schneider, Stefan Feuerriegel · 2025
Read Partnering with Generative AI: Experimental Evaluation of Human-Led and Model-Led Interaction in Human-AI Co-Creation CCS: Human-centered computing Empirical studies in HCI CCS: Applied computing Arts and humanities CCS: Human-centered computing Empirical studies in HCI by Sebastian Maier, Manuel Schneider, Stefan Feuerriegel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-23324.txt.
Introduction
Sebastian Maier, Manuel Schneider, and Stefan Feuerriegel run a randomized experiment on how people co-create with generative AI under different interaction modes. Language models can support creative work, but interface design may decide who leads. They compare two reflective, human-led modes, where the model prompts elaboration with suggestions or questions, against a proactive model-led mode that rewrites ideas on its own.
With four hundred eighty six participants, they assess idea quality and diversity across modes. The results show how much initiative to give the model. Designers of creative tools can use the evidence to balance human control against AI drive.
Catalog record
| CREATOR | Sebastian Maier, Manuel Schneider, Stefan Feuerriegel |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-23324
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Partnering with Generative AI: Experimental Evaluation of Human-Led and Model-Led Interaction in Human-AI Co-Creation CCS: Human-centered computing Empirical studies in HCI CCS: Applied computing Arts and humanities CCS: Human-centered computing Empirical studies in HCI Sebastian Maier, Manuel Schneider, Stefan Feuerriegel Abstract . Large language models (LLMs) show strong potential to support creative tasks, but the role of the interface design is poorly understood. In particular, the effect of different modes of collaboration between humans and LLMs on co-creation outcomes is unclear. To test this, we conducted a randomized controlled experiment ( N = 486 N=486 ) comparing: (a) two variants of reflective, human-led modes in which the LLM elicits elaboration through suggestions or questions, against (b) a proactive, model-led mode in which the LLM independently rewrites ideas. By assessing the effects on idea quality, diversity, and perceived ownership, we found that the model-led mode substantially improved idea quality but reduced idea diversity and users’ perceived idea ownership. The reflective, human-led mode also improved idea quality, yet while preserving diversity and ownership. Our findings highlight the importance of designing interactions with generative AI systems as reflective thought partners that complement human strengths and augment creative processes. Study procedure with five conditions, refinement, and evaluation. Figure 1. Overview of the study procedure. (A) Participants ( N=486 ) were randomly assigned to one of five conditions: four LLM variants (vanilla, model-led, suggestion, question) or a control group without support. (B) Before submitting their final ideas, all participants were instructed to refine their idea using the LLM. (C) Finally, idea quality was assessed by expert raters with domain expertise in the automotive industry; idea diversity was measured via text-embedding similarity of idea descriptions; and perceived ownership was self-reported by participants for the ideas they generated. Study procedure with five conditions, refinement, and evaluation. The figure illustrates the study procedure. Panel A shows five experimental conditions in which participants first write down an initial idea. In the control condition, they receive no support. In the vanilla condition, the initial idea is directly entered into the system. In the model-led condition, the system requires additional input from the participant. In the suggestion-mode condition, the system provides multiple suggestions. In the question-mode condition, the system asks the user questions. Panel B shows that all participants can then refine their idea before submitting it. Panel C shows the evaluation stage, where ideas are assessed for diversity, quality, and perceived ownership. 1. Introduction Large language models (LLMs) have opened new opportunities for supporting creative work ( Lee and Chung 2024 ; Doshi and Hauser 2024 ; Hubert et al. 2024 ; Feuerriegel et al. 2024 ) . The ability of LLMs to generate meaningful text enables writers to co-develop storylines, entrepreneurs to brainstorm novel business directions, and designers to expand early-stage ideas. Unlike earlier creativity support tools, LLMs allow for interaction in natural language, making them well-suited for iterative exploration and collaboration in creative processes. Using LLMs as co-creation tools holds promise for boosting creative outcomes, yet the benefit of LLMs is contested due to three tensions. First , there are ongoing discussions about whether LLMs are truly “creative” ( Chakrabarty et al. 2024 ) . Unlike humans, LLMs lack tacit knowledge and domain expertise, which are often essential for coming up with creative ideas of high quality , that is, ideas that are useful and contextually appropriate ( Baer 2015 ; Lin and Li 2022 ) . Hence, co-creation systems are especially powerful when they complement, rather than replace, human capabilities via interactive collaboration ( Rafner et al. 2023 ) . Second , if reliance on LLMs becomes widespread, the risk emerges that ideas will converge, reducing diversity across creative domains ( Anderson et al. 2024b ; Doshi and Hauser 2024 ; Meincke et al. 2025 ) . However, homogenization threatens one of the core characteristics of creativity—that is, novelty—and raises concerns about cultural and intellectual variety ( Nakadai et al. 2023 ) . Third , while LLMs might support the quality of creative output, their dominant role in shaping ideas may undermine users’ perceived ownership ( Xu et al. 2024 ; Joshi and Vogel 2025 ; Qin et al. 2025 ; Draxler et al. 2024 ; Li et al. 2024 ; Mirowski et al. 2023 ) . However, undermining users’ perceived ownership is closely tied to lower user satisfaction and output valuation ( Krauss and Wienrich 2025 ) . Here, we argue that the three tensions can be addressed through careful interaction design by focusing on the way humans are involved in the process. While existing studies on LLMs in creative contexts have examined the creativity abilities of LLMs (see Section 2 and the overview paper in ( Holzner et al. 2025 ) ), the role of the interaction design is largely ignored. In particular, existing empirical studies focus on a simple interaction format (”one-shot“) ( Holzner et al. 2025 ; Hubert et al. 2024 ; Koivisto and Grassini 2023 ) , where the input to the model is limited to a single-prompt, so that iterative refinements are ignored. Yet, creativity theory stipulates that creativity is not a one-off act but an ongoing, cyclical process of generating, evaluating, and refining ideas ( Wallas 1926 ; Guilford 1967 ; Frich et al. 2019 ) . Hence, it is natural that also LLMs for co-creation should be designed as iterative thought partners that scaffold human reflection rather than as autonomous creators ( Rafner et al. 2023 ) . Motivated by this, we stipulate that LLMs can be integrated in two main ways: (1) model-led, with the model taking initiative by rewriting or expanding ideas, or (2) human-led, with the model eliciting elaboration through questions or suggestions while keeping humans in control of the creative process. Yet, empirical evidence on how different human-led vs. model-led interaction designs and thus general HCI recommendations for such co-creativity systems are lacking. Hence, we explore how different designs may support perceived idea ownership, leverage idea quality, and preserve idea diversity in creative processes. Research Question. How does the interaction mode with LLMs (human-led vs. model-led) influence the quality, diversity, and ownership of ideas in co-creative tasks? To address this research question, we conducted a pre-registered randomized controlled experiment ( N=486 ). In particular, we compared five different conditions: (1) a human-led question-mode , where the LLM asked reflective questions that should trigger human thinking; (2) a human-led suggestion-mode , where the LLM provided tips to refine the idea; (3) a model-led mode in which LLMs proactively refined participants’ ideas; (4) a vanilla LLM , with the default behavior of the LLM that was left unchanged; and (5) a control group without any LLM support. Participants were asked to generate creative ideas using an established task, namely, the Alternative Uses Test (AUT) ( Guilford 1967 ) , which involves repurposing existing objects (i.e., finding new applications for car sensors). Afterwards, ideas were evaluated along three dimensions: idea quality, idea diversity, and perceived idea ownership. Our research makes the following contributions: (1) Empirically, we provide the first large-scale controlled experiment ( N=486 ) directly comparing model-led and human-led interaction modes with LLMs in a complex creativity task, which shows that model-led idea refinement improves idea quality