Expanding the Generative AI Design Space through Structured Prompting and Multimodal Interfaces Conference: ; April 27th, 2025; Yokohama, Japan CCS: Human-centered computing User interface design CCS: Human-centered computing Interaction design CCS: Human-centered computing Empirical studies in HCI

Nimisha Karnatak, Adrien Baranes, Rob Marchant, Huinan Zeng, Tríona Butler, Kristen Olson · 2025

Read Expanding the Generative AI Design Space through Structured Prompting and Multimodal Interfaces Conference: ; April 27th, 2025; Yokohama, Japan CCS: Human-centered computing User interface design CCS: Human-centered computing Interaction design CCS: Human-centered computing Empirical studies in HCI by Nimisha Karnatak, Adrien Baranes, Rob Marchant, Huinan Zeng, Tríona Butler, Kristen Olson (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-14320.txt.

Introduction

Nimisha Karnatak and co-authors build a design tool that helps small business owners create ads with generative AI beyond plain text prompts, in work from twenty twenty-five. Novice users often struggle to put brand feel into words. Output then comes out generic and hard to fine tune.

A study with six United Kingdom business owners named three pain points: stating brand intuition, making fine edits, and getting brand true results. The tool answers with three input panels for branding, audience and goals, and an inspiration board. Users feed context and visual taste in a structured way. The work shows how guided interfaces can lift co creative control for novice designers.

Catalog record

CREATORNimisha Karnatak, Adrien Baranes, Rob Marchant, Huinan Zeng, Tríona Butler, Kristen Olson
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-14320

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Expanding the Generative AI Design Space through Structured Prompting and Multimodal Interfaces Conference: ; April 27th, 2025; Yokohama, Japan CCS: Human-centered computing User interface design CCS: Human-centered computing Interaction design CCS: Human-centered computing Empirical studies in HCI Nimisha Karnatak, Adrien Baranes, Rob Marchant, Huinan Zeng, Tríona Butler, Kristen Olson Abstract . Text-based prompting remains the predominant interaction paradigm in generative AI, yet it often introduces friction for novice users such as small business owners (SBOs), who struggle to articulate creative goals in domain-specific contexts like advertising. Through a formative study with six SBOs in the United Kingdom, we identify three key challenges: difficulties in expressing brand intuition through prompts, limited opportunities for fine-grained adjustment and refinement during and after content generation, and the frequent production of generic content that lacks brand specificity. In response, we present ACAI (AI Co-Creation for Advertising and Inspiration), a multimodal generative AI tool designed to support novice designers by moving beyond traditional prompt interfaces. ACAI features a structured input system composed of three panels: Branding, Audience and Goals, and the Inspiration Board. These inputs allow users to convey brand-relevant context and visual preferences. This work contributes to HCI research on generative systems by showing how structured interfaces can foreground user-defined context, improve alignment, and enhance co-creative control in novice creative workflows. 1. Introduction Small business owners (SBOs) represent the majority of business owners globally ( Nations 2024 ) . Visual advertising and branding are critical to the success of small enterprises; yet, many SBOs lack formal training in design and often do not have access to professional creative services ( Ballard 2016 ; Bhalerao et al. 2022 ) . The growing availability of generative AI systems presents new opportunities to support novice designers such as small business owners by reducing barriers to entry in advertising and branding and by broadening access to creative production tools. To investigate how current generative systems support SBO creative workflows, we conducted a formative study with six small business owners (SBOs) in Manchester, United Kingdom. Using semi-structured interviews, we examined participants’ advertising practices, experiences with generative AI, and expectations for AI-assisted design. Our findings surfaced three recurring challenges: (1) the cognitive effort required to translate abstract, domain-specific goals into usable prompts; (2) dissatisfaction with generic outputs that failed to reflect brand tone, values, or audience needs; and (3) concerns about losing creative control due to over-automation, signaling the need for systems that enable co-creation and preserve user agency. These themes align with ongoing HCI work on promptability ( Morris 2024 ) and the “gulf of envisioning” ( Subramonyam et al. 2024 ) , which highlight how non-expert users struggle to effectively guide generative systems. To address this, we developed ACAI (AI Co-Creation for Advertising and Inspiration), a multimodal generative AI prototype that reimagines the prompt interface for novice designers. ACAI features a structured, panel-based user interface composed of three modules: the Branding & Asset Panel , the Audience & Goals Panel , and the Inspiration Board Panel . These modules enable users to specify brand assets, define campaign objectives, and upload visual references. A multimodal large language model (MLLM) processes these structured inputs into a unified “super prompt” that guides advertisement generation aligned with user-defined goals. This work makes three primary contributions to HCI literature: 1. A formative study surfacing promptability and brand-alignment challenges among SBOs using AI tools. 2. Design requirements emphasizing multimodality and constraint-aware generation. 3. The design of ACAI, a multimodal GenAI-powered prototype. 2. Related Work We draw on two strands of prior research that shape our approach. The discussion below reviews foundational work in these areas and situates our contribution as an extension of ongoing efforts to design inclusive, context-aware generative systems. 2.1. Contextualization in Generative and Interactive Systems Context-aware systems dynamically adapt to user needs by leveraging situational information captured through various modalities, such as GPS, sensors, and APIs ( Dey 2001 ; Chen 2005 ; Aliannejadi et al. 2021 ; Clarizia et al. 2019 ) . These systems have been widely adopted in domains such as mobile assistance and smart environments, where real-time responsiveness is essential. Building on this foundation, recent work has begun integrating context into generative workflows. For example, CARING-AI ( Shi et al. 2025 ) generates spatially and temporally situated AR instructions using real-world environmental data, while ContextCam ( Fan et al. 2024 ) incorporates user-captured context to guide image generation. These systems demonstrate how contextual signals can increase semantic alignment and enhance user engagement. However, current text-to-image generation models primarily rely on text-based prompts and offer limited affordances for encoding structured user intent or domain-specific constraints ( Lee et al. 2023 ) . While these systems cproduce compelling outputs, they often fall short in supporting intention-aligned co-creation, particularly for novice users in task-specific domains. Motivated by these limitations, we position ACAI to bridge the gap between generic text-to-image systems and context-sensitive creative workflows by foregrounding explicit, user-defined contextualization. 2.2. Prompting Challenges and Interface Design for Novice Users A growing body of HCI research has highlighted the barriers novice users face when interacting with prompt-based generative systems. Subramonyam et al. ( Subramonyam et al. 2024 ) identify this challenge as the gulf of envisioning, which refers to a cognitive disconnect between what users intend to create and how they are able to articulate those intentions in forms that a system can interpret. Similarly, Zamfirescu-Pereira et al. ( Zamfirescu-Pereira et al. 2023 ) demonstrate that non-AI experts frequently default to ineffective prompting patterns, revealing a lack of support for articulating goals in ways that models can reliably interpret. To address these challenges, recent work has explored interaction designs that simplify how users articulate intent. PromptCrafter ( Baek et al. 2023 ) introduces a dialogue-driven interface that helps users iteratively refine prompts with LLM support. These efforts reflect a broader shift in HCI toward designing promptable generative experiences, which are particularly effective in lowering entry barriers for novice and non-expert users. Building on this trajectory, our work foregrounds small business owners (SBOs) as a critical yet underrepresented user group in HCI. By designing for their situated creative workflows and contextual constraints, we contribute to ongoing efforts to develop task-aligned generative systems that broaden the inclusivity and usability of promptable interfaces for non-expert users. Figure 1. Formative Study: 6 UK SBOs identify visual elements they would like to incorporate into their branding. 3. Formative Study The last three authors conducted an in-person formative study with small business owners (SBOs) in Manchester, UK, to examine how SBOs perceive, engage with, and evaluate generative AI tools for advertising and brand development. The study aimed to understand existing branding practices, assess the alignment between generative AI value propositions and business needs, and identify adoption barriers. Participants were recruited through a professional research agency using an existing database of individuals open to participating in research. A screener questionnaire was distributed via email to identify eligible participants based on the following inclusion criteria: (1) over 18 years of age, (2) located in or near Manchester, and (3) a small business owner responsible for branding and the online presence of a business with a physical store or studio in the UK. We conducted six in-person, semi-structured interviews, each lasting approximately

Read the complete text